Paolo Posted June 25, 2020 Share Posted June 25, 2020 (edited) I distribute a plugin written in Python that makes use of the PIL module. To import the module I use the Marionette function VerifyOrGetLib() as seen in some marionette examples from Marissa Farrell and others. if platform == 2: #windows PIL = 'https://pypi.python.org/packages/bb/f7/d644a9a65a25ff0bf43b7f69ce3ef9803742a5d6e438ddfc7e925c007e6c/Pillow-7.1.2-cp35-cp35m-win_amd64.whl#md5=6d62274af5c88c1cfe31c977f7531ed6' else: #Mac PIL = 'https://files.pythonhosted.org/packages/7d/b4/755848342213bcb5a5c570657ca21f20ebc104daf15d7847b08e18b4d2fd/Pillow-7.1.2-cp35-cp35m-macosx_10_10_intel.whl' if Marionette.VerifyOrGetLib('PIL', PIL): Now I am receiving messages from some Windows users saying the installation fails. Unfortunately I am on a Mac and cannot test it directly. While the directory Python Externals is correctly created inside the user folder a message is shown saying that the module cannot be downloaded. But the above link, if copied directly in a browser, indeed download a 2Mb file (Pillow-7.1.2-cp35-cp35m-win_amd64.whl) Here's the code of the VerifyOrGetLib() (contained in the Marionette module) def VerifyOrGetLib(missingLib, missingLibURL): # try to install user's package exit_code = False appfolder = vs.GetFolderPath(1) appuserfldr = vs.GetFolderPath(12) cmd = 'Python Externals' cmd = appuserfldr + cmd if not os.path.exists(cmd): os.makedirs(cmd) try: try: importlib.import_module(missingLib) exit_code = True except ImportError: #show dialog here question = 'Python module ' + missingLib + ' is not installed. Would you like to download and install it now?' res = vs. AlertQuestion(question,'', 1,'','','','') if res == 1 : if sys.platform == 'win32': piploc = 'Python35\Scripts\pip3.exe' pathpip = 'Python35\Scripts' cdpath = os.path.join(appfolder, pathpip) cmdpip = os.path.join(appfolder, piploc) wd = os.getcwd() if os.path.exists(cmd) and os.path.exists(cmdpip): os.chdir(cdpath) ret_code = subprocess.call([cmdpip, 'install', missingLibURL, '-t', cmd ]) if ret_code > 0: vs.AlrtDialog('Module ', missingLib, ' cannot be downloaded.') exit_code = False else: exit_code = True sys.path.append(cmd + '\\') os.chdir(wd) vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.') elif sys.platform == 'darwin': import pip import shlex pos1 = missingLibURL.rfind('/') pos2 = missingLibURL.rfind('whl') downloadedfile = missingLibURL[pos1+1:pos2] +'whl' destination = appuserfldr + downloadedfile command_line = "curl -o " + '\"' + destination + '\"' + " "+ missingLibURL args = shlex.split(command_line) ret_code = subprocess.call(args) if ret_code == 0: sys_executable_orig = sys.executable sys.executable = '/usr/bin/python' ret_code = pip.main(['install', destination,'-q','--target', cmd]) sys.executable = sys_executable_orig if ret_code > 0: vs.AlrtDialog('Module ', missingLib, ' cannot be installed.') exit_code = False else: vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.') exit_code = True else: vs.AlrtDialog('Module ', missingLib, ' cannot be installed.') exit_code = False sys.path.append(cmd+'/') else: exit_code = False except: pass return exit_code Is someone experiencing the same problem? Any help is wellcome! Edited June 25, 2020 by Paolo 1 Quote Link to comment
Paolo Posted June 29, 2020 Author Share Posted June 29, 2020 Is there someone that can test the marionette node contained in this file (it should install PIL python module in the user folder) on Vectorworks 2020 SP4 on Windows? Download PIL.vwx It worked some time ago, it does not anymore. Something has changed with the recent SP4 (maybe) and the VerifyOrGetLib() seems to refuse to install python modules. I have checked (on Mac) and VerifyOrGetLib() is identical to SP3 version (see the code in the first post). The bug should reside in pip3.exe, since the ret_code is 1 (what does it mean?), see code below. I have no way to test it by myself, cause I am on Mac (and Mac does not have this problem…) any help? @Marissa Farrell, @JBenghiat, @relume if sys.platform == 'win32': piploc = 'Python35\Scripts\pip3.exe' pathpip = 'Python35\Scripts' cdpath = os.path.join(appfolder, pathpip) cmdpip = os.path.join(appfolder, piploc) wd = os.getcwd() if os.path.exists(cmd) and os.path.exists(cmdpip): os.chdir(cdpath) ret_code = subprocess.call([cmdpip, 'install', missingLibURL, '-t', cmd ]) if ret_code > 0: vs.AlrtDialog('Module ', missingLib, ' cannot be downloaded.') exit_code = False else: exit_code = True sys.path.append(cmd + '\\') os.chdir(wd) vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.') Quote Link to comment
tbexon Posted July 30, 2020 Share Posted July 30, 2020 (edited) I really wished I seen this thread yesterday... Something has definitely broken within VerifyOrGetLib / Pip. I was just about to release a new plug in package, and was doing some final testing, and discovered that my external libraries installation script has stopped working for some reason. I've posted my install code below, (very similiar to ops). I should note i'm doing this on windows, haven't tested on Mac. import shutil import os import vs import Marionette import platform import sys def installOpenSSL(): Usr = vs.GetFolderPath(-12) (major, minor, maintenance, platform, buildNum) = vs.GetVersionEx() if platform == 1: Path = Usr + 'temp/tempImg.png' else: Path = Usr + 'temp\\tempImg.png' vs.AlrtDialog(str(sys.version)) module = 'https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl' if Marionette.VerifyOrGetLib('OpenSSL', module): pass else: vs.AlrtDialog('Error Installing Module') installOpenSSL() I tried skipping the Marionette call, in case the issue lay somewhere within there and invoked the subprocess.call directly, which returns Error code 1. To gain some slightly better feedback as to what was going on I swapped to subprocess.Popen. I tried a combination of different pip commands, the below code should list all installed packages. import vs from collections import OrderedDict import operator import copy import os, sys, subprocess, urllib.request, importlib, logging import itertools appfolder = vs.GetFolderPath(1) appuserfldr = vs.GetFolderPath(12) piploc = 'Python35\Scripts\pip3.exe' pathpip = 'Python35\Scripts' cdpath = os.path.join(appfolder, pathpip) cmdpip = os.path.join(appfolder, piploc) ret_code = subprocess.Popen([cmdpip, 'pip list' ], stdout = subprocess.PIPE, stderr=subprocess.STDOUT ) output, err = ret_code.communicate() vs.AlrtDialog(str(output)) vs.AlrtDialog(str(err)) For all commands the error I get is: File "D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe", line 1 SyntaxError: Non-UTF-8 code starting with '\x90' in file D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details Now i'll be honest just getting this far has stretched my coding knowledge, and I don't have the knowledge to suggest any fixes or solutions from this point. I also accept there's a high chance my use of subprocess commands may be the reason i'm getting this particular error, however there's no denying somethings broke since SP4 Edited July 30, 2020 by tbexon 1 Quote Link to comment
Nicolas Goutte Posted July 30, 2020 Share Posted July 30, 2020 11 minutes ago, tbexon said: File "D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe", line 1 SyntaxError: Non-UTF-8 code starting with '\x90' in file D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details Double check that the "'pip list' " you have written as a parameter to subprocess.Popen has no special character. Quote Link to comment
Paolo Posted July 30, 2020 Author Share Posted July 30, 2020 Hi @tbexon, thank you for your thorough investigation, I too ended beyond my knowledge, also I am on Mac and not able to test any solution directly. So I have ended up suggesting manual installation of modules to my (Windows) plugin users. Also the Marionette.VerifyOrGetLib bug has been reported, so I hope it will be fixed soon. Quote Link to comment
tbexon Posted July 30, 2020 Share Posted July 30, 2020 1 hour ago, Nicolas Goutte said: Double check that the "'pip list' " you have written as a parameter to subprocess.Popen has no special character. If my understanding of what I read here is correct It should run without any special characters, however I tried "pip install" as well which definitely doesn't require any special characters to run. Quote Link to comment
tbexon Posted July 30, 2020 Share Posted July 30, 2020 As a work around can anyone think of any reason why adding into the install.py script some code to manually copy what would normally be the final contents of external python packages into the user folder? Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted July 31, 2020 Vectorworks, Inc Employee Share Posted July 31, 2020 @Paolo , Could you please report the exact number of the build 2020 sp4 you are using? Could I have also any available information about windows (version, build), where the problem with pip 3 occurs. Do you have any other python installation on your PC ? I have tested your file Download PIL.vwx with 2020 SP4(551884), everything seems OK. I have Windows 10 Pro, build 1903. Please, take a look at the attached video. 18 hours ago, tbexon said: cmdpip, 'pip list' 2020-07-31-12-42-26.mp4 1 Quote Link to comment
Paolo Posted July 31, 2020 Author Share Posted July 31, 2020 Thank you @K.Lalkovski, as stated in my previous posts, I report problems from Windows users with 2020 SP4. From my side (2020 SP4 on MacOS) all is working. I will ask them to send me more details on this and post them as soon as i get them. Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted July 31, 2020 Vectorworks, Inc Employee Share Posted July 31, 2020 @Paolo If possible let the user execute the following script from the command line. Goes to Python folder of VW(Python35). import sys import locale print("sys.stdin.encoding = ", sys.stdin.encoding) print("locale.getpreferredencoding = ",locale.getpreferredencoding()) print("locale.getpreferredencoding = ",locale.getpreferredencoding(False)) print("sys.getdefaultencoding = ",sys.getdefaultencoding()) print("sys.getfilesystemencoding = ",sys.getfilesystemencoding()) Quote Link to comment
tbexon Posted August 1, 2020 Share Posted August 1, 2020 @K.Lalkovski I'm on SP4 Build 550627 - Which I notice is a different build to yourself, I THINK i'm on the most up to date version, as I tried reinstalling VW from a fresh download as part of my fault finding process. output from script: === RESTART: D:\Program Files\Vectorworks 2020\Python35\get encoding info.py === sys.stdin.encoding = utf-8 locale.getpreferredencoding = cp1252 locale.getpreferredencoding = cp1252 sys.getdefaultencoding = utf-8 sys.getfilesystemencoding = utf-8 Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted August 3, 2020 Vectorworks, Inc Employee Share Posted August 3, 2020 @tbexon Thank you for the info. Could you please execute this script as well? import sys print("sys.getwindowsversion = ",sys.getwindowsversion()) Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted August 4, 2020 Vectorworks, Inc Employee Share Posted August 4, 2020 @tbexon Could please try this pip3.exe in the attached archive pip3.zip 1.Goto VW folder\Python35\Scripts 2.rename old pip3.exe to pip3old.exe 3.unzip new pip3.exe. 4.Start VW and try on your machine the script above that was giving the error "SyntaxError: Non-UTF-8 code starting with '\x90'" in file D:\Program Files\Vectorwork". Report the results here.If there is no more such a syntax error try to download PIL for Python 3.5 via Marioentte.VerifyOrgetLib and let me kknow about the result. pip3.zip 1 Quote Link to comment
tbexon Posted August 12, 2020 Share Posted August 12, 2020 @K.Lalkovski Apologies for the delay getting back to you, I've been away from my PC for a while! So I followed your instructions, and found that the new pip3 solved the issue, VerifyorGetLib appears to work once again. To confirm something else hadn't changed in the meantime I swapped back to the old version of pip3 and received the same expected errors as before. Is that a different version of pip from the once distributed within the VW installer? I know I tried completely uninstalling and reinstalling VW early on in my fault finding process! 1 Quote Link to comment
Paolo Posted August 18, 2020 Author Share Posted August 18, 2020 @tbexon, @K.Lalkovski, I know (from a beta tester) that the beta version 2021 bs still has the faulted version of pip3. Do you, @K.Lalkovski, know if this problem is going to be fixed or not? Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted August 18, 2020 Vectorworks, Inc Employee Share Posted August 18, 2020 @Paolo The pip3 problem is fixed. You just need a 2021 version with a build number bigger than 560093. 1 Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted August 18, 2020 Vectorworks, Inc Employee Share Posted August 18, 2020 @tbexon Thank you for your reply. It's not about the versions, I've sent you this pip3 copy just to see if my suspicion of what is going wrong is correct and my fix is working. Quote Link to comment
tbexon Posted August 18, 2020 Share Posted August 18, 2020 38 minutes ago, K.Lalkovski said: @Paolo The pip3 problem is fixed. You just need a 2021 version with a build number bigger than 560093. Will this fix be applied to a future 2020 update? Or is it exclusive to 2021? I’m aware, especially in these troubling times, a lot of my potential users will still be using 2020 for the foreseeable future, and it would be good if I didn’t have to have a separate installation method for them. 1 Quote Link to comment
Vectorworks, Inc Employee K.Lalkovski Posted August 20, 2020 Vectorworks, Inc Employee Share Posted August 20, 2020 @tbexon The fix for pip is now included in VW 2020 SP5. 2 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.