Jump to content
Paolo

PIL module import fails on some Windows computers

Recommended Posts

Posted (edited)

I distribute a plugin written in Python that makes use of the PIL module.

To import the module I use the Marionette function VerifyOrGetLib() as seen in some marionette examples from Marissa Farrell and others.

 

	if platform == 2: #windows
		PIL = 'https://pypi.python.org/packages/bb/f7/d644a9a65a25ff0bf43b7f69ce3ef9803742a5d6e438ddfc7e925c007e6c/Pillow-7.1.2-cp35-cp35m-win_amd64.whl#md5=6d62274af5c88c1cfe31c977f7531ed6'
	else: #Mac
		PIL = 'https://files.pythonhosted.org/packages/7d/b4/755848342213bcb5a5c570657ca21f20ebc104daf15d7847b08e18b4d2fd/Pillow-7.1.2-cp35-cp35m-macosx_10_10_intel.whl'
			
	if Marionette.VerifyOrGetLib('PIL', PIL):

 

Now I am receiving messages from some Windows users saying the installation fails.

Unfortunately I am on a Mac and cannot test it directly.

While the directory Python Externals is correctly created inside the user folder a message is shown saying that the module cannot be downloaded.

But the above link, if copied directly in a browser, indeed download a 2Mb file (Pillow-7.1.2-cp35-cp35m-win_amd64.whl)

 

Here's the code of the VerifyOrGetLib() (contained in the Marionette module)

def VerifyOrGetLib(missingLib, missingLibURL):
	# try to install user's package
	exit_code = False
	appfolder = vs.GetFolderPath(1)
	appuserfldr = vs.GetFolderPath(12)
	
	cmd = 'Python Externals'
	cmd = appuserfldr + cmd
	
	if not os.path.exists(cmd):
		os.makedirs(cmd)
	try:	
		try:
			importlib.import_module(missingLib)
			exit_code = True
		except ImportError:
			#show dialog here
			question = 'Python module ' + missingLib + ' is not installed. Would you like to download and install it now?'
			res = vs. AlertQuestion(question,'', 1,'','','','') 
			if res == 1 :
				if sys.platform == 'win32': 
					piploc = 'Python35\Scripts\pip3.exe'
					pathpip = 'Python35\Scripts'  
					cdpath = os.path.join(appfolder, pathpip)
					cmdpip = os.path.join(appfolder, piploc)
					wd = os.getcwd()  
					if os.path.exists(cmd) and os.path.exists(cmdpip):
						os.chdir(cdpath)
						ret_code = subprocess.call([cmdpip, 'install', missingLibURL, '-t', cmd ])
						if ret_code > 0:
							vs.AlrtDialog('Module ', missingLib, ' cannot be downloaded.') 
							exit_code = False
						else:
							exit_code = True
							sys.path.append(cmd + '\\')
							os.chdir(wd)
							vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.')
				elif  sys.platform == 'darwin':
					import pip
					import shlex
					pos1 = missingLibURL.rfind('/')
					pos2 = missingLibURL.rfind('whl')
					downloadedfile = missingLibURL[pos1+1:pos2] +'whl'
					destination = appuserfldr + downloadedfile
					command_line = "curl -o " + '\"' + destination + '\"' + "  "+ missingLibURL
					args = shlex.split(command_line)
					ret_code = subprocess.call(args)
					if ret_code == 0:
						sys_executable_orig = sys.executable
						sys.executable = '/usr/bin/python'
						ret_code = pip.main(['install', destination,'-q','--target', cmd])
						sys.executable = sys_executable_orig
						if ret_code > 0:
							vs.AlrtDialog('Module ', missingLib, ' cannot be installed.')
							exit_code = False
						else:
							vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.')
							exit_code = True
					else:
						vs.AlrtDialog('Module ', missingLib, ' cannot be installed.')
						exit_code = False
					sys.path.append(cmd+'/')
			else:
				exit_code = False
	except:
		pass	
	return exit_code  

Is someone experiencing the same problem?

Any help is wellcome!

Edited by Paolo
  • Like 1

Share this post


Link to post

Is there someone that can test the marionette node contained in this file (it should install PIL python module in the user folder) on Vectorworks 2020 SP4 on Windows?

 

Download PIL.vwx

 

It worked some time ago, it does not anymore.

Something has changed with the recent SP4 (maybe) and the VerifyOrGetLib() seems to refuse to install python modules.

I have checked (on Mac) and VerifyOrGetLib() is identical to SP3 version (see the code in the first post).

The bug should reside in pip3.exe, since the ret_code is 1 (what does it mean?), see code below.

I have no way to test it by myself, cause I am on Mac (and Mac does not have this problem…)

 

any help?

 

@Marissa Farrell, @JBenghiat, @relume

if sys.platform == 'win32': 
					piploc = 'Python35\Scripts\pip3.exe'
					pathpip = 'Python35\Scripts'  
					cdpath = os.path.join(appfolder, pathpip)
					cmdpip = os.path.join(appfolder, piploc)
					wd = os.getcwd()  
					if os.path.exists(cmd) and os.path.exists(cmdpip):
						os.chdir(cdpath)
						ret_code = subprocess.call([cmdpip, 'install', missingLibURL, '-t', cmd ])
						if ret_code > 0:
							vs.AlrtDialog('Module ', missingLib, ' cannot be downloaded.') 
							exit_code = False
						else:
							exit_code = True
							sys.path.append(cmd + '\\')
							os.chdir(wd)
							vs.AlrtDialog('Module ', missingLib, ' has been successfully installed.')

 

Share this post


Link to post
Posted (edited)

I really wished I seen this thread yesterday... 

 

Something has definitely broken within VerifyOrGetLib / Pip.

 

I was just about to release a new plug in package, and was doing some final testing, and discovered that my external libraries installation script has stopped working for some reason. I've posted my install code below, (very similiar to ops). I should note i'm doing this on windows, haven't tested on Mac.

 

import shutil
import os
import vs
import Marionette
import platform
import sys   

def installOpenSSL():
    Usr = vs.GetFolderPath(-12)
    (major, minor, maintenance, platform, buildNum) = vs.GetVersionEx()
    if platform == 1:
        Path = Usr + 'temp/tempImg.png'
    else:
        Path = Usr + 'temp\\tempImg.png'

    vs.AlrtDialog(str(sys.version))

    module = 'https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl'

    if Marionette.VerifyOrGetLib('OpenSSL', module):
        pass
    else:
        vs.AlrtDialog('Error Installing Module')

installOpenSSL()

 

I tried skipping the Marionette call, in case the issue lay somewhere within there and invoked the subprocess.call directly, which returns Error code 1. To gain some slightly better feedback as to what was going on I swapped to subprocess.Popen. I tried a combination of different pip commands, the below code should list all installed packages.

 

import vs
from collections import OrderedDict
import operator
import copy
import  os, sys, subprocess, urllib.request, importlib, logging
import itertools

appfolder = vs.GetFolderPath(1)
appuserfldr = vs.GetFolderPath(12)
piploc = 'Python35\Scripts\pip3.exe'
pathpip = 'Python35\Scripts'  
cdpath = os.path.join(appfolder, pathpip)
cmdpip = os.path.join(appfolder, piploc)

ret_code = subprocess.Popen([cmdpip, 'pip list'  ], stdout = subprocess.PIPE, stderr=subprocess.STDOUT )
output, err = ret_code.communicate()
vs.AlrtDialog(str(output))
vs.AlrtDialog(str(err))

 

For all commands the error I get is: 

 

  File "D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe", line 1
SyntaxError: Non-UTF-8 code starting with '\x90' in file D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

 

Now i'll be honest just getting this far has stretched my coding knowledge, and I don't have the knowledge to suggest any fixes or solutions from this point. 

 

I also accept there's a high chance my use of subprocess commands may be the reason i'm getting this particular error, however there's no denying somethings broke since SP4

Edited by tbexon
  • Like 1

Share this post


Link to post
11 minutes ago, tbexon said:

 


  File "D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe", line 1
SyntaxError: Non-UTF-8 code starting with '\x90' in file D:\Program Files\Vectorworks 2020\Python35\Scripts\pip3.exe on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

 

 

Double check that the "'pip list'  " you have written as a parameter to subprocess.Popen has no special character.

Share this post


Link to post

Hi @tbexon,

thank you for your thorough investigation, I too ended beyond my knowledge, also I am on Mac and not able to test any solution directly.

So I have ended up suggesting manual installation of modules to my (Windows) plugin users.

Also the Marionette.VerifyOrGetLib bug has been reported, so I hope it will be fixed soon.

 

 

Share this post


Link to post
1 hour ago, Nicolas Goutte said:

 

Double check that the "'pip list'  " you have written as a parameter to subprocess.Popen has no special character.

 

If my understanding of what I read here is correct

 

It should run without any special characters, however I tried "pip install" as well which definitely doesn't require any special characters to run.

Share this post


Link to post

As a work around can anyone think of any reason why adding into the install.py script some code to manually copy what would normally be the final contents of external python packages into the user folder? 

Share this post


Link to post

@Paolo ,

Could you please report the exact number of the build 2020 sp4 you are using? 

Could I have also any available information about windows (version, build), where the problem with pip 3 occurs.

Do you have any other python installation on your PC ? 

I have tested your file Download PIL.vwx with 2020 SP4(551884), everything seems OK.

I have Windows 10 Pro, build 1903.

Please, take a look at the attached video.

18 hours ago, tbexon said:

cmdpip, 'pip list'

 

 

  • Like 1

Share this post


Link to post

Thank you @K.Lalkovski,

as stated in my previous posts, I report problems from Windows users with 2020 SP4.

From my side (2020 SP4 on MacOS) all is working.

I will ask them to send me more details on this and post them as soon as i get them.

Share this post


Link to post

@Paolo If possible let the user  execute the following script from the command line. Goes to Python folder of VW(Python35).
import sys
import locale
print("sys.stdin.encoding = ", sys.stdin.encoding)
print("locale.getpreferredencoding = ",locale.getpreferredencoding())
print("locale.getpreferredencoding = ",locale.getpreferredencoding(False))
print("sys.getdefaultencoding = ",sys.getdefaultencoding())
print("sys.getfilesystemencoding = ",sys.getfilesystemencoding())

Share this post


Link to post

@K.Lalkovski

 

I'm on SP4 Build 550627 - Which I notice is a different build to yourself, I THINK i'm on the most up to date version, as I tried reinstalling VW from a fresh download as part of my fault finding process. 

 

output from script:

 

=== RESTART: D:\Program Files\Vectorworks 2020\Python35\get encoding info.py ===
sys.stdin.encoding =  utf-8
locale.getpreferredencoding =  cp1252
locale.getpreferredencoding =  cp1252
sys.getdefaultencoding =  utf-8
sys.getfilesystemencoding =  utf-8

 

Share this post


Link to post

@tbexon Thank you for the info. Could you please execute this script as well? 

import sys
print("sys.getwindowsversion = ",sys.getwindowsversion())

 

 

Share this post


Link to post

@tbexon Could please try this pip3.exe in the attached archive pip3.zip

1.Goto VW folder\Python35\Scripts

2.rename old pip3.exe to pip3old.exe

3.unzip new pip3.exe.

4.Start VW and try on your machine the script above that was giving the error "SyntaxError: Non-UTF-8 code starting with '\x90'" in file D:\Program Files\Vectorwork". Report the results here.If there is no more such a syntax error try to download PIL for Python 3.5 via Marioentte.VerifyOrgetLib and let me kknow about the result.

pip3.zip

  • Like 1

Share this post


Link to post

@K.Lalkovski Apologies for the delay getting back to you, I've been away from my PC for a while!

 

So I followed your instructions, and found that the new pip3 solved the issue, VerifyorGetLib appears to work once again.

 

To confirm something else hadn't changed in the meantime I swapped back to the old version of pip3 and received the same expected errors as before.

 

Is that a different version of pip from the once distributed within the VW installer? I know I tried completely uninstalling and reinstalling VW early on in my fault finding process!

  • Like 1

Share this post


Link to post

@tbexon Thank you for your reply. It's not about the versions,

I've sent you this pip3 copy just to see if my suspicion of what is going wrong  is correct and my fix is working.

Share this post


Link to post
38 minutes ago, K.Lalkovski said:

@Paolo The pip3 problem is  fixed. You just need a 2021  version with a build number bigger than 560093.

 

Will this fix be applied to a future 2020 update? Or is it exclusive to 2021? I’m aware, especially in these troubling times, a lot of my potential users will still be using 2020 for the foreseeable future, and it would be good if I didn’t have to have a separate installation method for them.

  • Like 1

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


 

7150 Riverwood Drive, Columbia, Maryland 21046, USA   |   Contact Us:   410-290-5114

 

© 2018 Vectorworks, Inc. All Rights Reserved. Vectorworks, Inc. is part of the Nemetschek Group.

×
×
  • Create New...