vs.ForEachObject seems slow

[Myke]

Hello,

I was using this script to import json data and update the lights on a plot, and it works. But very slowly VW is unresponsive since the script is still running. Is there a way to optimize this so that the performance is better.

If im not mistaken the ForEach function should be similar to a standard for loop and shouldn't be that bad in terms of time.

 

Also there seems to be an object limit that I've hit, were this script tries to run but then quits with nothing being updated. 200 looks like the limit anything above that and the script just quits.

 

I would appreciate any suggestions.

 

Thanks

import vs
import json
import http.client

instrType = ""
instrMode = ""
wattage = ""
weight = ""		#Added
position = ""
purpose = ""	#Added
channel = ""
unitNumber = ""	#Added
universeAdress = ""
circuitNumber = ""
circuitName = ""
uid = ""
fixtureMode = ""
dmxLine = ""
dmxFootprint = ""
deviceType = ""
color = ""
template1 = ""	#Added
template2 = ""	#Added
user1 = ""
user2 = ""
user3 = ""
user4 = ""
user5 = ""
user6 = ""
className = ""


def eachLight(h):
    vs.SetRField(h, "Lighting Device", "Wattage", wattage)
    vs.SetRField(h, "Lighting Device", "Postion", position)
    vs.SetRField(h, "Lighting Device", "Channel", channel)
    vs.SetRField(h, "Lighting Device", "Purpose", purpose)
    vs.SetRField(h, "Lighting Device", "Weight", weight)
    vs.SetRField(h, "Lighting Device", "Unit Number", unitNumber)
    vs.SetRField(h, "Lighting Device", "UniverseAddress", universeAdress)
    vs.SetRField(h, "Lighting Device", "Circuit Number", circuitNumber)
    vs.SetRField(h, "Lighting Device", "Circuit Name", circuitName)
    vs.SetRField(h, "Lighting Device", "Inst Type", instrType)
    vs.SetRField(h, "Lighting Device", "Fixture Mode", instrMode)
    vs.SetRField(h, "Lighting Device", "DMX Line", dmxLine)
    vs.SetRField(h, "Lighting Device", "num channels", dmxFootprint)
    vs.SetRField(h, "Lighting Device", "Device Type", deviceType)
    vs.SetRField(h, "Lighting Device", "Color", color)
    vs.SetRField(h, "Lighting Device", "Gobo 1", template1)
    vs.SetRField(h, "Lighting Device", "Gobo 2", template2)
    vs.SetRField(h, "Lighting Device", "User Field 1", user1)
    vs.SetRField(h, "Lighting Device", "User Field 2", user2)
    vs.SetRField(h, "Lighting Device", "User Field 3", user3)
    vs.SetRField(h, "Lighting Device", "User Field 4", user4)
    vs.SetRField(h, "Lighting Device", "User Field 5", user5)
    vs.SetRField(h, "Lighting Device", "User Field 6", user6)
    vs.SetClass(h, className)

# Substitute open(path) for GET request with a json.load
try: 
    connection = http.client.HTTPConnection("localhost:29212")
    connection.request("GET", "/VectorworksGet")
    response = connection.getresponse()
    rawData = response.read().decode('utf-8')
    data = json.loads(rawData)
    isAvailable = True;
    connection.close()

except Exception:
    print(Exception.__traceback__)
    isAvailable = False;

if (isAvailable):
    for p in data['LightingDevices']:
        instrType = p['instrumentType']
        instrMode = p['fixtureMode']
        wattage = p['wattage']
        weight = p['weight']
        position = p['position']
        purpose = p['purpose']
        channel = p['channel']
        unitNumber = p['unitNumber']
        universeAdress = p['patch']
        circuitNumber = p['circuitNumber']
        circuitName = p['circuitName']
        dmxLine = p['dmxLine']
        dmxFootprint = p['dmxFootprint']
        deviceType = p['deviceType']
        color = p['color']
        template1 = p['template1']
        template2 = p['template2']
        user1 = p['userField1']
        user2 = p['userField2']
        user3 = p['userField3']
        user4 = p['userField4']
        user5 = p['userField5']
        user6 = p['userField6']
        className = p["class"]
        uid = p['__UID']
        criteria = "(N='" + uid + "')"
        vs.ForEachObject(eachLight, criteria)

 

Edited April 17, 2022 by Myke

[Pat Stanford]

How many lights are you running this on?  SetRField is a fairly slow procedure so since you are doing many fields if you are also doing many lights it could be slow.

 

Have you tried to see if it is the eachLight call or the json load that is slow? Try putting some vs.AlrtDialog(Date(2,2)) commands around the ForEachObject line and you can see the time (to the nearest second) before and then after the script runs. But you will have to hit enter to continue.

 

Put the AlrtDialog calls inside the eachlight code and see if it is taking exceptionally long to write one light. But be careful as you will have to hit enter twice for each light found by the criteria.

 

vs.GetTickCount will give you a longing with 1/60th of a second accuracy from system startup if you want something finer.

 

HTH

[Myke]

For the number of lights, I would like it to scale near infinitely the current limit is 200 as far as I've tested. That's nowhere near enough. It does indeed loop for every light, but there is no change to the fields when above the 200 fixture limit. To solve this I could simply limit the loop so that it updates the first 200 fixtures and then the next 200 and so on. But that increases the time problem that Im having. 

The Json loads fine there is no issue with that its just that there isn't an update at all

 

Hope this sheds some more light

[Pat Stanford]

@Sam JonesI think you have mentioned before that SetRField is slow on lights. Can you offer any guidance?

[Sam Jones]

In my experience, there is no limit for the number of objects the ForEachObject() procedure can handle.  SetRField isn't particularly slow, but VS is interpreted, and you are making 30 assignments of string variables for each of 300 objects.  I have 2 recommendations.  One, go get coffee and/or do the dishes. Or two, try and get your data into .XML files.  Data input with XML files is lightning fast.  They load really fast, and they assign really fast.  However the speed problem is killing you, there shouldn't be a limit on the number of Lighting Devices.

[Myke]

I might have figured out a couple of things.

Json.loads() has a maximum so I am overfilling the buffer and will need to implement a stream of some kind. The other thing I've learned is that the math for the time is something like n^2 so it scales terribly. Basically I need to remove a loop, and parse as few parameters as possible.

 

I do have some questions about updating objects with either vs.SetRField or vs.ForEachObject.
- Where does the symbol UID come into play? If I only wanted to update one symbol I should use the UID generated to specifically point to it?

- Are they're any other attributes that are mandatory to effectively update a symbol such as a Layer and Class.

 

Thanks

[Sam Jones]

Use the UID of the Lighting Device instance which is in the "Name" parameter at the bottom of the OIP.  Doing a GetObject() on that string value will give you a handle to that particular Lighting Device, and then use SetRField() using that handle.  You only need to SetRField() for the parameters you want to change.  All the rest will remain unchanged.  HTH.

[DomC]

Hello
I think the issue with the script that makes it slow is, that you loop the light devices and in that loop you give a search-request with ForEachObject. If you have many objects in your drawing a search-request over all objects in the Drawing could take maybe one second and if you have that 250 times in a loop it will take 250 seconds as example.

The better way in my opinion
1. Use ForEachObject just once on all Objects that could be your target light objects ()
2. Then loop all light devices from data (as you do)
3. Then do it as following:
 

if (isAvailable):       
	lights = [] #name, handle
	def get_lights(h):
		n = vs.GetName(h)
		lights.append([n,h])

	c = "((PON='Lighting Device'))" 
	vs.ForEachObject(get_lights, c)  

	for p in data['LightingDevices']:
		uid = p['__UID']
		for light_name, h in lights: #loop through the found light objects in the drawing
			if light_name == uid:
				instrType = p['instrumentType']
				instrMode = p['fixtureMode']
				wattage = p['wattage']
				weight = p['weight']
				position = p['position']
				purpose = p['purpose']
				channel = p['channel']
				unitNumber = p['unitNumber']
				universeAdress = p['patch']
				circuitNumber = p['circuitNumber']
				circuitName = p['circuitName']
				dmxLine = p['dmxLine']
				dmxFootprint = p['dmxFootprint']
				deviceType = p['deviceType']
				color = p['color']
				template1 = p['template1']
				template2 = p['template2']
				user1 = p['userField1']
				user2 = p['userField2']
				user3 = p['userField3']
				user4 = p['userField4']
				user5 = p['userField5']
				user6 = p['userField6']
				className = p["class"]
				eachLight(h)

The loop in loop will be very fast. Much faster then the search-request (ForEach ...) inside a loop.
Untested code but I bet a beer this would do the job faster

 

Edited April 23, 2022 by DomC

[Myke]

@DomC

That definitely works and it's much faster that the original. I was actually going to test using another method.

 

The idea was to create a lookup table on the app that imports the data from Vectorworks, it would track all of the changes and then send them to VW. The only information that would be contained is the UID and any fields that got changed. That would be the minimum amount of data to import into VW reducing the time complexity significantly.

I'm still in the process of understanding your script but Im also very interested in merging both together to get something very performant. 
It looks something like this, basically I took your set of values and found them in the global scope to just loop over them. Unfortunately that means there are 3 loops nested inside of each other. I think I can get rid of the outer most one by removing the JSON name when I send it, but as for the loop over the keys and values I really want to know if there are ways I can condense the two
 

if isAvailable: 
    lights = [] #name, handle
    def get_lights(handle):
        name = vs.GetName(handle)
        lights.append([name,handle])
    # I think c means criteria?
    criteria = "((PON='Lighting Device'))" 
    vs.ForEachObject(get_lights, criteria)
    

    for light in lxdata["LightingDevices"]:
        uid = light['__UID']
        for light_name, handle in lights:
            if light_name == uid:
               
        
                for key, val, in light.items():
                    if key in globals():
                        globals()[key] = val
                        # print(val)
                        print(key)
                        eachLight(handle)

 

 

[Myke]

One last thing, this script looks like it works, I just need to refresh the view of the lights that are getting edited. Does vs.LDevice_Reset work like the Refresh Lighting Devices menu button?

Thanks for the help. Cheers
 

Here is the current version of the script for anyone interested:

import vs
import http.client
import json

instrumentType = None
# fixtureMode = None
# wattage = None
# weight = None
frameSize = None		
position = None
purpose = None 	
channel = None
unitNumber = None	
universeAddress = None
circuitNumber = None
circuitName = None
dmxLine = None
dmxFootprint = None
# deviceType = None
color = None
template1 = None	
template2 = None	
user1 = None
user2 = None
user3 = None
user4 = None
user5 = None
user6 = None
coordinates = None
__UID = None
__class = None
layer = None

def eachLight(h):
    # vs.SetRField(h, "Lighting Device", "Wattage", wattage)
    vs.SetRField(h, "Lighting Device", "Postion", position)
    vs.SetRField(h, "Lighting Device", "Channel", channel)
    vs.SetRField(h, "Lighting Device", "Purpose", purpose)
    # vs.SetRField(h, "Lighting Device", "Weight", weight)
    vs.SetRField(h, "Lighting Device", "Unit Number", unitNumber)
    vs.SetRField(h, "Lighting Device", "UniverseAddress", universeAddress)
    vs.SetRField(h, "Lighting Device", "Circuit Number", circuitNumber)
    vs.SetRField(h, "Lighting Device", "Circuit Name", circuitName)
    vs.SetRField(h, "Lighting Device", "Inst Type", instrumentType)
    # vs.SetRField(h, "Lighting Device", "Fixture Mode", instrMode)
    vs.SetRField(h, "Lighting Device", "DMX Line", dmxLine)
    vs.SetRField(h, "Lighting Device", "num channels", dmxFootprint)
    # vs.SetRField(h, "Lighting Device", "Device Type", deviceType)
    vs.SetRField(h, "Lighting Device", "Color", color)
    vs.SetRField(h, "Lighting Device", "Gobo 1", template1)
    vs.SetRField(h, "Lighting Device", "Gobo 2", template2)
    vs.SetRField(h, "Lighting Device", "User Field 1", user1)
    vs.SetRField(h, "Lighting Device", "User Field 2", user2)
    vs.SetRField(h, "Lighting Device", "User Field 3", user3)
    vs.SetRField(h, "Lighting Device", "User Field 4", user4)
    vs.SetRField(h, "Lighting Device", "User Field 5", user5)
    vs.SetRField(h, "Lighting Device", "User Field 6", user6)
    vs.SetClass(h, __class)


try: 
    connection = http.client.HTTPConnection("localhost:8000")
    connection.request("GET", "/Test")
    response = connection.getresponse()
    rawData = response.read().decode('utf-8')
    lxdata = json.loads(rawData)
    isAvailable = True;
    connection.close()

except Exception:
    print(Exception.__traceback__)
    isAvailable = False;
    
    
if isAvailable: 
    lights = [] #name, handle
    def get_lights(handle):
        name = vs.GetName(handle)
        lights.append([name,handle])
    # I think c means criteria?
    criteria = "((PON='Lighting Device'))" 
    vs.ForEachObject(get_lights, criteria)
    

    for light in lxdata:
        uid = light['__UID']
        
        for light_name, handle in lights:
            if light_name == light['__UID']:
               
        
                for key, val, in light.items():
                    if key in globals():
                        globals()[key] = val
                        # print(val)
                        print(key)
                        eachLight(handle)
else:
    pass
# print(instrumentType)

 

Edited May 6, 2022 by Myke

[Jesse Cogswell]

What you are looking for is to add vs.ResetObject(h) at the bottom of the eachLight function.  In this case, it is more or less interchangeable with vs.LDevice_Reset(h), ResetObject is will do a reset of whatever object you feed it while LDevice_Reset will only operate on Lighting Device objects.  I usually just use ResetObject since it's what I'm used to using since a lot of my code predates the LDevice_Reset command being part of the VW scripting language.

 

It might also be a good idea to throw a vs.ReDrawAll() command in there at the end of your code to force a redraw of the entire screen.  I've seen issues where running a script like this while on a Sheet Layer will make the changes, but not properly render themselves in a viewport without a screen refresh.  Big caveat, make sure that this command is at the end of the code and outside of any loops, no need to run this command for each and every lighting fixture.

Edited May 6, 2022 by Jesse Cogswell