Overhaul Productivity: Reduce Wait Times (Project Sharing Sync, Section Viewports, Verifying File)

[Amorphous - Julian]

We’d like to start a discussion on using VW as a productivity tool. 

 

Currently, 'output' (ie pumping out drawings in an efficient manner) is our greatest bottleneck in using Vectorworks as our productivity tool. 'Output' is what practices like ours depend on, in order to send (and justify) invoices to our clients. 

 

Following are the items we are most desperate to see changes, to improve the BIM workflow:

 

(A1) SLOW TEAM WORK SPEEDS (SAVE AND COMMIT)

(A2) SLOW HIDEEN LINE RENDER TIME (EG FOR SECTION VIEWPORTS)

(A3) SLOW WORKSHEET CALCULATIONS 

(A4) SLOW OPENING OF DOCUMENTS (IE VERIFYING DOCUMENT)

--------------------------------------------------------------------

--------------------------------------------------------------------

A - THE ISSUES

(A1) : SLOW TEAMWORK WORKFLOW 

 

• Currently, when we work on teamwork projects, the save and commit operation is very very slow. (We have very high spec network connection, computer terminals and server, see spec at the end of post) 

 

• Our files size is between 500MB - 1GB. 

 

• Sometimes on the larger files, the Save and commit operation on each terminal can take up to 2-3 minutes. 

 

• Our staff actually go out to get a coffee when this happens.

 

• The most ridiculous part of the teamwork situation is, if I want to release a single element (say, a wall) to a teammate. The ENTIRE working file needs to sync to the project file. And then the teammate need to sync their working file. 

 

• If both these operations took 3 minutes, that's 6 minutes time just to have a wall released.

 

• And yes, we have tried selecting the particular element to be released (say, a wall) and right-click for menu options and 'release'. The same clunky operation of the entire file 'saving and committing' occurs.

 

(A2) : SLOW SECTION VIEWPORTS 

 

• As a file grows in size, so does the time it takes for section viewports (hidden line) to be generated, and it doesn't just increase incrementally, it does so exponentially. We're talking about simple line drawings.

 

• This becomes a real problem when a viewport is out-of-date (often when 'save viewport cache is turned on, viewports still disappear), and you just told your clients 'I'll send through that elevation/section right away).

 

• What you think its a very simple section (simple elevation with shallow depth, restricted height range, classes with complex geometries are turned off) would take 2-3 minutes. 

 

• Since each sheet contains 2-3 elevations, and we may send, say, 2-3 sheets. We suddenly find ourselves spending 10 minutes just waiting for very simple elevations to render.

 

• It sucks to say this, but sometimes we know 'it would have been quicker to draw those 20-30 lines in 2D'

 

(A3) : SLOW WORKSHEET CALCULATIONS 

 

• We run a full BIM model in our office, so everything, including schedules come from our model.

 

• We find the process of recalcuations to be extremely slow, especially given the records attached to objects are really simple. This surely can be improved. 

 

(A3) : SLOW FILE OPENING (VERIFYING FILE)

 

• Once a document is read from the server (fast), we can wait up to one minute for 'verifying File....' to occur (slow)

 

--------------------------------------------------------------------

--------------------------------------------------------------------

B - OUR SUGGESTIONS:

 

(B1) ALLOW REQUEST, RELEASE AND SYNCING OF SINGULAR ELEMENT

 

• ArchiCAD teamwork has a function to 'request' a particular 'checked out' item (say, a wall) from another user. The other user who receives the request can release just that one wall, and only the changes to just that one wall gets synced between the working files and project file.

 

• Vectorworks should have the same function. (Remember, Vectorworks is a productivity tool, and when it hinders productivity like the 6 minutes of wasted time in the above example, it is a really really big issue).

 

(B1.1) MAKE 'SAVE AND COMMIT' A MULTI-THREAD PROCESS

 

• In the above example, if we expand the number of team members on a project to 4. A save can commit operation can take 8-12 minutes. 

 

• This is ridiculous.

 

• The slowness is not due to the network speeds or computer speeds, it seems to be the sheer amount of processing that is required to process the changes.

 

• If this operation is a multi-core operation, I would imagine this to be a lot faster. Again this is a MASSIVE productivity issue for our office. 

 

(For those of you who will be offering tips to reduce file size- thanks for the advice, we have done all that, and there is good reason why we need to keep information we have within the file).

 

(B2) OVERHAUL THE VW CODING FOR HIDDEN LINE, PLUS MAKING THIS MULTI-CORE PROCESS

 

• This seriously needs urgent change. We are always sitting in the office until early hours in the morning to publish

 

• Remember, the first round of publishing is really just for 'checking and markups'. And if we have to wait around for a few rounds of this. It is neither productive or fun.

 

(B3) WORKSHEET SPEEDING UP

 

• We can't comment on how your back end works. But the time taken to calculate the simple information we would like to display seems disproportional.

 

Just a final point I'd like to reiterate- these are really important productivity issues that we battle on a daily basis. @Jim Wilson I really hope we don't have to wait until version 2020 until we see these improvements. 

 

--------------------------------------------------------------------

--------------------------------------------------------------------

 

C OUR COMPUTER SPECS:

All of our computers servers are : Dual 6-core X5690 Mac Pro towers | 32GB RAM | Radeon RX580 8GB | SSD Disks | Sonnet 10GBe network card 

Our computers servers is : Dual 6-core X5690 Mac Pro tower | 32GB RAM | NVme Drive (in PCIE slots) for VW BIM files | Solarflare Dual 10GBe Network Card

Our network has a Zyxel 10GBe network switch, our network runs on CAT7 Cables

Our server has A trunked pair of fibre optic cable (10GBe) connecting to the switch

(open to further upgrades, but benchmarks we have seen shows an upgraded cMP is pretty close to an iMac Pro)

 

Edited September 19, 2018 by Amorphous

[JMR]

It seems VW is a single-core operation most of the time.

 

You can check via (the Mac equivalent) the task manager which reveals the reality, it's practically 99% of the drafting/plan work time at ~ 12.5%. Which is one of eight cores.

 

The code needs updating to utilize modern processors asap.

 

 

[Patrick Fritsch]

I thought 2019 was advertised as multi threading, but when I leave task manager running on my second monitor I only see it using one? 

[Amorphous - Julian]

UPDATE: The 'save and commit' speeds are continuing to provide a massive headache for our office....

 

I have just screen-recorded a 'save and commit' process in the following video. You can see in the video the process took around 6:00 minutes. That is twice as long as I thought it was in the posts above. 

 

 

While the 'save and commit' process is in progress, other team members can't do very much on the project, because all changes to the Working File needs 'permission requests' into the Project File.

A project file that is engaged in a 'save and commit' process rejects all 'permission requests'***

 

Lets do some maths here, assume a team of 4 workers.

 

1. Worker A saves and commits, Workers B, C and D cannot do anything during that process.
   @ 6:00 minutes for the process, that is 4 x 6:00 minutes (total 24 minutes) of lost staff time (man hours) for just one terminal to refresh working file. 
2. Worker B then saves and commits. That's another 24 minutes of man hours.... etc. etc (repeat four times for four workers)

So, for the project team of four to just have one round of refresh, we lose 96 minutes (1 hour, 36 minutes) of man hours.

 

For reasons I described in another post under 'troubleshooting' forum. It is not possible for us to only do one round of refresh per user per day, as that risks our 'Working Files' to being permanently disconnected from the 'Project File' with irrevocable consequences (work done on say, creating new viewports cannot be simply copied and pasted from one file to another)

 

@Jim Wilson with sincerity, we really do struggle with this issue on a daily basis, and hope you can reflect this to your team and find a quick resolution. We love using Vectorworks and would love that love reciprocated. 

 

------------------------------------------------------------------------

 

***One way around this this 'permission issue' is for other members of the team 'check out' as many items as possible, prior to this 'save and commit'. But this kind of process is impossible to coordinate and plan, as we always need a number of elements as part of the workflow (different 'objects', 'classes', 'layers' 'viewports' etc).

 

[Kevin McAllister]

On 8/29/2018 at 6:36 PM, Amorphous said:

(B2) OVERHAUL THE VW CODING FOR HIDDEN LINE, PLUS MAKING THIS MULTI-CORE PROCESS

 

• This seriously needs urgent change. We are always sitting in the office until early hours in the morning to publish

 

• Remember, the first round of publishing is really just for 'checking and markups'. And if we have to wait around for a few rounds of this. It is neither productive or fun.

 

I think this is a big issue for everyone, even those that don't use project sharing. If you're working form a full 3d model, making any small change means many viewports need updating afterwards. Rendering for check sets (maybe multiple times) and then finally rendering for an issue set can take a lot of time.

 

On 9/21/2018 at 5:38 AM, Patrick Fritsch said:

I thought 2019 was advertised as multi threading, but when I leave task manager running on my second monitor I only see it using one? 

 

I think only Renderworks and the new on demand tessellation features are true multicore. I know geometry calculations are not.

 

Kevin
 

[Matt Overton]

On 8/30/2018 at 11:36 AM, Amorphous said:

(B1.1) MAKE 'SAVE AND COMMIT' A MULTI-THREAD PROCESS

 

• In the above example, if we expand the number of team members on a project to 4. A save can commit operation can take 8-12 minutes. 

 

• This is ridiculous.

 

• The slowness is not due to the network speeds or computer speeds, it seems to be the sheer amount of processing that is required to process the changes.

 

• If this operation is a multi-core operation, I would imagine this to be a lot faster. Again this is a MASSIVE productivity issue for our office. 

 

Yes please, even for smaller files it would be a win to offline save, commit and sync.

I mean it's an inherent part of our job to management co-ordination of out of date information both within our teams and at a larger project scale. We can deal with the risk here that is minutes, not days like is common when we deal with other consultants.

 

That said the old model of checking out whole layers seemed problematic it just meant we needed to keep the number of layers we always had for Workgroup Referencing. 

 

Also, In term of streamlining workflow. I think we could add "Updating Story Linked objects" as a background potential operation. Especially given it happens often inside the organisation dialogue while we might be making numerous edits. Slowing down the progress of each edit while waiting for non-visible operations to occur.

[JMR]

The fact that the application uses only single core most of the time is a bit baffling. Would I be off with an old Pentium 4 at 4.4GHz? Or a Core2Duo?  I've invested $500 to a Core i7 processor, the capacity of which I'm using most of the time is only 1/8th.

[Matt Overton]

On 9/27/2018 at 12:11 AM, JMR said:

The fact that the application uses only single core most of the time is a bit baffling. Would I be off with an old Pentium 4 at 4.4GHz? Or a Core2Duo?  I've invested $500 to a Core i7 processor, the capacity of which I'm using most of the time is only 1/8th.

 

Even a basic step like offlining single threaded operations so the interface gets back to user input would have been a major win and would have set them up to then multi-thread those processes in the background. 

[Amorphous - Julian]

@Matt Overton @JMR @Kevin McAllister

According to the following post (and various others too), an Upgraded Mac Pro 4,1 or 5,1 is the best Mac Computer you can get on the market right now, beating even the iMac Pro, and I quote:
 

https://create.pro/blog/mac-pro-51-best-system-creative-professionals-2018-internal-expandability-unparalleled-workstation-customisation/

 

'We also found that in testing our Mac Pro 5,1 still continued to beat the iMac Pros graphics computer performance in every test'

 

All of the computer terminals in my office is of the above specification with X5690 processors, SSD/NVMe Drives and RM580 Graphics Cards. In other words, our terminals all have 12-core processors. 

 

However, operating Vectorworks on what is allegedly best Computer Terminals is still slow and laggy.

 

Having fully graded to the best machine of the day, we can only point this lack of productivity in our workflow to one culprit...

 

 

[Amorphous - Julian]

Per my post above about 'release' in 'project sharing' not working properly.

 

When we 'right click' - 'release' any objects, VW prompts us to 'Cancel', 'Discard' or 'Commit' changes to the file before the 'release' can be done.

 

If we select 'Commit', then the Working File will suddenly disassociate from the Project File.

 

This happens 80% of the time. And then we have to go through the 20-minute exercise of 'let's create a new version' as detailed above.

 

Can someone from Vectorworks please address these very real frustrations we face daily? I feel like we are just users talking amongst ourselves.

[JMR]

We also get this.

 

Very often it occurs when the commit operation crashes while in progress. It occurs also when the "administrator" is not even working on the project.

 

[Guest Tolu]

If you are getting the message the that "The commit operation failed because your permissions were changed..." and you are certain the administrator did not release your objects, could you please send me your PF and WFs (in a state that exhibits this problem) via a private message?

 

Thanks,

Tolu

[Hugues]

On 8/29/2018 at 9:36 PM, Amorphous said:

 

(A3) : SLOW WORKSHEET CALCULATIONS 

 

• We run a full BIM model in our office, so everything, including schedules come from our model.

 

• We find the process of recalcuations to be extremely slow, especially given the records attached to objects are really simple. This surely can be improved. 

 

(B3) WORKSHEET SPEEDING UP

 

• We can't comment on how your back end works. But the time taken to calculate the simple information we would like to display seems disproportional.

 

 

 

@Amorphous Improving worksheet recalculation speed is on our radar.

Are you recalculating all worksheets in your file? or just a single worksheet? (Note that there are two 'recalculate' commands; one will update all worksheets in your file and the other will just update the active worksheet). Updating all worksheets in your file will be slower of course.

The recalculation speed is proportional to the file size or the number of objects in the file. The nature of the data to be extracted from the objects also determines the performance.

Do you have a typical file that demonstrates slow worksheet calculations? Please send it to me via private message.

Because slowdowns can be caused by many different factors, it is very helpful to us to see many different examples.

 

 

[Amorphous - Julian]

4 hours ago, Tolu said:

If you are getting the message the that "The commit operation failed because your permissions were changed..." and you are certain the administrator did not release your objects, could you please send me your PF and WFs (in a state that exhibits this problem) via a private message?

 

Thanks,

Tolu

 

Hi Tolu thanks for your reponse. I’ve sent you the file via a PM, but to summarise:

 

Everytime we have a permission issue. We ‘save a new version’. I am up to ‘Version 175’ for one project and ‘Version 59’ for another.

 

We don’t make a record of which version had which type of permission problem, so I can’t send you that PF and WF ‘in that state’ until it next happens.

 

What I can tell you is that the most common permission issue is:

- a user (‘User X’) checks out ‘an item’ in WF

- VW crashes on the computer terminal of ‘User X’

- the ‘item’ checked out by ‘User X’ at the time of the crash presents permission issues

- When anyone (including ‘User X’) tries to edit that said ‘item’ in the file, it says ‘User X’ has the item checked out and thus can’t be edited 

- we have to save a new version to get around this permission issue

- In ArchiCAD, the BIM admin can ‘force’ a user out of the file to resolve permission issues. Would be great to have the same in VW.

[Amorphous - Julian]

@Tolu please also check out this troubleshoot post for other users with PS issues:

 

 

[Guest Tolu]

2 hours ago, Amorphous said:

We don’t make a record of which version had which type of permission problem, so I can’t send you that PF and WF ‘in that state’ until it next happens.

Understood. Please save the files when next it happens, and send them to me. I can look at the History to determine what happened.

 

2 hours ago, Amorphous said:

In ArchiCAD, the BIM admin can ‘force’ a user out of the file to resolve permission issues. Would be great to have the same in VW.

Vectorworks already has this feature. An admin can release the exclusive lock that User X has on an object/layer using the "Release" command. In VW2019, it has been renamed to "Administrative Release".  Note: I suggest only using this as a last resort, however. Because doing this will prevent User X from being able to commit their work into the PF. 

[Guest Tolu]

2 hours ago, Amorphous said:

please also check out this troubleshoot post for other users with PS issues:

This user was using Google File Stream, which is not supported. Project Sharing supports Google Backup & Sync. You can also use Dropbox and Box.

 

If you are looking for a cloud solution, I would recommend Dropbox because of its LAN Sync feature.

[SNOJNKY]

On 9/25/2018 at 10:21 AM, Kevin McAllister said:

 

I think this is a big issue for everyone, even those that don't use project sharing. If you're working form a full 3d model, making any small change means many viewports need updating afterwards. Rendering for check sets (maybe multiple times) and then finally rendering for an issue set can take a lot of time.

 

 

I think only Renderworks and the new on demand tessellation features are true multicore. I know geometry calculations are not.

 

Kevin
 

Kevin, you're so right!  Waiting nearly 20 minutes to update a window schedule is absurd.  And a 500MB file should not take 3-4 minutes to save or ro regenerate a section viewport.   

-Carol 

 

[Amorphous - Julian]

I'm happy to report that since moving from VW2018 to VW2019, the 'Save and Commit' times have dramatically reduced. 

 

On our 1.2GB file (600MB as project file, 1.2GB as working file), the 'save and commit' time is now 60 seconds or less.

 

However, a very important part of what this thread is about, namely 'Section Viewport render times', remain a big headache for our productivity with VW.

 

For some simple viewports, we now get up to 10min rendering time for hidden line. For complex viewports, we get up to 30min. 

 

@Jim Wilson since this thread was started on 30 August last year, I would like to ask if you are able update us on what has been done, or what the plan is, to improve Hidden Line Render times? When can we expect a change?

 

PS. There should be an easy way for users to 'Convert hidden line render to lines and polygons within Viewport', so we can continue to work, at 1-1 scale, with what has been rendered inside a viewport (I'll start a new feature request on this). 

[PVA - Admin]

Its the sectioning speed thats hurting things more than the hidden line portion of the viewport creation apparently, and yes it has been slated to be improved. 

[Kevin McAllister]

2 hours ago, Jim Wilson said:

Its the sectioning speed thats hurting things more than the hidden line portion of the viewport creation apparently, and yes it has been slated to be improved. 

 

Its interesting having moved from a 2012 MacBook Pro to a 2017 iMac Pro recently. These bottlenecks are very apparent and make the new machine almost irrelevant for these tasks. Hopefully when these improvements finally appear they will make a big difference.

 

Kevin

[Amorphous - Julian]

@Jim Wilson thanks for your response. 

 

I did some experiment, and found that in some instances, the extreme slowless is caused by 'surface hatches'. 

 

Please see enclosed simple example, sided by side, where one takes 8 seconds (with no surface hatch), and the one with surface hatch takes many many times that amount of time (with surface hatch on). Should I post this 'surface hatch' issue as a separate troubleshooting case?

 

You mentioned that sectioning speeds is slated for improvement. Can you indicate if is an improvement that can be expected in VW2019 SP3?

 

Thanks again. 

[PVA - Admin]

On 3/2/2019 at 7:27 AM, Amorphous said:

Please see enclosed simple example, sided by side, where one takes 8 seconds (with no surface hatch), and the one with surface hatch takes many many times that amount of time (with surface hatch on). Should I post this 'surface hatch' issue as a separate troubleshooting case?

Woah, yes absolutely. Thats an insane difference for just hatches as the variable, let's track and log that one entirely. Could you send me a sample?

[Matt Overton]

On 3/2/2019 at 11:27 PM, Amorphous said:

@Jim Wilson thanks for your response. 

 

I did some experiment, and found that in some instances, the extreme slowless is caused by 'surface hatches'. 

 

Please see enclosed simple example, sided by side, where one takes 8 seconds (with no surface hatch), and the one with surface hatch takes many many times that amount of time (with surface hatch on). Should I post this 'surface hatch' issue as a separate troubleshooting case?

 

You mentioned that sectioning speeds is slated for improvement. Can you indicate if is an improvement that can be expected in VW2019 SP3?

 

Thanks again. 

What if you change the surface hatch to something more simple?

How does that effect the render time?

 

There must be 10's a layers in that hatch.

[Amorphous - Julian]

@Jim Wilson 

please give me your email and I'll shoot it over.

@Matt Overton 

yes.

This is a problem with surface hatches at the moment. You cannot choose 'tile' hatches in 'Textures' as the surface hatch.

So in order to do a 'timber' or 'stone' hatch so we have the correct elevational hatch in hidden line drawings (Eg elevations and sections), we have to make tens of layers to achieve that. 

If 'tile' attaches were allowed to be attached to 'texture', the we would not have this problem. 

I'll play around simpler hatches and see.

[Bruce Ferguson]

Tolu has entered a bug for the slowness with surface hatches: VB-158507.

@Jim Wilson If you received a file, can you attach it to the bug?

@Amorphous I assume the two samples you show are section viewports. Do you see the slowness when you update the section viewport?