VectorBench: a Vectorworks benchmarking tool

[Christiaan]

Prompted by previous discussions and a comment by Jonathan Reeves I posted a wish for a benchmarking tool in the beta group (VE-102881). I'm posting here too so it can be discussed.

 

One question I'd love answered, for instance, is how important the extra RAM in an M1 Ultra is, for Vectorworks. At what point do you run out of RAM rendering.

 

Also, I don't know if this is possible but, it would be extremely useful if we could measure not just discreet tasks, like the time is takes to render something, but also the responsiveness of Vectorworks under high load; how fast one can navigate a large project with many classes and layers; how fast one can pan, zoom, change layers, copy paste, swap active windows, use multiple panes, etc.; and, crucially, at what point you run out of RAM.

Edited April 21, 2022 by Christiaan

[Christiaan]

@Pat Stanford I heard through the grapevine that you may have developed a simple timing script for rendered viewports?

[Pat Stanford]

I have a version that runs in VW2022. But I have not figures out how to add RedShift to part of the package.  Give me a couple more days and if I can't get that added, I will clean it up and post what I have.

 

[Jeff Prince]

This would be a great tool for determining appropriate hardware selection.  I think an understanding of how Vectorworks performs certain workflows and where bottle necks are would also be helpful.

 

Example… I have performed identical site model updates on identical files using my 2015 MacBook Pro 2.5 GHz w/16 GB ram and a 2017 IMac 4.2 GHz with 64 GB Ram.  I’m no IT pro, but both of these computers are i7 based and I would expect the iMac to be slightly faster based on the faster processor.  However, in practice my MacBook Pro finishes the task in the same time.

 

 It’s like Vectorworks just slowly churns along on the task, never maxing the memory, and perhaps under utilizing a single core to get the work done.  I’ve done similar comparisons with a 2020 iMac that was maxed out, and had marginal improvements to certain workflows, but not site model updates.  

It’s really hard for me to open my wallet for new tech if the critical workflows are not accelerated by such investment, especially if site models are only using one core and click speeds not increasing appreciably.  Makes one contemplate staying on 2021 and avoiding adopting all the new workflows until Apple and vectorworks leverage the full potential of recent hardware innovations, like the 2020 improvements to the VGM.

[Pat Stanford]

OK, here is the VW2022 version of my render timer script.  If someone would like to "donate" or more complicated scene, that is probably appropriate for todays hardware.  As shown below, on my 2.4 GHz i9 MacBook Pro 16" with 32 Gig of RAM, the multiple Design Layer Render tests ran pretty quick. I thought when I originally wrote this in VW2010 that the FQRW time was measured in like 4-7 minutes, but I can't find it.  My Custom Renderworks settings are at the bottom.

 

If you are on a design layer you will get a dialog box. Put check marks in front of every render mode you want to run.  The script will run and time each of the render modes and give you a report like below.

 

If you are on a the sheet layer and run the script it will render the viewport in whatever mode you have set and give you the Update time.

 

I have not been able to figure out how to add Renderworks Styles (yet.  I'm still hopeful. And stubborn). Once I get them figured out I will add some of the stock styles to the list.

 

If I can figure out Styles I may also convert this to Python so I can upload the report showing the hardware spec and render times on a web page so you don't all have to post your results here in the comments.

 

The below quote if from the last time I posted a version of this script in May of 2020.  It looks like on the same hardware VW2022 renders about twice as fast as VW2020.

 

Quote

The attached VW 2022 file is set up in Perspective view. This is so the rendered area is cropped and not dependent on the screen size. Open the File and double click on the Total Timer script. Check all the boxes in the dialog box that opens and click OK. The script will run through all of the rendering modes and then give you a list of the time each render took.

 

On my 2019 MacBook Pro 16" 2.4GHz i9  32GB of ram I got the following times:

OpenGL 0.1

Fast RenderWorks 3.2

Final Quality RW 15.5

CustomRW 15.3

ArtisticRW 1.7

 

The Custom options may change depending on your custom settings.

The times will vary depending on what else your computer is doing at the time. For the most consistant results restart your computer. Launch VW and nothing else and run the scripts.

 

 

Total Timer v2022.vwx

 

[Peter Vandewalle]

I've been benchmarkling a bunch of M1/Max/Pro Macs recently. I used among others this file with some scripts.

Render Speed Test.vwx

The file contains 3 scripts:

Render Speed Test py: Renders every layer. To really compare different computers, this is not ideal since the screen size will have an impact on the render time.

Render Speed TestViewports: Renders all viewports on a sheet layer.

Reset Viewports: You need this script to reset the viewports before running the Viewports test. Otherwise they won't render since the viewports are up to date 🙂

[zeno]

the time factor is crucial.

 

There are importan mac youtuber in Italy who in the absence of other software ready for apple silicon show Revit Benmchmark emulated on windows. The opportunity should be developed soon. The thing is simple to reproduce: it would be enough to duplicate objects on many design layers. Ore something needs high CPU/GPU processes But I recommend: do not put any structural elements, which could be a bad boomerang

 

I wouldn't just focus on rendering. I wouldn't call it an exclusive field.

 

thank you

Edited April 29, 2022 by zeno

[JRA-Vectorworks-CAD]

Thanks for sharing the Render Speed Tests. These are really useful.

It would be great if Vectorworks could develop these with a nice simple User interface and package it as a type of "VectorBench" up like Cinebench so it could be used as a cross platform way of testing new hardware. I think it would be a great way to get free marketing for Vectorworks in the AEC community, and also help people understand the benefit in upgrading software and hardware properly.

 

Here's my results on a MacBook M1 Pro with 16GB RAM if anyone is interested.