Jump to content
tekbench

.LAZ file coming in to document way out of scale.

Recommended Posts

Hey all. First serious shot at this, so bear with some noob questions.

Trying to pull in a .LAZ file to make a site model.

I found a good LiDAR map of the site I need to model. File size isn't too bad. About 85MB.

Here is the place I downloaded the file:

https://viewer.nationalmap.gov/basic/

I search around using the term "bogachiel state park", and choose that blob.

No idea of the relative scale other than it is an X,Y,Z coordinate file. And that makes sense. Shouldn't need a scale so to speak.

 

I open it in an open-source web utility.

I used this one:

http://plas.io/

Realize the file is X,Y,Z format, and looks correct. It's the file I need, and it seems to be happily displayed in this web viewer.

I put a 'real world' model on my file, and set the file units to Feet, and it looks correct. The viewer plops a model of the Empire State building on the cloud file as a sanity check. I say it looks about right and carry on.

 

Create a new Vectorworks file. No templates.

Drag and drop the file in.

It pulls in all 23 million plus data points.

Asks me what units the file is in.

I select 'Feet'. Here is the rub:

There doesn't seem to be an input button for "X,Y,Z" on the file import dialog. Just 'lengths' like Millimeters, centimeters, meters, feet, inches.

I choose feet, because I have no idea what the software is really looking for, and the dialog makes me choose something.

Cloud opens up, and it is completely out of whack. 

Points are all good, cloud looks correct, 3D views look correct, but the scale is way off.

Whats my best bet here? I don't have a clear way to 'scale' the cloud data, nor do I think that is a good idea. I can get a general idea from a bridge and some frontage, but not enough to be accurate if I scale things.

What I would like is a properly scaled version of this. If I pull a dimension along the river edge, I should see about 900' from a bridge to a turn. Instead I see a distance of about 801,000'. What am I missing here?

 

 

Share this post


Link to post

UPDATE - 

I just tried pulling the file into a number of new drawings, multiple times. Each time choosing a different unit standard on the file import document. 

The default selection of 'Inches' yielded the closest to scale drawing, but it is still off by quite a bit. I'm calling this un-solved.

 

I threw caution to the wind, and decided to forge ahead and attempt to create a site model out of my cloud data. Albeit way out of scale.

I selected the Point Cloud item.

I selected 'isolate points'.

I isolated approx. 150,000 points. a rectangle around the area i'm interested in modeling.

I deleted the rest of the points.

I chose 'landmark' and chose 'create site model from data' (I think. Program is churning away so I can't remember the exact tool).

I choose some settings in that dialog (including the smoothing feature and changing the major and minor settings)

Computer is still churning and updating 20 minutes later. It has basically hung right at about 90% complete. It's been sitting at this point way longer than I would expect. I ended up having to crash out of the application.

 

1445298205_ScreenShot2019-07-29at2_18_58PM.thumb.png.9848ee105fe199051be119524188fea6.png

 

 

I guess I have some questions about the point cloud import tool.

1. Why do I have to choose a unit in the import dialog? The standard of the .las or laz appear to define that inside the file itself. I don't ever see that on the outside. I have no idea what I should choose.

2. If I can't see/understand/translate the .LAZ file before hand, how should I prep my drawing before importing it? Does it even matter?

3. how many points can I reasonably expect to import? in my case, we're at about 230 million. works just fine to import the file, but not create a site plan.

4. How many loci can I reasonably expect to extract during the 'create site model object' operation? In my case, we're close to 300,000 or so. Does not work. I see the limit is 500,000, so we are well short of that. I've even tried cutting that number down to 150,000.

 

I have found the hanging problem may be related to the dialog box in the 'Create Site Model'. If I use the smoothing feature, it seems to hang every time. If I choose the defaults, it creates the site model, no issues, in a few minutes. the model is a little rough with a lot of triangles, so not really close enough for what I need.

 I'm still experimenting with this, but it doesn't seem to be working very well on my small-ish file.

I also choose the 'check data for problems' and it says the data is fine. I'll bang on it a little more, but for now it's just turning my laptop into a space heater then crashing.

 

Macbook Pro (15 inch, 2019)

2.4 ghz, intel core i9

32 gig 2400 mhz ddr4 RAM

Radeon Pro 560x 4 gig RAM

settings set to always use best graphics option.

Share this post


Link to post

Update - 

I left this running last night at the office and went home. It had finished this morning. I have no idea how to tell how long it took to process. I'm sure it is in a log somewhere, but I have no idea where. 

Is there a reasonable way to determine how big this site model file is going to get? My 85 mb cloud point map turns into (in one test case) a 15 GB Vectorworks site model. This in turn makes the drawing useless for me obviously. I need a super computer just to deal with it. That is 150,000 loci, with smoothing, 5' terrain slope change intervals.

The GOOD news here is my site model looks great. It is essentially exactly what I'm looking for. It's just huge, and my computer won't deal with it very well.

Again, this is 150,000 loci. Well under the limit. I feel like i'm doing something that shouldn't take this long to process.

Also the scale is still way off, and I have no idea how to fix that yet. I've attached a screen shot of the 2d, top plan view, and of the OIP.

Kelly

 

Screen Shot 2019-07-30 at 8.56.10 AM.png

Screen Shot 2019-07-30 at 8.58.55 AM.png

Share this post


Link to post
Posted (edited)

Hi, Kelly,

there was some earlier discussion on this. Couple things:

-  lidar files assume millimeters

-  point cloud import dialog offers control of how many points to import. Lower count equals lower hardware overhead and file size at cost of resolution. Find your balance level.  Import dialog changed in version, but function is same. 

 

 

HTH

-B

Edited by Benson Shaw
File size
  • Like 3

Share this post


Link to post

Wow. That is an interesting read. 

I'm seeing exactly what @ericjhberg is seeing. VW just does some maths of some random flavor on the Point Cloud import, and it is not correct.

Remember that step one is to pull the .LAZ file into Vectorworks. this has no dialog other than total number of points in import and 'units'. 

I'm importing 230,000,000 points. The scaling issue happens if I import 230,000,000 or 230,000 points. Same result regardless of number of points imported.

I'm choosing millimeters, centimeters, meters, Kilometers, feet, inches - I tried them all. Same result regardless of units chosen.

I'm creating a new document, saving, and setting the document up in the units that match the .LAZ file. Same result regardless of Vectorworks file setup.

 

Here's my non-advanced 'create new document with no units set-up method:

File.

New.

Drag and drop .LAZ file onto drawing.

 

Here's my advanced 'Create new document with units set-up' method.

File.

New.

Save.

Open Document Settings

Change units/everything to MM.

Import all points from the point cloud. (sizing is obviously wrong at this point, but bear with me. I wanted to go through the isolation step and re-measure)

Isolate points.

Delete outside isolation.

Measure.

 

Point Cloud data is NOT coming it at 1:1. It is NOT working correctly. Period. I'm not sure what this bug is, but It is now holding up the entirety of my project.

The issue is directly with the import of the .LAZ file. In Step One, regardless of whatever I try, VW can't get the point cloud .LAZ file imported at the right scale.

 

Attached is the project and a screen shot of the units page.

The VW file is an isolated point cloud (not even a site model yet) with a red and black poly sitting on top of it. The long side of the red poly is about 1300' in US measurements. That red Poly is an exact, to scale trace of the property line. It is used in a different drawing with no issues, it's scaled correctly, field verified and it is what I want. 

the Black Poly is the approximate shape and size of the property in question AFTER importing the point cloud. Obviously, the Point Cloud is coming in at a factor bigger than 1:1.

Scaling the object will not work (as suggested by VW staff) for absolutely obvious reasons.

I have a ticket into VW that I filed on Monday, but no response as of yet.  

Attached is the VW file.

any insight would be welcome. I need to get fill and cut data to the dirt guys ASAP. 

Reminder -

Other applications read this point cloud data just fine, At scale, at the right size. The problem is ONLY when importing it to Vectorworks that it is failing to come in at the right size.

I can email the point cloud or put it on dropbox if anybody wants it. Just can't upload it here on the forum.

 

PointCloudFailing.vwx

Screen Shot 2019-08-02 at 11.15.58 AM.png

Share this post


Link to post
Posted (edited)

Hey All. the attached screen shots should show you what the issue is. I think, honestly, Vectorworks is confusing Meters and Feet. I import a .LAZ or the identical .LAS file and get the same results.

With my new drawing set up as shown above (mm for everything), Vectorworks happily pulls in the Point Cloud Data.

VW is showing approximately the correct dimensions, but swapping Meters for Feet. Meaning - The scale is wrong, and Vectorworks is naming the length in MM, but the measurement in meters is nearly numerically equal to the measurement in Feet.

Example. 

Line xyz is actually a known length of 1000 feet. Vectorworks is calling that line 1000 meters instead of feet, and scaling the document accordingly. 

When using the settings above, I essentially am getting the Point Cloud imported in MM, but VW is substituting Meters for Feet without doing the division to make the foot distance correct (roughly 1/3 the size)

I can't prove any of this with accuracy due to the separate tools used, but I find it really fishy that if VW thinks it the point cloud is approximately "100" long, and Google Earth thinks it is "100" long, that line is approximately "100" Long. Google Earth knows that it is 100 feet long, but Vectorworks thinks it is 100 METERS long.

This happens regardless of what import units I select when I import the point cloud. No matter what units I choose, I never get the correct length.

Guide to files attached:

Google Earth Triangle - Screen shot of Google Earth Pro, some lines, and what Google Earth Pro says the length of those lines are. These lines correspond (roughly) to a current survey that was just done (less than a year ago). I have the survey, and can verify, for example, the shop is 'about' 120 feet long. I can verify the distance from the bridge to the trailer is 'about' 950 feet long.

VW Triangle - Screen shot of Vectorworks file, some lines, and what Vectorworks says the length of those lines are POST Point Cloud import.

VW LIDAR IMPORT - Screen shot of the whole .LAZ import, and a small rectangle around the area i'm interested in. Essentially this is the rectangle I draw when I use 'Isolate Points'.

Ideas?

VW LIDAR IMPORT.png

2070482039_VWTriangle.thumb.png.cd5ad9e6838de35edaccd69a6bd73448.png

Google Earth Triangle.png

Edited by tekbench
Double checking dimensions on file.
  • Like 1

Share this post


Link to post

Wow, Kelly - Thanks for all the work and reporting on this.  Same, thanks to @ericjhberg & @RussU if you are watching this thread. This is a problem that will become ever more prevalent as LAZ data become increasingly available and imported more often into vwx.

 

-B

Share this post


Link to post

I'm going to try and dial this in a little bit more. From what I see, the Point Cloud data is correct, and is being translated by multiple software packages correctly. It is NOT being translated by VW properly.

All the third party viewers are opening the file just fine, and the measurements are reasonably close. within a few feet.

Everything but Vectorworks sees the Bridge to Trailer dimension as 'Just a little under 1000'.

I re-tried this whole exercise setting up a new VW file as 'feet' vs. 'feet and inches'. File imports, and all the dimensions are shown (albeit incorrectly) in as MM regardless of how I set up the drawing.

 

Share this post


Link to post
17 hours ago, tekbench said:

I re-tried this whole exercise setting up a new VW file as 'feet' vs. 'feet and inches'. File imports, and all the dimensions are shown (albeit incorrectly) in as MM regardless of how I set up the drawing.

 

1

Kelly have you tried to import in meters vs mm?

 

Share this post


Link to post

Can also confirm - 

This is an issue in 2019, SP4. I just updated everything on my machine. It was an issue on 2019, SP1 as well.

So this recognized bug has been evident through 4 service packs now.

Bummer. Makes the tool 98% unusable. I'm going to try scaling the point cloud to see if I can get it close to the right size. 

Seems silly that a major math error has taken this long to get fixed. No less a math error in a calculation that seems pretty rudimentary.

I've been on the phone with VW tech support. They are seeing the same issue with the same said files and are digging into it. It seems the bug has managed to worm its way through a number of service packs.

 

Share this post


Link to post

@tekbench I played with the file and managed to get thing corrected and to scale for you. Tech should be sending the file out soon. There does seem to be a slight issue with importing that file but the new one they are sending is corrected, georeferenced and set to design 0,0.

 

I hope this helps

 

  • Like 1

Share this post


Link to post

Thanks for that. I'll keep a look out for the new file.

Again, every other software package seems to import the file just fine. It's just the Vectorworks import math issue that is creating problem here.

I appreciate you guys getting that all set to go for me.

Kelly

Share this post


Link to post

Just checking in here. Haven't seen the fixed/corrected file yet. I tried to scale my point cloud yesterday (that worked), but after 19 hours of building a site model, VW just hung. So i'm interested in if this new scaled file you guys are putting together works any better to build a site model.

Share this post


Link to post

I just sent the file to you via PM.

 

 

  • Like 1

Share this post


Link to post

Ahhh. cool. appreciate it. I see it in the ol' inbox. Thanks.

Share this post


Link to post

OK. This is plenty to get me started. I appreciate that.

I still would like to get the actual import of .LAZ files sorted out. Or at least get how you managed to fix the point cloud within Vectorworks to get it to relatively the correct size and shape.

Or at the very least, get a clear breakdown from engineering on how the tool is supposed to work, and if there has been a bug or known issue assigned to the problem.

Thanks again for fixing that particular file for me.

  • Like 2

Share this post


Link to post

@bgoff could you please outline for the forum your process for the import or rescale or whatever you did.  Any additional comment regarding best workflow appreciated.

 

-B

  • Like 1

Share this post


Link to post

I spent all day yesterday tweaking on this drawing. Some talking points and a new bug has appeared (I think):
KEY INFO NEEDED - Need to know how this was re-scaled/fixed/adjusted to be the correct dimensions.

ORIGINAL BUG - The original issue of not being able to correctly import .LAZ files is still accurate and still stands.

NEW BUG - Can't seem to undo 'Create Berm' within the edit surface window without hanging machine. Regardless of how easy it was to create the berm or work with the berm.

 

If you pull the .LAZ in, isolate points, and create a Site Plan using the all the 'defaults' - depending on your system this is fine. Otherwise It will end your afternoon with a beachball.

If you pull the .LAZ in, isolate points, and create a Site Plan using custom settings that seem fairly reasonable - depending on your system this can end your afternoon with a beachball.
If you pull the .LAZ in, isolate points, and create a Site Plan using about 50,000 points- in my case on my hardware this can be a delight.


What I can say is this - The number of points selected in the 'create site plan' dialog is key in determining how long your computer will churn on the file. the 'other stuff' matters, but in short, this number is critical IMHO. I've attached a file called "LAZ File Corrected FRESHTEST" for anybody that wants to try this out. The file is 3.4meg. hardly a large file.

 

ON MY HARDWARE - 

NON-FUNCTIONING METHOD, CAN'T SEE THE BUG DUE TO THE BEACHBALLS AND CRASHES - 

Open the file.

Select the isolated Point Cloud.

Landmark - Create Site Model - Site Model from Source Data.

There are 111,683 points. Pull all those points in.

Select 3D Mesh, no smoothing and all the rest of the default dialogs.

Click OK.

Site model appears eventually. Maybe an hour later, maybe 11 hours later (seriously, all night, finally appeared), maybe never. maybe crashes. If it DOES appear- 

Click on 'Edit Proposed Site Model Surface'

Wait.

Select the Berm/Swale tool. Click ok on the dialog.

Wait.

Draw your Berm with settings 16' wide, 10' tall. Make it a couple hundred feet long, with a little bit of a curve.

Wait.

It appears. Maybe an hour later, maybe never. maybe crashes. If it DOES appear - 

Apple+Z to Undo

Crash. Woe be unto you. Beachball.

 

FUNCTIONING METHOD, BUT BUG UNCOVERED - 

Open the file.

Select the isolated Point Cloud.

Landmark - Create Site Model - Site Model from Source Data.

There are 111,683 points. Move slider to about 50,000 points.

Select 3D Mesh, no smoothing and all the rest of the default dialogs.

Click OK.

Site model appears quickly.

Click on 'Edit Proposed Site Model Surface'

Wait a tiny little bit.

Select the Berm/Swale tool. Click ok on the dialog.

Wait a tiny little bit.

Draw your Berm with settings 16' wide, 10' tall. Make it a couple hundred feet long with a little bit of a curve.

It appears rather quickly. A delight.

Apple+Z to Undo

Crash. Woe be unto you. Beachball.

 

From what I can tell, 111,683 points to create a site plan will crush the spirit of a brand new MacBook Pro. Drawing will not function, will not work and will constantly hang and crash. 

50,000 points is a delight and an ease to work with and looks OK on a 2019 MacBook Pro.

 

In BOTH of these cases - 

You can't 'undo' create berm in the site surface edit window. It will hang, and eventually you will need to crash out of the application.

 

Hope these repro steps and attached file help get to the bottom of the issue.

Machine performance details also attached. Cinebench score of 2899 points.

Screen shot of my 'about' page.

 

 


 

LAZ File Corrected FRESHTEST.vwx

Screen Shot 2019-08-08 at 2.10.36 PM.png

Screen Shot 2019-08-09 at 1.51.28 PM.png

  • Like 1

Share this post


Link to post

Wow, again, Kelly - Thanks for all that analysis!  I also have hangs with undo.  Not just with berms and site models.  Seems large or point rich files need some extra calc time for the undo, especially in groups and other containers.  Often much more time to undo than the original "do" action.  I will try to record when and where and put it in the trouble shoot forum as I encounter it.  When I get a min, I will try to load your attached file.

 

-B

 

 

Share this post


Link to post

@bgoff any update on the workflow that you used to fix Vectorworks to accept the .LAZ file at the correct scale? I have some more work coming in, and using the .LAZ file is something we really need in the workflow.

  • Like 1

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

 

7150 Riverwood Drive, Columbia, Maryland 21046, USA   |   Contact Us:   410-290-5114

 

© 2018 Vectorworks, Inc. All Rights Reserved. Vectorworks, Inc. is part of the Nemetschek Group.

×