Survey expectations in the point cloud era

[line-weight]

For the first time in a while, I've had a site surveyed by a 3rd party surveyor (most of my jobs are urban and on small plots so I very often just do any surveying myself).

 

Previously my expectations for a site survey would be a DXF file, with various elements in different layers/classes. If I was lucky the classing would be useful/meaningful.

 

The survey would also have spot heights over the entire site, at reasonable intervals. If I was lucky those would be in the form of 3d locii - otherwise 2d but with the elevations written next to them. Maybe it would even come with contours already extrapolated.

 

For this survey, I just asked for a topographical survey with spot heights, and certain objects (tree trunks, boundary, building footprints) marked. It was clear that they would do a lidar type survey regardless and they told me they'd send me a point cloud, along with a PDF/DXF version of a conventional 2d plan.

 

Because this has all changed a bit since I last had such a survey done... a couple of questions before I start querying/complaining about anything.

 

1. The DXF file has spot heights marked across the site but they are just 2d locii. This is usable - but I'd have to manually give each of them their proper elevation in order to make a site model from them. Given that a point cloud exists, is it unreasonable to expect that the spot heights they extract and give to me in this simplified DXF form should be 3d locii? In other words, is that something that would be normal to process from the point cloud raw data?

 

2. The blank areas (some are large-ish) I'm guessing are bits of ground that they didn't pick up in full detail, as a result of where they positioned their survey device. But it's a reasonable/normal expectation that they should pick up the full extent of the site, right? If they missed a bit, then they ought really to go back and collect it?

 

3. One of the things I asked for is the site boundary, which is mostly defined by fences. Some of those fences are somewhat buried in hedges. I haven't tried checking yet, but I have a feeling that the fence lines they have drawn on the DXF plan are based on looking at the point cloud data and saying, the fence is probably about here. There are not reassuring circle or square objects representing significant fence posts (generally, I'd expect these to be recorded at each change in fence line direction). When something significant (like a corner fence post) is hidden from a lidar sensor by foliage or other objects, they need to either add another sensor location, or get in there with a manual measurement relative to something that has been picked up - again, is that a reasonable expectation?

 

4. Likewise... the existence of a shrub or tree isn't an excuse not to record the ground level immediately below it, right? Obviously within reason - but traditional methods would often involve choosing survey points strategically, or moving some branches aside, in order to get a fix on the base of a significant tree trunk or similar, I would say.

 

5. For future surveys... do I need to be more specific about how much interpretation from the point cloud data is done by them, rather than me? In other words, is it a common expectation now that the survey company might simply provide the point cloud data, and it's up to me to turn that into a form that can, for example, be represented as a human-readable 2d site plan?

 

Obviously, what I actually complain about will take into account exactly what was agreed, and the fee paid, and so on, but it would be useful to get a sense of what most people would expect nowadays.

 

(I'm in the UK, if that makes any difference)

[Tom W.]

I have only ever had point clouds for the buildings not the site. So I get the topo survey in DWG format (two versions: a 2D file + a 3D file of the same geometry) and the building survey as point clouds. They have two guys going around taking spot heights for the topo survey + another guy doing a laser scan of the building/s.

 

The spot heights in the 3D DWG come in as 3D loci so I can create a Site Model directly from them.

 

The best Point clouds I have had are ones where I have cleared vegetation etc from around the building in advance + cleared the interior as well. Obviously this is only possible in certain circumstances but the more objects that get in the way of the scanner 'seeing' the building the more blanks you're going to get in the scan.

 

My surveyor surveys all his buildings using a laser scanner + either supplies the architect just the resultant point clouds directly or more frequently uses it himself to create the 2D plans/elevations/sections for the architect as most of the time that's what people want (they are not working in 3D or if they are they're more comfortable working from the 2D DWGs than point clouds).

 

 

[line-weight]

I'm now examining the point cloud they have provided.

 

There are quite a lot of parts where there appear to be two parallel-ish ground planes going on. Am I being paranoid in worrying that this means they have not aligned the results from the various scan points properly? Or is this quite normal and probably just representing areas with long grass or similar?

 

[Tom W.]

I would go back to the surveyor + ask about this to me it doesn't look good at all. This is the kind of thing I'd expect to see when I've done a lidar scan with my iPad not from a professional scan I've paid for.

 

Here are some images of a scan I had done a year or two ago:

 

 

 

 

 

 

 

 

[Tom W.]

BTW I've not sure if I've seen this 'officially' stated anywhere but Point Clouds don't seem to like perspective views. I always use orthogonal view on the design layer so it never affects me but on the odd occasion I've changed to perspective I've noticed the pixels go huge + it all looks crap.

[zoomer]

14 minutes ago, Tom W. said:

This is the kind of thing I'd expect to see when I've done a lidar scan with my iPad not from a professional scan I've paid for.

 

 

May be related to the Clip Cube narrow area.

Otherwise, yes, looks exactly like my iPhone Lidar attempts 🙂

[line-weight]

Hm.

 

Those screenshots I posted, they are from an imported version of the point cloud where I only imported a certain % of points (less than 50%, may even have been something like 10 or 20) because my computer couldn't cope with importing the whole thing.

 

But I think that should only affect the density of points, not the double-layer thing.

[line-weight]

2 hours ago, zoomer said:

May be related to the Clip Cube narrow area.

Yes to be clear, these are thin slices through the point cloud using the clip cube.

[Jeff Prince]

4 hours ago, line-weight said:

There are quite a lot of parts where there appear to be two parallel-ish ground planes going on. Am I being paranoid in worrying that this means they have not aligned the results from the various scan points properly? Or is this quite normal and probably just representing areas with long grass or similar?

 

That's not normal of a profession scan.  It looks like something done by a handheld device where they walked across the site and returned to the same spot (iPhone, etc).

When a tripod based scanner is used, the software typically cleans up and normalizes overlap between stations.

 

On 8/27/2024 at 5:48 AM, line-weight said:

5. For future surveys... do I need to be more specific about how much interpretation from the point cloud data is done by them, rather than me? In other words, is it a common expectation now that the survey company might simply provide the point cloud data, and it's up to me to turn that into a form that can, for example, be represented as a human-readable 2d site plan?

 

Yes!  I write a "survey needs" document for each project which clearly states the expectations of the data collected and how it will be delivered.  I oftentimes request things that are not typical of a standard architectural or property boundary survey.   Like tree locations with caliper, boulders and rock outcropping edges, 3D features (such as points, grade brakes, swale lines, TOS, BOS, etc), and photos of all kinds of things.  I've even asked my surveyor to put things on specific "classes" so I would have less processing to do after the fact.  A good surveyor will know how to execute all of this quickly and easily.

 

I also request a sample file representative to the quality I should expect prior to surveyor actually going to my site.  This gives me a chance to make sure their work is compatible with my workflow and make any adjustments prior to them going to the field.  After I've done one job with a particular surveyor, this is not needed again because we have a good workflow developed at that point.  Good communication usually produces great results.

[line-weight]

2 hours ago, Jeff Prince said:

That's not normal of a profession scan.  It looks like something done by a handheld device where they walked across the site and returned to the same spot (iPhone, etc).

When a tripod based scanner is used, the software typically cleans up and normalizes overlap between stations.

 

It does appear to have been done with a tripod scanner because you can see the tripod location circles dotted around the survey:

 

 

 

But my impression is that the measurements taken from each location haven't been properly aligned with each other. It's not just the doubled-up ground surfaces - this is a horizontal section taken through an area of vertical tree trunks (it's a slice about 500mm from top to bottom) and it looks to me like there are doubled images of many of the tree trunks, I'm assuming picked up from different scan locations but not successfully aligned with one another.

 

 

 

For example right in the bottom left ... is that three images of the same tree trunk? If so, these are the amounts (in mm) by which they are variously misaligned with each other and this seems way outside the level of accuracy that seems reasonable to expect.

 

 

[line-weight]

2 hours ago, Jeff Prince said:

After I've done one job with a particular surveyor, this is not needed again because we have a good workflow developed at that point.  Good communication usually produces great results.

Yes...in the past there were a couple of companies that I'd use now and again and would know what to expect from them.

 

Then, because of the nature of work I now do, 10-15 years have passed without me really needing professional surveys, and technology has moved on, hence this thread!

 

I don't even really know if it's normal to use laser scanning & point clouds to do topographic surveys. I am wondering if they are using a method that's not really intended for the purpose. Most of their work seems to be internal floorplans.

 

I didn't actually ask for a point cloud, just a topo survey plan in DXF form with spot heights and certain significant features marked. They gave me that plus the point cloud, but looking closely at the point cloud doesn't make me feel confident that the DXF plan they've given me is really accurate.

[Peter Neufeld.]

Hello,

Yes in this image if you look closely, you can actually see the shadow of the LIDAR scanner and yes the white circle being the area beneath the tripod that it can't scan. 

 

 

 

 

If you go to a horizontal view and isolate the points you should be able to get a reasonable Point Cloud just of the surface.

 

Then if you right click on it or go Model>Point Clouds>Extract 3D Loci from Point Cloud you can then get Vectorworks to do that and use the 3D loci as source data for the Site Model/DTM:

 

 

 

 

Note that there are a lot more loci where the tripod is which isn't right so that will need tidying up. I have put in an enhancement request that users can enter a grid setting instead, like 'every 500mm' rather than letting it calculate how it does currently on a graphical colour density basis it seems.

 

Jeff is correct in that communicating with the surveyor or scanning company would be worthwhile. It would seem, but I'm no expert, that your point cloud hasn't been done particularly well.

 

Cheers,

Peter

 

[line-weight]

1 hour ago, Peter Neufeld. said:

If you go to a horizontal view and isolate the points you should be able to get a reasonable Point Cloud just of the surface.

 

Then if you right click on it or go Model>Point Clouds>Extract 3D Loci from Point Cloud you can then get Vectorworks to do that and use the 3D loci as source data for the Site Model/DTM:

 

 

Note that there are a lot more loci where the tripod is which isn't right so that will need tidying up. I have put in an enhancement request that users can enter a grid setting instead, like 'every 500mm' rather than letting it calculate how it does currently on a graphical colour density basis it seems.

 

 

Thanks.

 

Yes, I agree that being able to extract in a grid of consistent density would be very useful.

 

In the case of my point cloud it's quite difficult to extract just the points that related to ground surface, because it is a steeply sloping site. So simply taking a horizontal slice doesn't really work. And while I can rotate the clip cube in plan, I can't rotate it around the X or Y axes (as far as I know).

 

But it seems to me that this process (extracting a number of useful 3d loci from just the ground surface) is something the survey company should do as part of what they provide.

[line-weight]

By the way... I have been watching the video below (now quite old) about methods of building a solid model using a point cloud as the basis.

 

At around 15.00 in, the working plane tool is used to pick up a vertical wall. But it is not explained what is actually happening here. How does the working plane tool decide what the surface plane is, when it's just looking at points? Does it somehow see an area of points and choose a best fit plane? It seems to me inevitable that however it does it, it's not going to pick up an exactly vertical plane which is probably what you want if you believe the real-world surface to be near enough vertical. And then any small errors are going to be liable to get magnified when you do (for example) an extrude. Am I missing something, or is there another method that allows picking up a plane in a more controlled/precise way?

 

 

 

 

 

[Tom W.]

2 hours ago, line-weight said:

By the way... I have been watching the video below (now quite old) about methods of building a solid model using a point cloud as the basis.

 

At around 15.00 in, the working plane tool is used to pick up a vertical wall. But it is not explained what is actually happening here. How does the working plane tool decide what the surface plane is, when it's just looking at points? Does it somehow see an area of points and choose a best fit plane? It seems to me inevitable that however it does it, it's not going to pick up an exactly vertical plane which is probably what you want if you believe the real-world surface to be near enough vertical. And then any small errors are going to be liable to get magnified when you do (for example) an extrude. Am I missing something, or is there another method that allows picking up a plane in a more controlled/precise way?

 

 

 

 

 

 

I think the video is slightly disingenuous. I think what Luis is doing is isolating a section through the stairs with the Clip Cube then using the right-click 'Set Working Plane' command to align a Working Plane with the face of the Clip Cube. I think in the video he initially clicks on the Point Cloud + sets a Working Plane there for purely illustrative purposes: it then cuts to a Working Plane set to the face of the Clip Cube cutting a section through the steps.

 

Using the Clip Cube + aligning Working Planes to it is integral to modelling from Point Clouds.

[line-weight]

1 hour ago, Tom W. said:

Using the Clip Cube + aligning Working Planes to it is integral to modelling from Point Clouds.

 

So, effectively trying to match the clip cube faces to the Point Cloud geometry, then using the face of the clip cube to set the working plane? Presumably this only works for faces that are truly vertical or truly horizontal - it would be good to know what the best workflows are when the faces are sloping for example.

 

I had a bit of a look around for video tutorials on converting Point Clouds into models in VW but didn't find all that much. I may just need to look a bit harder.

[Tom W.]

2 hours ago, line-weight said:

 

So, effectively trying to match the clip cube faces to the Point Cloud geometry, then using the face of the clip cube to set the working plane? Presumably this only works for faces that are truly vertical or truly horizontal - it would be good to know what the best workflows are when the faces are sloping for example.

 

I had a bit of a look around for video tutorials on converting Point Clouds into models in VW but didn't find all that much. I may just need to look a bit harder.

 

If you have leaning/sloping walls + want to model these accurately using 3D Solids then the way I've done it before is to take a horizontal section through the building just above floor level using the Clip Cube then place a Working Plane on the top of the Clip Cube to allow me to trace over the point-cloud-clip-cube-floor-plan using the 2D draughting tools. Then turn off the Clip Cube + Extrude the polys upwards + in a 3D environment use the Push/Pull + Taper Face tools to align the surfaces of the solids with the surfaces of the point cloud. Quicker + easier than it sounds + a pretty accurate way of modelling the actual conditions. I modelled a complicated timber roof structure this way, using the Deform tool to bend + twist extrudes to match the existing timber beams: the clip cube/working plane just gives you somewhere to draw a polygon that once extruded to give you a rough shape in the right general area you can then quickly manipulate with the 3D modelling tools in the point cloud environment to match the scan of the real object.

 

When you have 3D solids + a point cloud occupying the same space you can very clearly see how well the geometry is aligning with the scan + it's very easy to push/pull or taper or bend the surfaces to get them to match the surfaces formed by the points.

 

I don't think there's much out there in the way of Point Cloud tutorials but there is this one if you've not already seen it:

 

https://university.vectorworks.net/mod/scorm/player.php?a=621&currentorg=articulate_rise&scoid=1242

 

I don't remember the exact details but do remember I thought it was good.

[bcd]

Another nice alternative to this approach is to model 3d using the points directly:

 

As above find a suitable clip cube rotation & orientation to isolate your target wall / surface etc.

3d Polygon to trace the target surface clicking directly on the cloud

Modify>Convert to Subdivision

Extrude roughly the thickness of the wall.

Close the now open face of the wall.

Spin the model around and drag the SubD edges or vertices to align with the point cloud

[line-weight]

So - going back to the original survey I was provided with - the more I looked at it the more I could see there were quite big problems with it. I went back to the survey company and they fiddled with things and sent me a revised version of the point cloud. Actually we did that twice and it improved a bit each time but there were still various ghost images that were displaced by quite large distances. Eventually they accepted that it wasn't good enough, and went back to site to re-do it.

 

The new survey is quite a bit better. There are still places I can see displaced ghost images. Not sure if you'd always expect a little bit of that, or should a properly done lidar survey be completely clean?

 

Within the point cloud (imported into VW) there are areas where the points have colours that clearly aren't the colours of the objects scanned. Fore example blue points on an area of red tile-hanging and yellow and red spots within tree foliage. Does this indicate anything particular?

 

 

 

[Tom W.]

3 hours ago, line-weight said:

The new survey is quite a bit better. There are still places I can see displaced ghost images. Not sure if you'd always expect a little bit of that, or should a properly done lidar survey be completely clean?

 

If it's the same object scanned twice + in a different location each time then this is wrong + shouldn't be happening. All the different scans should line up perfectly.

[line-weight]

1 hour ago, Tom W. said:

If it's the same object scanned twice + in a different location each time

Yup this basically.

[Jeff Prince]

1 hour ago, Tom W. said:

 

If it's the same object scanned twice + in a different location each time then this is wrong + shouldn't be happening. All the different scans should line up perfectly.

agreed.  I have a large area scan where the surveyor had over 30 instrument positions, everything lined up perfectly with no duplication of points.

[line-weight]

Well, I don't think I'll be using these guys again.

[zoomer]

4 hours ago, Jeff Prince said:

agreed.  I have a large area scan where the surveyor had over 30 instrument positions, everything lined up perfectly with no duplication of points.

 

 

This sounds like, what you expect from Scans with Leica/Faro Scanners.

 

I could imagine, if you use other systems like from drones with Lidar of Photogrammetry

that this could happen because of small inaccuracies like lens distortions and such.

If you "re-scan" an area later again you might be off position.

 

I see it with my iPhone attempts here in an extrem manner.

No matter if Lidar point cloud or photogrammetry.

 

BTW,

iOS Lidar is said to be limited to 5 m. But that does not prevent it form shooting

random points anywhere 60 m away, e.g. when you scan a bit above a buildings roof.

Which is annoying for the point cloud result, where the point of interest would have been

only 10x5x5 m.

As AFAIR,

point clouds in VW, even when using clip cube, fit geometry will still zoom to the

complete overall point cloud.

Edited September 13 by zoomer

[Jeff Prince]

23 minutes ago, zoomer said:

This sounds like, what you expect from Scans with Leica/Faro Scanners.

 

I could imagine, if you use other systems like from drones with Lidar of Photogrammetry

that this could happen because of small inaccuracies like lens distortions and such.

If you "re-scan" an area later again you might be off position.

 

That's correct.  Drone using photogrammetry and ground targets will still demonstrate some distortion unless the surveyor does a great job of post processing.  The same job done on a tripod mounted scanner should be nearly perfect after post processing, assuming the crew set things up properly.