Point clouds - How to get more then 100m points

[Bluetones]

How do people deal with the limitation of "only"100m points when importing point clouds? 

When I have too many points, I find that I lose a lot of resolution.

Do you use an external program, and which one? 
Do you slice up the original into pieces then import them all and then re-attach?

 

Or, am I wasting time thinking aboiut this?  🙃

[shorter]

Waste of time thinking Vectorworks, or any other CAD solution who's primary purpose is CAD or BIM, is the programme to handle point clouds.

 

There are dedicated softwares out there but tend to belong to the author of the point cloud, e.g. Leica, and these are built to process point clouds far better than 'normal' software.

 

Point Clouds are, no pun intended, pointless and it still fails me why surveyors continue to issue them to the design team expecting the design team to be grateful!

[Jeff Prince]

3 hours ago, Bluetones said:

Do you use an external program, and which one? 

 Autodesk Recap or Agisoft Metashape are pretty darn good, the latter being the gold standard for classifying and decimating point clouds.  There are a few people here using them and posts describing the advantages.

[zoomer]

2 hours ago, shorter said:

or any other CAD solution who's primary purpose is CAD or BIM,

 

I think Bricscad also does a great job with Point Clouds usage and Tools to make something useful from them.

[line-weight]

2 hours ago, shorter said:

Point Clouds are, no pun intended, pointless and it still fails me why surveyors continue to issue them to the design team expecting the design team to be grateful!

 

It's been a bit of a learning curve but I have now successfully used point clouds, issued by surveyors, a couple of times within vectorworks.

 

In future I may well ask for some surveys as pointcloud format only. When surveyors convert the pointcloud data into some other form, usually I just end up tidying up whatever they produce. I'd rather have the source data itself & convert it into a VW model myself. This will not apply for all projects or for all people, and the way VW handles them has several problems but I certainly wouldn't call them "pointless".

[Tom W.]

I think Point Clouds are indispensable. 

[shorter]

I should clarify... A surveyor issuing a point cloud without delivering a model or 2D data is pointless.

 

I am talking in the context of receiving information upon which to base the design.

 

Point clouds obviously have their uses (my wife uses them in historic building conservation), but to issue one as the 'survey' with no model or drawings, is pointless, and we see so many surveyors put a price to delivering a point cloud, and then charge extra to generate something the design team can use.  This is 'pointless'.

 

Edited March 1 by shorter

[Tom W.]

9 minutes ago, shorter said:

A surveyor issuing a point cloud without delivering a model or 2D data is pointless.

 

It isn't if this happens to be precisely what the designer wants! What's pointless is paying the surveyor to translate the scan into 2D plans/elevations/sections if you don't need/want them. People have different needs. Some would prefer to work directly from the source data rather than the surveyor's interpretation of it. I have no problem considering the point cloud the survey.

[Jeff Prince]

3 hours ago, Tom W. said:

 

It isn't if this happens to be precisely what the designer wants! What's pointless is paying the surveyor to translate the scan into 2D plans/elevations/sections if you don't need/want them. People have different needs. Some would prefer to work directly from the source data rather than the surveyor's interpretation of it. I have no problem considering the point cloud the survey.

Maybe @shorter is stuck on semantics?  A SCAN is just data, a SURVEY is a legal term pertaining to the interpretation of data that manifests itself in some kind of document.

 

People who do scans don’t have to be licensed, but Surveyors are like architects and civil engineers in most of the developed world, have to be licensed and accept professional liability for their services.

 

I’m with Tom though, if I only need a scan, I don’t want to pay for a survey.

[Christiaan]

I found it very useful to have a point cloud along with a topographical survey recently. We have some sensitive work to do next to a boundary and while the survey set the existing boundary wall out and included some spot levels, the point cloud allowed me to accurately model the entire boundary wall and gave us confidence in what we were designing.

[Tom W.]

13 hours ago, Jeff Prince said:

Maybe @shorter is stuck on semantics?  A SCAN is just data, a SURVEY is a legal term pertaining to the interpretation of data that manifests itself in some kind of document.

 

In that case would a topo survey delivered in .txt format fail to qualify as a 'survey' as well...?

[Tom W.]

Apologies @Bluetones none of this answers your original query. My point clouds have generally been delivered as separate files (interior of the building; exterior of the building; additional buildings/features) although this doesn't necessarily stop them exceeding 100m points individually. However I've never been aware of any loss of resolution because of this. The only time I experience this is when I try + import a Point Cloud into a georeferenced file (i.e. so it lands at its correct geographical location) which results in a vertically banded effect with half the points missing. The only option is the import the point cloud at 0,0 then manually move it to the correct location, which is obviously a highly unsatisfactory thing to do. But I've never had any feedback about this or found out whether it's a known thing or if there's anything I should be doing differently.

[shorter]

I think it's more about definition and liability.

Some appear to be using the Scan AS the Survey...

 

If you are building your own model from the scan, you are presumably then taking on the associated liability, whereas I am saying the surveyor should be doing the modelling/drawing and taking responsibility and issuing only a scan is pointless since we want a model or 2D drawings delivered by the surveyor.

 

The geometrical relaibility of the model, though, is not necessary the surveyor's problem, since it's accuracy is down to the specification they were given and the tolerances defined in the survey specification.  Ours is quite exacting in this regard, and in @Christiaan's example, we would have asked the surveyor to model the boundary wall.

 

Perhaps it's the size of project, but I have not worked on any project recently where the architect delivered the survey model.

[Anders Blomberg]

3 hours ago, Tom W. said:

Apologies @Bluetones none of this answers your original query. My point clouds have generally been delivered as separate files (interior of the building; exterior of the building; additional buildings/features) although this doesn't necessarily stop them exceeding 100m points individually. However I've never been aware of any loss of resolution because of this. The only time I experience this is when I try + import a Point Cloud into a georeferenced file (i.e. so it lands at its correct geographical location) which results in a vertically banded effect with half the points missing. The only option is the import the point cloud at 0,0 then manually move it to the correct location, which is obviously a highly unsatisfactory thing to do. But I've never had any feedback about this or found out whether it's a known thing or if there's anything I should be doing differently.

I've been bugging them about the georeferencing for years and believe it might finally be fixed in a not so distant future. 

[Tom W.]

52 minutes ago, shorter said:

I think it's more about definition and liability.

Some appear to be using the Scan AS the Survey...

 

If you are building your own model from the scan, you are presumably then taking on the associated liability, whereas I am saying the surveyor should be doing the modelling/drawing and taking responsibility and issuing only a scan is pointless since we want a model or 2D drawings delivered by the surveyor.

 

The geometrical relaibility of the model, though, is not necessary the surveyor's problem, since it's accuracy is down to the specification they were given and the tolerances defined in the survey specification.  Ours is quite exacting in this regard, and in @Christiaan's example, we would have asked the surveyor to model the boundary wall.

 

Perhaps it's the size of project, but I have not worked on any project recently where the architect delivered the survey model.

 

So what format does the surveyor supply the models in?

[line-weight]

1 hour ago, shorter said:

Perhaps it's the size of project, but I have not worked on any project recently where the architect delivered the survey model.

 

It very likely depends on size of project.

 

For small domestic projects I very often carry out the survey myself. Why? Because:

1) It gives me an opportunity to get my head around how an existing building is put together - I need to visit in person to do this anyway - so might as well spend a few additional hours taking measurements

2) I have an idea of what I actually need to measure in detail and what I don't, because I probably already have an idea of what the likely alterations are going to involve. Communicating this to a surveyor isn't always straightforward.

3) It usually ends up much more efficient for me to translate my survey notes directly into a useful VW model, than it is to take the surveyor's 2d or 3d drawings and then translate them into a model that's set up the way I want it.

 

There are downsides to this - one is that doing a survey with pen, paper and laser measure is somewhat tedious, as can be the process of converting this into a 3d model (which involves making decisions about how to correct or average inevitable measuring errors). Another is that yes, technically it means I take on some liability, but I am usually confident enough in my method that I will spot any errors. And in the real world, if a surveyor makes a mistake, for a small project how likely is it I'd actually be able to pursue them for any consequential costs?

 

Requesting a point cloud from a surveyor - which is what I'm increasingly going to do from now on, I think - addresses some of these downsides. It means I don't have to crawl around on site as much, or stand in the rain trying to aim a laser measure at the underside of some eaves. If it's done well, I don't have to go through the process of trying to work out the best way to correct the errors that come from measuring things that are a bit out of square. I have (most of) the info I need directly on the computer and I can translate it directly into the VW model setup that I want to use.

 

It also makes it easy to model parts of a building in very basic detail initially - and then revisit them later to add detail as necessary. Once the point cloud is in there, it's available for direct reference throughout the entire project.

[shorter]

If you want to DIY and produce the survey yourself you can hire a scanner and carry out the scan yourselves with a little technical knowledge, and drink copious cups of tea while it does its stuff!

Edited March 3 by shorter

[Bluetones]

3 hours ago, shorter said:

If you want to DIY and produce the survey yourself you can hire a scanner and carry out the scan yourselves with a little technical knowledge, and drink copious cups of tea while it does its stuff!

Part of my whole reason for asking was that I do the scans myself, using Matterports workflow. Scan, upload to servers to process, then download a Point Cloud.

Then I model the space and design as needed. (I work in the Entertainment side of things, so events, conferences and the like.)

There have been some times where the PC I am inporting taps out @ the 100m points, and I can see where there's missing information.

One time, it was significant, because I didn't know where a whole layer of pipes going across the ceiling were. Luckily I could see from the pictures I took that they were there and extrapolated. Less then ideal...

I have also hired surveyors to do the scanning and deliver me a 3d model. But my boss didn't like the cost and lack of control.

 

Thanks all for chiming in! 

[thinkingpencil]

I endorse line-weight's comments on Monday at 1.30pm. Exactly my own experience. Doing the survey yourself enables you to understand the existing building thoroughly, be selective etc etc....

 

...I've also started using Nomad on my IPhone to do my own point cloud scans using its LiDar function. Limited range. But still effective for early stage massing models on small domestics...and slightly weirdly....

 

...you can the client and then put them in the model!!!? Though this has happened only in my imagination so far.

[cberg]

@Bluetones are you using Matterport's lower-density .xyz files or purchasing the .e75 file? 

 

I do my own Matterport scanning and import fairly complex files on pretty much every architectural project.  The new Matterport3 camera is a game-changer, and if you don't want to purchase one, you can always rent.  I prefer working with slightly downsampled versions of the point cloud.  Sometimes there is striping, but not often.  Typically, however, I use the Matterpak .xyz files, and have never noticed incomplete imports. 

 

If you are doing a building + landscape, you can always work with multiple point clouds.  While building the model, you can always delete the landscape. Once your existing conditions are set, you can then delete the building and use the landscape as context.  Given my limited civil/landscape skills, it is much more efficient than building a site model.  

 

 

 

[cberg]

@Bluetones thinking about your question a bit more, the missing point cloud issue you discuss may have to do with the capabilities of your Matterport camera.  The Pro2 camera only has a range of 15', so the missing pipes may be the result of your scanning technology not capturing those areas.  I had similar issues when working in a warehouse space. The Matterport3 reaches the tops of 60' trees, and works in daylight.  

[Bluetones]

@cberg I have the Matterport 3, but it was a scan of a large warehouse with a bunch of shelving, so very dense.

Part of my question is how to manage the data coming in and to possibly pre-process it before VWX. Slicing into pieces so I can maintain detail without having to use the e75 file. Usually the .xyz file is sufficient for my needs.

 

My work is in the Entertainment end of things, and it is amazing how many locations have TERRIBLE "CAD" files of their own locations. I once did a scan, and found that the ballroom in question was 13 feet larger then the hotel said it was. 😐 And they had given me a jpg of their room, with a scale in the corner and said that was their CAD!

[cberg]

Sounds like you may be working a little beyond my direct experience. 

 

I might start by using a dedicated point cloud editing software, and figure out how to slice the data into reasonable chunks that calibrate one to another in your model.  Then I would bring the various point cloud datasets into my model, organized by layer, so you can turn the information on and off.  I think(?) the 100M limitation you discuss applies to each point cloud object, but I am not 100% certain.

 

Being a cheapskate, I would start with something open-source like Cloud Compare before purchasing other software to test out your approach. 

 

Regarding the accuracy of digital information, I have found that Matterport scans do have their own accuracy issues, so if dimensions are critical, take laser measurements.  Let me know how you resolve.  (Learning from other professions is one of the nicer aspects of using VW...)

 

 

Edited March 5 by cberg

[Bluetones]

Matterport has an excellant discussion on accuracy where they talk about the build up of of minor errors over the length of a space. So, yes, I also take laser dimensions, but just of the overall, to give a sanity check to the whole thing. 

[cberg]

Here is another approach I would test....  You might be able to split the data within Matterport.

 

https://support.matterport.com/s/article/How-to-Split-a-Matterport-Model?language=en_US