Topografic points in modeler to surface?

[pekka varis]

I got these points from a company and they represent the topographic info about the hill that I work with now, I paint a skimap. But how can I convert these points to a sigle solid surface. I have no idea, and the closest thing I got was Particle Clone. I copied these pyramids to point so I got something but that’s a hard & punky hill to ski with : /

Please tell me how to make this right?

[zardoz]

if your computer doesn't crash this would be the way to go
http://www.ribsngibs.com/pages/CG/plugins/movePointOnBG.html

create a subdivided grid, place it over your point cloud, place the point cloud in some bg layer and the subdivided grid in the fg layer. Click on this plug and then click in the subdivided grid plane and it will conform to the points in the bg layer. But you seem to have a lot of points, so this may take a long time or a crash.

[pekka varis]

Thanks Zardoz, I´ll give this a try later today. I will separate the cloud about 10 piecies, to avoid the crash.

[pekka varis]

I just played a bit with that MovePointOnBG and I could not have any movement on foreground surface if I had only points as BG. I wrote to the ribsngibs.com if we can use this plugin with point cloud as bg layer. Thanks anyway man. Now i try the other solution I was helped in spinquad: render as one point polys with y-index gradient. Then take this image to photoshop, blur and use that image as deformation. WOrkaround, uhuh?

Pekka

[bjornkn]

I believe it should work if you use particle clone with a flat, centered plane instead of that pyramid to make the ground first. And then make a subdivided plane and use MovePoinntOnBG. I use MovePointOnBG a lot to make hd_instance forests etc, and it has never crashed on me. Itmay take a long time to finish with lots of points though.

[zardoz]

yep that's a good solution, and more precise than the "y-buffer" gradient. Tough recently I used this one to exactly this. I had a huge (millions of points) point cloud and I used hypervoxels in sprite mode to render an "y-buffer" gradient, then I displaced it in layout and back to modeler to tweak the scale. But you can also apply this map in modeler as a weight map and with the move tool choose weight map as falloff to get the same effect.

[pekka varis]

Great, I got it made today. I blurred & noised out the worst seams from pyramid y-gradient black&white images. I had to use it since clone particle in 10 parts (to avoid crach) took so much of my time already. Now the height is also correct, about 400m. Thanks zardoz and bjornkn!

[pekka varis]

Customer wants me to make this in much larger area, 15 km!! So all over :)

I want now to use PointOnBG plugin.

zardoz and bjornkn, I was told in spinquad to use single-point-polys. I think it is the best way to use a background with PointOnBG plugin. How do I make these? Do I particle clone them to point-cloud just like those pyramids?

[bjornkn]

I don't kbow what 1-point polys would do to help you here?
Have you tried SketchUp?
At http://www.crai.archi.fr/RubyLibraryDepot/Ruby/em_arc_page.htm there are at least 3 free plugins that might be useful for the job. Cloud.rb, Cloud_v6.rb and points_cloud_triangulation.rb

[zardoz]

For this I would probably go for the displacement again. Use hypervoxels to render sprites with a black to white gradient and use that as a displacement. I guess for something this big that's the best option.

[bjornkn]

Here's another tool for you: SimuTerra
http://www.compuneering.com/simuterra.php
It works great, and typically much faster than trying to make a huge terrain directly in SketchUp. It's not free though, but comes as a trial version.

[bjornkn]

I generally don't like working with such dense meshes when there is no need for them to be that dense. Often you have thousands and thousands of points and polys to describe a flat plain that could just as well have been modeled with one single face. And in the end you end up with a soft terrain that have no sharp edges at all. And it is equally detailed all over, which is usually not needed.
For this reason I usually prefer to hand model (semi-automatic) in SketchUp. I usually get 3D conour lines to work from, but the tehnique would be just the same with point clouds.
I then make a coarse zigzag line snapping to significant points (where there is a change of direction/slope). This is done much coarser in areas of less interest, or that will be covered by trees or other vegetation. In close-up areas I can make them just as fine as I want to.
When I have 20-60 line segments I skin them with the Sandbox tool "from contours" in SU, do some clean up, and continue with the next part.
Making a terrain like the attached one may take a few hours, but I end up with a useful terrain that has a lot of definition where I want it, and less detail in more distant areas. Instead of a 500,000+ poly model I end up with 13000 triangles that could probably be converted to 6500 quads in LW.
I could have just selected contour lines and run the tool directly, but then I'd end up with far too many polys for my liking. Or I could've made a 100poly model quickly, if that was what I needed. Very flexible, and with full control IOW :)

[pekka varis]

Your manual sketch look good, I tried to use the Cloud v6 plug-in sketch pro trial but it took 2h to import my 71mb / 2 500 000 point / 15km x 15km xyz format and in the end it crashed since i moved my mouse!! haaahhaa!!

I guess I can make it one more time with my trial 5,5 hours left...

Now since I have made already the ski-center importat parts, I think manually drawing the rest is one good practice. I check the simuterra too. I and as zardoz said, the y-gradient tech might be fastest right now.

[bjornkn]

I wouldn't spend the rest of my SU trial time by trying to import a 2.5mill point cloud! I don't think it will work anyway. Try with smaller files first.
That topo above contained dwg files with lots of data too, in hundreds of layers, so cleaning it up took a lot more time than making the topo. But it was still only a 25MB DWG file. And the area is only 600x600m.
What format is your point cloud source file?

[pekka varis]

It is the good old xyz -format. They used to make for molecular modeling but topo point clouds are sometimes transfered with that format. Inside it is only text like:
274657,235 374657,255 4657,234

I tried the example file that came with Cloud v6 plug-in, maybe I try the first xyz I used to make those pyramids at the start of this thread.

I just though that using PointOnBG plugin to "drop" flat surface to single-point-poly-point-cloud I can use more dense grid at important areas and looser in empty deserts.

[bjornkn]

Then it should be very easy to truncate the file and just use a small part of it first, to see if it works fine?
Could you upload a small part of it? It would be interesting to give it a try.

When using MovePointOnBG won't the points "fall through" if you have only single-point polys in the background? That's what they do if I move them off the edges of a mesh.

[zardoz]

I guess maybe you could try this first:
use 3Dem http://www.visualizationsoftware.com/3dem.html to import your xyz file and then convert to .dem
then use demscape http://amber.rc.arizona.edu/lw/demscape.html#Installation to open the .dem file in lightwave.
I don't know if this works because I don't have the xyz file to test it but you could try and tell us how it went.

[archijam]

Realistically. LW (CORE?) should be able to deal with this data in a more dynamic way.

See attached video for the landscape module of Vectorworks .. the contours are linked to the mesh and the contours can still individually be edited ...

http://download2.nemetschek.net/www_movies/VW12/land_com/land_com_02.htm

I really recommend watching the whole vid, even though it goes slowly .. it's from about 3 versions of Vectorworks earlier.

The implications for modelling go far beyond ArchViz, imagine product design using (animated-)sections, or medical imaging a la Volumed ..

To be fair, not many apps deal with this kind of thing well, or require plugins - but it would be a great way to market LW to the ArchViz sector..

[pekka varis]

You are right. One-point-polys do not work with PointOnBG plugin. So I am about to use the particle clone but with just 4 point polygons this time, the pyramids just made cloning slower.

Here is the small one, the first one 3,6 x 3,6 km point-cloud (+ single-point-polys) as object.

I tried to import xyz file (as .dat extension) to Sketch pro and it calculated the points right, but triangle step after that crashed the software. So no good, it can´t either export the points as obj.

[zardoz]

also from this tutorial I guess you can use ...arghh...Bryce to convert a grayscale tga to a .dem file and then I would use demscape to open the dem file.

http://www.sci.sdsu.edu/3dg/lab/

[pekka varis]

Wow, what a huge help you guys are giving!!

I must try that latest zardoz method, the 3DEM, before I go back particle-cloning and PointOnBG.

[pekka varis]

3DEM is by far the best and fastest to open the xyz format. I saved it as .dem object but now I have to play with lightwave (mine is old 8.0) becouse the demscape plugin crashes when I try to import the .dem

Anyway this seems to be the best method, I hope I can bring it in LW!

[pekka varis]

I tried all the combinations of DEMScape import plugin in my lw8 but all efforts ended with crashing it once I browse to .dem file. Just abnormal program error. I guess Vista, intel quad processors and lw8 are not compatible with this plugin. I tried to run modeler in XP mode too. Very sad here, since this would have been the ultimate way.

Here is anyway the xyz file (the 3,6 km small one) and also the .dem file I made with 3DEM if some of you have time to import it to LW with DEMScape.

Now I am back to particle cloning, but what the hell life goes on.

[zardoz]

lol you know what is funny? this opens another option.
I opened your xyz in 3dem and exported to terragen, wich is free and I had instaled (version 0.9something). Open in terragen and export to .lwo everything went fine.

The only thing is that terragen creates a grid like mesh, and I was trying with demscape because it creates a nice mesh with variable density.

[pekka varis]

Well, in meantime I tried this: chaged the 3DEM default colour to black&red and used this image to displacement. I was too jaggy, since 3DEM displays too little data in y as colour.

What if I try to reduce polygons from the mesh that Terragen generates, or is it clear that particle-clone and PointOnBG result in a better result? What do you think zardoz?

Anyway I will write to DEMScape developers a mail and ask what can they say about this crash of mine.

[zardoz]

well I've been doing some more research and I guess I found what looks like a nice solution to me.
So create a grayscale image from your point cloud (how did you import the xyz txt to modeler?), for this I would simply use the ortographic camera in layout, and sprite mode in hypervoxels (tweak the sprites size so you don't have holes), with a gradient from black to white related to the camera.
If needed blur the image in photoshop.
simply use the texturescape plugin from the same author of demscape
http://amber.rc.arizona.edu/lw/texturescape.html
he also has some importers
http://amber.rc.arizona.edu/lw/elevations.html
import your image to the texturescape plugin and press activate in the numeric panel. Now you can play with the settings. This plugin will give you a mesh with variable density.

[zardoz]

or you can use the .lwo from terragen to create this grayscale render (if it has enough resolution).

[pekka varis]

This is it! I can feel it. Sorry I must stop now and continue later.

[zardoz]

but how did you import the xyz .txt into lightwave?

[pekka varis]

It was done by a professional engineer here in Finland. I just wrote an email to him and told what we were doing. He used Autocad to extract 3D data from xyz and them he sended dwg to me. I used free dwg to dxf converter (rapid dxf I hope I remember this right).

But for this new 15km model he had to ask a collegue to split up the xyz becouse it was too huge for autocad! So they have made this work I do not know how many hours today, and free yes!

[bjornkn]

I tried to convert that LWO file to something useful for SU, but in vain.
The DEM file would not load into SU (it said it was wrong file format)
But the xyz file loaded fine into the Cloud_v6 plugin.
Very slow with a lot of data, but when I used only about 1000 points it loaded in a few seconds, like shown here as a slice of terrain.
Making 15km2 at that high resolution would not be possible in SU.

[zapper1998]

also from this tutorial I guess you can use ...arghh...Bryce to convert a grayscale tga to a .dem file and then I would use demscape to open the dem file.

http://www.sci.sdsu.edu/3dg/lab/

wow bryce dang i forgot i had bryce 5.5 hmmm

[MicroMouse]

I converted the xyz and dem files with AccuTrans 3D to lwo files for you.

Wayne

[pekka varis]

Thank you MicroMouse.

I have been working with a webpage design ovar weekend and today I continue this case. I am going the way zardoz found. But first downer was my lightwave 8 crashes when I render sprite hypervoxels, I can see them in layout but rendering crashes. So I go to my orginal method, particle clone flat boxes (no pyradimds any more) to points and then render the bw image. I got the 15km pointcloud in 3 parts, and first particle clone is about to take some 1h for modeller to finish...

Good news is I got a real life airplane photographer to take a lot of images over the area, there images help me a lot to play with the forest and other natures treasures.

[pekka varis]

I just made a decision to buy 9.4 so I get all the 64bit power out of my pc. I was clone particle-ing like 10 hours those polygons over the points few days ago.

I hope the update smooth out things a bit with this giant object.

I am going the zardoz way, and I hope the plug-ins work better with 9.6.

Is there a way to render the ortographic black&white image in a way that I get more y-information than the 256 levels of gray? Something like 16bit image or so? Also can I then after blurring it in photoshop, bring it back and use 16bit image in texturescape plugin.

In not, I ams still going this way :)

[archijam]

I converted the xyz and dem files with AccuTrans 3D to lwo files for you.

Wayne

Is there any reason why you are not using the (generously converted) accutrans file? Looks pretty good from here ...

[zardoz]

I guess that if you render to .hdr or .exr you'll get more than 256 shades

[pekka varis]

Archijam, I would love to use the conversion you made, but we have now a lot larger area to cover for this same project. I got one 71 Mb xyz file and one engineer converted it for free to 3 files, dwg point clouds.

If I pay you (is this even allowed here?) could you convert the xyz or dwg files to obj? It will be a huge obj and that is why the optimation in Zardoz method is so fine.

As I wait for your reply, I will play with 9.6 and this particle cloning. I also try to find some tutorials about .hdr or .exr displacement mapping.

[zardoz]

I mentioned .hdr or .exr as the file type export.
if you save your render in jpg or tga you're going to get a 24bit pic, but if you save in .hdr I guess you'll have more to play with.

also the .psd export in image filter has an option to export in 16bit export that will also give more than 256 shades.

[pekka varis]

What we are doing here is a good workflow study for all the out of "zoning" or "city planning" areas that animators might get to play with. These areas are at least here in Finland delivered only by this xyz format.


I am learning now the new node displacements. At least the realtime viper and normal texture gradients based on null object distance are great. I spent last time 30min finding the right gradient scale for the hill..

[archijam]

It wasn't me that converted it, it was Wayne, who is the developer of Accutrans3D (http://www.micromouse.ca/) - I mention it because it is a great app to pick up (20 USD) which converts far too many things into others:

Case in point:
I converted a several hundred meg KMZ file into OBJ - topography, buildings, trees - about a quarter of zurich - to LWO today. I have no idea how I could have done it without Wayne's fantastic little tool ...

[pekka varis]

Ok This 20$ software closes the cap. I got xyz-to-dwg converted by those happy engineers but I can´t relay on they help all the time.

I also tried the TextrueScape x64 and it is working with 16bit textures so we got enough resolution for the hills in Lappland 15km area. The best resolution for xyz topographic data in Finland is 10m grids.

Yesterday I particle cloned those flat polygons and viewed Ironman dvd, I am almost done with all the 3 objects and soon ready to render them to black&white heightmaps.

Maybe smaller point clouds would not lead to crash in 9.6 when using sprite hypervoxels. Anyway I am so far in this particle clone I don´t bother to try it right now.

One question arise: There is no adjustable optimation settings for amount of points in AccuTrans 3D so how could you use that AND TextureScape plugin together without this ortographic-camera y-gradient rendering step?

[pekka varis]

Ok, large steps I have token today. See the images, how modern could this Tunrda be turned? Thank god most of it is "safed area" so man can not build anything to it!

Now tomorrow I will study the export methods. Zardoz were right again with the exporting, I could not remember his words while at lauout; I rendered to 32 bit psd as output file, but what, all I got in photoshop was poor old 8bit image!

I am scared to move the black too close to the waterline since I am afraid of clipping. The large scale dynamics in gray is soooo important. I have also scaled the height in this point to 20 times larger in Y.

Max Headroom you remember?

[bjornkn]

Case in point:
I converted a several hundred meg KMZ file into OBJ - topography, buildings, trees - about a quarter of zurich - to LWO today. I have no idea how I could have done it without Wayne's fantastic little tool ...
How did you do that? I checked the Acutrans site, and it says that Collada can only be written, but not read?
Is there maybe a newer beta version available?

[bjornkn]

What exactly is your goal Pekka?
Are you going to make this 2.5mill poly terrain and cover it with trees and vegetation?
Is it going to look like the stepped/terraced render above?
Are you going to fly around this terrain, or just render still(s)?
Are you going to render some closeups?

I can't help but feel that you are on the wrong track. The data you get in may sound like it have a lot of detail, but with a 10x10m grid it doesn't really. If there is a road or creek there it may or may not receive a single point, but very unlikely it will get more than 1. Everything will be soft (or terraced); you wll most likely never see most of the polys anyway, and in the areas where you may want to focus there is not enough detail.
It sounds like overkill to me, and you still don't get enough detail where you need it?

Guess I'm lucky that here in Norway we can generally get such data as 3D contour lines ;)

[pekka varis]

Thank you for your support, everybody. Bjornkn, I hope I am still at the right track. The images tell it all!

texturescape.jpg - This was a good plugin but for this work the excellent polygon optimation was too rough. As subDit was smooth but way too heavy, and waterlines turner kinda odd. I recommend this plugin for just rough start before manual modelling work.
The maker of Texturescape, Marvin Landis wrote to me: "Depth would need to be shown as 32, because a floating point real number is represented in 4 bytes. So unless the LW image loader shows 32 as the depth, it is probably converting your 16-bit data" So it looks like it is working only with 8 bit height information for now.

accutrans3d.jpg and sub-skihill.jpg The solution for the orginal topic was, you guessed right, Accutrans 3D. So I bought this 20€ masterpiece. I imported the lwo and volume selecterd, copypasted to 6 layers and ran Reduce Polys+ two times with defaults.

Then I made one more layer from the hill that is important, and turned it to subD. I moved each point manually to match up as good as possible, the rest of the Tundra looks good as normal polygons with surface smoothing and bumpping. Anybody knows a good trick to get rid of the minimal seams in "snow" between the two objects like these?

waters.jpg - Then I drilled the water information witch I bought as a separate delivery from Lapland. From Illustration to eps8 and then drilling with Y, stencil and made water surface. This was the turning point in getting this project to professional enough. Ugh!

tundra1.jpg Here is the raw model. I made one downhill ski track to snow just to test the texture displacement and rendering times, but you can´t see it now :) Is it all very fast, and the model is quick to work with. Now I continue with the Tree topic.. See you there?

Bjornkn, trees next, yes. fly around this terrain, yes and laaaaarge stills for some 5m printouts. Closeups from the hill itself, Jyppärä. Zardoz, Thank you man!

[bjornkn]

It looks like you've got it up and running :)
What exactly were the steps involved so far, from you received the original 15x15km xyz files?
How did you get them into LW, and how did you get the skin on top of it?
And how many polys/points are there?
And now you're gonna smooth it all out with a cover of snow, and then add roads, ski tracks, buildings, thousands of hd_Instance trees, rocks, cars, people etc? Sounds like a big job :)
Please keep on posting updates!

[MicroMouse]

There is no Collada import module for AccuTrans. There might be one later this year.

Wayne

[bjornkn]

There is no Collada import module for AccuTrans. There might be one later this year.
Wayne
That's why I was surprised to see that it was used for converting a several hundred MBs KMZ (which AFAIK is Collada) file.
Accutrans looks like a powerful converter though :)

[MicroMouse]

I think the KMZ file was exported from Sketchup as an OBJ file.

Wayne

[pekka varis]

Here is the workflow:

1. Opened xyz in Accutrans 3D as "scan" type.

2. Saved it as LightWave 2 (huge objects) lwo format.

3. In Modeler I scaled it up few thousand %

4. Used volume selection mode to slice it to 6 layers, otherwise it would be too heavy.

5. For each piece, I run Reduce Polys + two times in a row with defaults.

6. cut out the important hill to 7th layer and made it SubD. I run Reduce Polys + (not in SubD mode) until It was light enough to view fast in perspective view but not too far from the orginal shape.

7. Went through each side of this hill and manually moved the points to get rid of seams. movePointOnBG maybe would have been a good idea to use here.

8. Got the dfx waterlines from Lapland, I cut in Illustrator the holes for small isles. Saved the file as Illustrator eps 8.

9. Opened it in modeler with import EPSF loader, standard curve division, closed polygons, auto axis drill on. This was slow to open, like 1h or so.

10. Scaled the waters over the ground to align perfectly.

11. Volume selected the right water to each of 7 layers and copypasted only those waters to own layer. Only parts of water-polygons from each these water-layers could be drilled at time. So I selected something like 10-20 lakes at time, or only one long river, and used Drill, Y, Stencil, surface: water.

12. I had to triple and use Reduce Polys + to some of the long rivers, since drilling gave me errors. But only for 5-10 objects with these waters.

Thas was it! I will not make any other snow surface as object, I will manage all that with textures and materials. Ski Hill itself will use displacement as the image shows.

[bjornkn]

Well, unfortunately although SketchUp writes OBJ it doesn't read KMZ files. SketchUp is my main modelling app for architectural/topo work, but I would never try to load hundreds of MB of geometry into it. As all SketchUp files contain all images inside the skp file itself the files may get quite large even if they contain little geometry.

I'm not trying to be difficult here, but would really like to know how to handle/convert such KMZ files into LW. Currently my main pipeline from SU to LW is via 3ds format, which is not really optimal, with all those triangles and 8.3 file names etc, but at least it usually gets the geometry and UV/textures through. So far I have had little luck using other export formats, like FBX, OBJ and Collada for importing into LW. I can always get the geometry, but usually lose the UV mapping and textures,
If I could export as KMZ and then convert to LWO (using Accutrans?) that would (probably) be great :)

[bjornkn]

Thanks :)

Here is the workflow:

1. Opened xyz in Accutrans 3D as "scan" type.

So then you got the skin/polys already in the first step?

How many polys/points do you have now then?

[pekka varis]

Yes, I got skin/ polys right away.

Now I got 1146923 polygons with waters drilled in.

[bjornkn]

So, how is it going Pekka?
Any WIPs to show us?
It will be very interesting to see how this ends up :)

[pekka varis]

bjornkn, I am planning a aerial shot with the customer to get more data for visualation:

http://www.catchline.fi/login/hetan-ilmakuvaus/

Check out the topic "Management of trees in huge landscape", I will continue my work there.

Pekka