PDA

View Full Version : advanced data vis in lightwave, particles with attributes?



metallo
04-20-2010, 10:00 AM
i have a data set that describes the location of some items. the data set tells me when the item should appear and when it should disappear. the data set alson contains information i would like to use to scale the item over time.

my question is if anyone could point me in a direction as to how to get this information into a particle system. the size of the data set will be from 100,000 to over 1 million individual particles that need to each have their own scale and location, etc. each particle would end up being a cylinder or other type of object (sphere, disk, etc) and would grow or shrink over time.

if i were doing this with just a couple hundred objects, i would simply create motion data for each object. when dealing with thousands though, getting that many objects in lightwave isn't going to happen easily, so i figured a particle system might do the trick.

thanks

Red_Oddity
04-20-2010, 10:15 AM
You could try to convert the data set to a particle cache, Python allows for writing parsers quite easily.

Hope this helps a bit.

metallo
04-20-2010, 11:50 AM
ok, this sounds like a potential lead, however i know nothing of the particle cache format. i've been looking at python and learning it, i'm currently using visual basic and could learn python and write any file format as long as i know the specifications.

Lightwolf
04-20-2010, 04:40 PM
It's an automatic translation from Japanese to English, but it should do the trick:http://translate.google.com/translate?hl=en&sl=ja&tl=en&u=http%3A%2F%2Fwww.dstorm.co.jp%2Fproducts%2Flw8%2 Fdeveloper%2Fdocs%2Fpfx.htm

Cheers,
Mike

metallo
04-20-2010, 04:49 PM
lightwolf, thanks for that; however i've seen it already. problem is that it shows me a particle can have only a position, start and end. i'm hoping there would be a way to scale/rotate those particles as well.

on top of that, i want to associate geometry with those particles. i'm almost thinking the MDD file would be better for this but my current knowledge is limited.

as an example, imagine 100,000 cylinders that start out with a scale of zero (or they aren't born yet). then as time goes on, certain cylinders will be born....over time, the cylinders would grow and shrink, etc.

if the format could be understood, i could translate my ASCII data file(s) into whatever it needs to be. the other question is if lightwave could even handle something this complex, my suspicion is that it is possible.

i could do this probably with other software but my interest is in doing it inside lightwave so that i have the flexibility to combine animation elements, camera angles, and effects that other stand-alone tools can't accomplish.

Lightwolf
04-21-2010, 12:55 AM
lightwolf, thanks for that; however i've seen it already. problem is that it shows me a particle can have only a position, start and end. i'm hoping there would be a way to scale/rotate those particles as well.
There might be size associated with newer versions of PFX, certainly not rotation though.


on top of that, i want to associate geometry with those particles. i'm almost thinking the MDD file would be better for this but my current knowledge is limited.

as an example, imagine 100,000 cylinders that start out with a scale of zero (or they aren't born yet). then as time goes on, certain cylinders will be born....over time, the cylinders would grow and shrink, etc.

HDInstance. Anything beyond a few thousand items (as in 2-300 thousand) will be a massive challenge (if not impossible) for LW - especially once you add geometry to the items.

MDDs wont really help either, you might as well write a complete scene file then. But I suspect that'll just bring LW to a halt beyond 2K items.

Cheers,
Mike

Jarno
04-21-2010, 01:23 AM
There might be size associated with newer versions of PFX, certainly not rotation though.

PFX does store the initial rotation, size, and spin for each particle. But only the positions are saved for each frame.

---JvdL---

bjornkn
04-21-2010, 01:34 AM
I'm quite sure you could use hdInstance to show a million spheres/cylinders, and get them to animate/scale/rotate. You'd need one poly for each of them to be able to control scale/rotation,
But the problem is how to get those animated polys into LW.
Write a script that generates the object with animaton? MDD?
LW gets very sluggish with many objects, but can handle a million polys quite easily.

Red_Oddity
04-21-2010, 02:28 AM
You could create one object with the amount of polygons as needed for the data set object, you need to make sure all polygons are unwelded though so each poly can have scale and rotation described in an MDD file, that could work.

Then you can use that MDD object to instance your dataset objects with HDInstance

metallo
04-21-2010, 08:07 AM
i feel like the idea is coming together, red and bjornknt have summed it up i think. the questions i have now are what the exact geometry of that part would be and what format is the MDD file. i can definitely write a program to generate scene files, they're ascii. i could figure out .lwo files, the SDK describes the format (i think), and if i knew the MDD layout, i could write that as well.

here it a proposed workflow:
1) generate base object with 1,000,000 square (or cube?) polygons placed in space at the intended locations.
2) write an MDD file that would displace the verticies of those polygons as appropriate
3) load .lwo (with MDD applied) into a scene
4) purchase HD Instance and apply my desired geometry (sphere, cylinder, box, disk, etc) to the .lwo polygon instances

any further guidance will be appreciated

bjornkn
04-21-2010, 08:47 AM
Or
4) Make a smaller proof-of-concept version LWO/LWS/MDD and post it here for some of us who already have hdInstance to test run it?
No cubes in the LWO "particle" file, or else you'll get one instanced object on each cube face. Use quad or tri.
You can use weight maps do decide where to put different types of instance objects, like pyramids, cubes, spheres or whatever.

OTH, hdInstance is a must-have plugin anyway ;)

metallo
04-21-2010, 08:50 AM
proof of concept for someone to try is a good idea. can you please explain how the scale data would be passed to the instanced object if i have just a quad or tri polygon? is the scale based on the area of the polygon?

bjornkn
04-21-2010, 09:05 AM
The user guide is available online at http://www.happy-digital.com/instance2.asp
I would use Density type: Polygon. Then you get an instance in the middle of each polygon.
If you make each poly a trapezoid you could set each instance to point at the shortest edge. You could control scaling by a weight map. And you could tell hdInstance to scale either before or after base transform, which I believe would allow the instances to be scaled by scaling the base polys.
No randomness for placement, scaling nor anything else, as you'll provide all animation yourself.
I could make a little test scene later tonight, but right now I'll have to go away for a few hours.

bjornkn
04-21-2010, 11:59 AM
Sorry, but after a little test it turns out that the instanced objects will not scale with the base polygon, not even if each base poly is a separate object :(
Rotation works fine with a morph though, but that is only half of the solution.

metallo
04-21-2010, 12:05 PM
thanks for running the test, good information that prevents a wild goose chase.

bjornkn
04-21-2010, 04:06 PM
A little correction: The instances do scale with the base polygons, but only if the poly is scaled using the scale tool inside Layout, not when being displaced/scaled by an endomorph. Don't know if that helps much?
Maybe there are some other tricks/tools?

faulknermano
04-24-2010, 04:09 AM
Will single-point polys work? Assign a weight map on a per-point basis for scale control. And then an endomorph for rotation control. (Don't have HDInstance so I can't confirm the theory).

bjornkn
04-24-2010, 04:22 AM
The problem is that each object needs to have a controlled animated scaling and rotation. As long as weight maps are not animatable they won't really help other than setting the initial scale. If using points instead of polys hdInstance will lose the ability to control rotation with morphs or rotations.

Red_Oddity
04-24-2010, 02:38 PM
This really shows the area that old LW is lacking in, this is one of those things that is so easy in something like Maya with the particle system and the mesh instancer.

Trying to think of another way to do this other than generating a scene file with over a 100000 individual objects (something i can't test anyway since LW seems to crash or hang after a certain amount of items has been reached in Layout.)

metallo
04-24-2010, 02:41 PM
if i could do this sort of vis with maya, i'd give high consideration to purchasing it! i'll have to look into this, thanks for the lead red_odity!

i'm still dedicated to finding a LW solution though...

faulknermano
04-24-2010, 05:19 PM
You could ask Graham (dev of HDInstance) to provide some sort of solution, as I think the only true instancing solution in LW is in HDInstance. I'd suggest either an expanded, custom-built PFX file format that HDInstance could export from the LW particle system and read back into itself for use. Particles already have the necessary buffers within the system to write this off, and if the format could be designed, it could also be published so you can programmatically generate one yourself from your dataset.

Or, probably not as efficient, but possibly easier to implement: an extra external file that rides alongside the PFX file, but instead contains the necessary data such as rotation, scale, etc. This could be used by HDInstance as auxillary information about how to transform the instances.

The other only solution, which is probably not acceptable, is to generate scene files in segments, allowing for a upper limit to the number of items in the scene, and break them apart based on that. That way, you don't overload LW. You still have to comp them back in though.