PDA

View Full Version : Exponential PFX Calc Time... and then she crashes



Chrusion
11-21-2012, 12:56 PM
Why does PFX calculation time increase exponentially for even the simplest of emitters and collision objects for simulations of many hundred frames?

See the attached scene.

800 frames results in under 250,000 particles (but each dies after 4 frames). Calculation starts out at 1/2 second/frame, progressively getting slower and slower (2 seconds at frame 100, 30 seconds at frame 300, 1 minute at frame 500, etc.) until at the end of 800 frames it's taking over 2 1/4 minutes/frame... all the while the emitter is doing nothing different (repeating 4 second 360 deg rotation) and particles are hitting the collider no differently than the first 120 frames.

NOW, after 4 hours of waiting for the above to finish calcing and then saving the Spray emitter's PFX file, I active the Splash emitter (parented to the Erase collider) and hit Calculate which IMMEDIATELY results in a CRASH. Yes, I've tried deleting the *.dynacache file in the Dynamics folder. No effect.


The attached scene is as simple as I can get this one part of 12 sims I need to calc, because it appears I have to do each one individually, saving the PFX particle motion file for each emitter one at a time before proceeding on to the next emitter sim as otherwise the system crawls to a halt or crashes immediately, like it's doing now, or a few frames into the sim.

I discovered that only one emitter can be used with one collision Erase object when a child Emitter is parented to the collider or at least LW crashed for me otherwise. As you can imagine, the scene becomes complex with many emitters and copies of the various collider objects. So, instead of doing all three rotating nozzle assemblies, I'm only doing the first two, which still results in 4 container Erase colliders (one for each spray emitter - two/nozzle), 8 paddle colliders, 14 particle emitters (4 sprays and 10 splashes), and 1 Bounce collider for all the child splash particles, which will be the only ones rendered.

I can get thru all this simulation crap if I knew how to prevent the calc times from skyrocketing over 800 frames AND if someone can figure out WHY the attached scene instantly crashes when scrubbing the timeline or hitting PFX calc.

Amazingingly, I was able to sim the first of the 8 paddle splashes. Took 12 hrs, but it finished without crashing. I figured if PFX can do something THAT complex then it should BLAZE thru the far simpler splashes against the container. Nope. Help.

Supplied, but not in the scene, are the paddle colliders in layers 4 and 5 of the PaddleEmitters object.

So much appreciated. Thanks.

PS: This is with a FRESH install of LW 11.0.3 x64, complete with new configs and rescanned plugin folders.

Brötje
11-22-2012, 05:53 AM
I downloaded it and take a look at it later today.

What I can tell you, is that PFX can sometimes fill up your RAM with ease. I once had 48Gb of PFX in my memory. Seems to be an error.

Anyway, I'll let you know if I can find something.

Chrusion
11-22-2012, 07:32 AM
I'm rebuilding the scene, well taking the existing full scene, and whittling it down to just the essential dynamic elements, similar to the previous attached scene, but this time with out parenting the colliders to the main objects and see if maybe the multiple levels of parenting I used to build the machine I'm animating and then parenting colliders as the last in the chain might be the cause of the crash. The only children will be the emitters.

Thanks for taking a look.

Chrusion
11-22-2012, 09:47 AM
So far, so good. No crashing. But this is doing the sim in 11.0.1 x32. For some reason, 11.0.3 x64 isn't recognizing any plugins in the Dynamics folders, so I don't have any FX tools and Bullet menu items on my layout interface. Even when I scanned the \support\plugins\CORE\dynamics\dynamics.lwp.bundle \Contents\Win64 folder that has the dynamics.dll file, LW says 0 plugins found in 1 file. No Dynamics category either in the Menu Editor. So, back to 11.0.1 I had to go. Calcs are 20 times faster without the multi-level parenting. 800 frames in 15 minutes instead of 5 hrs just for the simple spray emitter erasing against the container collider. The Splash emitter (child of collision) that was immediately crashing upon clicking Calculate, now is not, but the calcs times/frame are still logarithmically increasing past frame 300.

jeric_synergy
11-22-2012, 02:09 PM
I'd suggest Packaging that Scene and sending it to FogBugz so that NewTek Development can have a Real World example to test their code on.

I fear that their own test scenes are simply not stressful enough. (Just like their filenames are probably shorter than RW filenames.)

Chrusion
11-24-2012, 02:22 PM
Here's a preview of the simulation. 1,170,557 particles coming from 8, "free-standing" emitters, because the PFX file from each of the 8 separately calculated simulations (from 8 individual scenes) determines where the particles originate and die no matter where an emitter is located.

http://youtu.be/11u4LFsLhG4

jwiede
11-24-2012, 07:32 PM
I'd suggest Packaging that Scene and sending it to FogBugz so that NewTek Development can have a Real World example to test their code on.

I fear that their own test scenes are simply not stressful enough. (Just like their filenames are probably shorter than RW filenames.)
Agree 100%. Whatever's going on in that scene, there are at least a few substantial performance issues exhibited (w.r.t. parenting) that really ought to be fixed, and obviously the crashing behaviors are unacceptable in any case. Providing them with the scene gives them something to test against -- I tend to agree with jeric_synergy, there seems to be a lack of testing done with "real-world complexity" scenes, esp. w.r.t. particles / dynamics / FFX.

Chrusion
11-25-2012, 04:25 PM
I could give them a scene that has ALL 14 emitters, their 12 duplicate yet respective parent colliders, and any other associated objects, one scene with multi-level parenting and one without, and let them try to get LW to calc all the dynamics at once, resulting in around 2 million particles and collisions! And Oh having a goal of it take only an hour or two, instead of the 18 hrs or so it took me to do them one by one, separating out the various relationships into effectively 14 scenes (8 actual). AND not having a single particle leak out of the containment vessel at the 90 deg corners. AND where ALL particles collide and ALL collisions spawn children.

erikals
11-25-2012, 09:18 PM
Lightwave plugin - LiquidPack FluidParticle might be a solution... haven't tested it with many particles though...
http://www.youtube.com/watch?v=EQdaeXHSpTA
http://www.youtube.com/watch?v=CeDw3BfrsSE

Chrusion
11-26-2012, 06:57 PM
NICE! Speedy built-in self-collision and surface tension! What PFX SHOULD have been from the very start. Why are all the previous bells and whistles in LW left far behind the development curve in favor of adding new bells and whistles? HV should be up to 7.0 by now and competing with Maya voxels, turbulence fluid dynamics, etc. sigh...

Mr Rid
11-26-2012, 09:40 PM
... AND if someone can figure out WHY the attached scene instantly crashes when scrubbing the timeline or hitting PFX calc...

Because the SprayEmitter nozzle is set to Object Vertices, when it is not an object and it has no vertices to emit from. The scene really should not crash, but rather the emitter should just not emit.

jeric_synergy
11-26-2012, 11:11 PM
I could give them a scene that has ALL 14 emitters,.....
Please do.

- - - Updated - - -


Because the SprayEmitter nozzle is set to Object Vertices, when it is not an object and it has no vertices to emit from. The scene really should not crash, but rather the emitter should just not emit.
Or even better, "Object Vertices" should be greyed out.

Chrusion
12-14-2012, 09:02 PM
Here's the 2 particle heavy scenes out of 13 scenes total for this project, showing the result of upwards of 4 million particles calculated, with only about 1.5 million or so rendered as instanced objects (first scene) and clip mapped HV sprites (second scene). The last scene was the most complicated. To make life simple, I reduced the hero scene down to just the proxy collision objects, nulls, and emitters and split the scene up into 10 sub-scenes wherein the unused objects/emitters not needed for the particular sub-calculations were deleted.

For example, particles emitted by a single water jet spray mesh colliding with the container vessel collider proxy mesh were calculated (one scene per spray), its PFX saved, and then the child emitter of the container collider enabled and the impact splash particles calculated and its PFX saved. Rinse and repeat for the other 3 sprays (didn't do the rear most 5th and 6th set). Rinse and repeat each spray for each of the two sets of rotating paddles... collision of spray particles with one rotating paddle assembly at a time first, followed by calculating the child emitter splash particles off the paddle colliders second. Weeee!

ConcreteMixerInteriors_720_24p.mp4 (http://chrusion.com/movies/ConcreteMixerInteriors_720_24p.mp4)

One thing I noted with the child emitters is that they failed to emit when the pre-calculated parent particles (water spray meshes) collided with the flat upper walls of the container vessel. The curved bottom produced the best 'reaction.' So I experimented by adding more geometry slices to the flat wall polys. This didn't work, so with the extra detail, I made the flat polys ever so slightly uneven. Bingo! Child particles liked that and spewed forth as desired. I also noted that only about 10% of the parent particles that clearly collided actually spawned children, so I had to really up the amount of parent particles to get the needed density of children.

erikals
12-15-2012, 10:56 AM
Looks Great! :king: