PDA

View Full Version : Why does Layout recalculate dynamics?



DrStrik9
01-30-2016, 10:13 AM
I've been wrestling with this for three days. (2015.3) Bullet is being more that a little bit wonky. I spend almost 12 hours waiting for dynamics to be calculated, save the scene, note that the Dynamics is also saved.

Restart Layout, open scene, and it is once again calculating dynamics! WHY?

Also, after exporting MDD, NOTHING is there in the MDD directory.

This is VERY frustrating.

DrStrik9
01-31-2016, 07:31 PM
I think maybe this is what happens when you make your bullet sim too complex. I sure would like to know how to tell where that boundary is ...

lino.grandi
02-01-2016, 02:46 AM
I think maybe this is what happens when you make your bullet sim too complex. I sure would like to know how to tell where that boundary is ...

This shouldn't happen, whatever the complexity of the scene is.

Bullet only gets recalculated if something related to dynamics changes in the scene. Moving or replacing an object is making that happen.

It would be nice to get a report with content, so we can take a look at the scene and verify the workflow.

jboudreau
02-01-2016, 01:03 PM
I've been wrestling with this for three days. (2015.3) Bullet is being more that a little bit wonky. I spend almost 12 hours waiting for dynamics to be calculated, save the scene, note that the Dynamics is also saved.

Restart Layout, open scene, and it is once again calculating dynamics! WHY?

Also, after exporting MDD, NOTHING is there in the MDD directory.

This is VERY frustrating.

Hi DrStrik9

You should have a folder called Dynamics in your root folder of your scene. In that folder there should be a file that is named the same as your scene with the extension dynacache for example "yourscenefilename.dynacache" What you can do is after you run the simulation save your scene and close layout. Go into that folder and copy that file yourscenename.dynacache file somewhere else basdically make a backup up of it I would also make a backup of your scene file too just in case you accidentally moved an object by mistake etc. This way if your simulation ever breaks again and you haven't changed, moved or replaced anything you can take the file and just overwrite the same one that is in your dynamics folder. Now when you open up the scene you will have your dynamic simulation back and you won't have to run it again.

Another way of doing it is once your simulation is done, save the scene, and then save the scene again under another name, This way if your scene gets messed up, anything moves you can always revert back to your previous scene and your simulation will already be cached.

Hope this helps
Thanks,
Jason

squarewulf
02-01-2016, 01:20 PM
Also, after exporting MDD, NOTHING is there in the MDD directory.

Make sure you append the .mdd file type to the end of your file name in layout. I have named my motion file and accidentally deleted the .mdd and layout was not able to find it again.

meshpig
02-02-2016, 01:43 AM
Save the scene with dynamics off and it tends not to on restart.

DrStrik9
02-02-2016, 05:06 PM
This shouldn't happen, whatever the complexity of the scene is.

Bullet only gets recalculated if something related to dynamics changes in the scene. Moving or replacing an object is making that happen.

It would be nice to get a report with content, so we can take a look at the scene and verify the workflow.

Nothing was changed in the scene, except the scene was saved after dynamics calc. This is why I asked why it was recalculating.

- - - Updated - - -


Hi DrStrik9

You should have a folder called Dynamics in your root folder of your scene. In that folder there should be a file that is named the same as your scene with the extension dynacache for example "yourscenefilename.dynacache" What you can do is after you run the simulation save your scene and close layout. Go into that folder and copy that file yourscenename.dynacache file somewhere else basdically make a backup up of it I would also make a backup of your scene file too just in case you accidentally moved an object by mistake etc. This way if your simulation ever breaks again and you haven't changed, moved or replaced anything you can take the file and just overwrite the same one that is in your dynamics folder. Now when you open up the scene you will have your dynamic simulation back and you won't have to run it again.

Another way of doing it is once your simulation is done, save the scene, and then save the scene again under another name, This way if your scene gets messed up, anything moves you can always revert back to your previous scene and your simulation will already be cached.

Hope this helps
Thanks,
Jason

Good idea on saving two versions of the scene. Also per meshpig, turn dynamics off (and place cursor at frame 0) before saving.

DrStrik9
02-02-2016, 05:25 PM
Make sure you append the .mdd file type to the end of your file name in layout. I have named my motion file and accidentally deleted the .mdd and layout was not able to find it again.

Not sure about PC, but on Mac, you don't choose the file name or the file type, only the MDD location. I have always manually created a directory called MDD within the content directory.

DrStrik9
02-02-2016, 09:15 PM
There sure are a lot of ways to screw dynamics up in 2015.3. :-(

For the last several hours, I've been trying to change the activating keyframe of a single object in 2015.3. Even if I turn dynamics off, remove the item from the dynamic list, resave the scene under a new name, change the keyframe, then add that object into dynamics again, it ALWAYS reverts to its OLD keyframe when calculating dynamics. This is just wonky. That old keyframed position is not in the scene, and yet, there it is, raising its ugly head, time after time.

This is extremely frustrating. What am I missing?

If I turn dynamics off after this, I can see BOTH object positions in the scene when at the new keyframe, as if it is duplicated. (There is only ONE of that object in the scene, and only one keyframe for it.) But I'm looking at TWO objects. One is at the old position, and the other is at the new position.

I thought maybe Layout has possibly bombed. So I resaved the scene one last time, and quit Layout. After restarting Layout and loading the newly-saved scene, and activating dynamics, the object is ONCE AGAIN at the old position, and when deactivating dynamics once again, the object is once again at BOTH positions on the single keyframe.

Resetting dynamics does nothing.

I give up.

DrStrik9
02-02-2016, 09:54 PM
(I never give up.)

I restarted my computer and replaced LW's Prefs from a recently backed-up set. Unfortunately, there is no change in this bizarre behavior.

I checked the Dynamics directory in the Content Directory, and the dynamics files from the last two saved scene versions are NOT THERE.

So WHERE are the mysterious unchangeable dynamics coming from? -- I also checked the MDD from an earlier scene, but MDD is not assigned in this scene.

The same bizarre dynamics behavior persists. This is really confusing.

--

Bottom line is that there is apparently no way to do successive minor changes to a dynamic scene, and then recalc the dynamics.

This means that all the subtle nuances and minor changes typically needed in developing a scene are not possible. With dynamics, you get one shot, and that's it. To make changes, you apparently need to start over completely.

I've attached two screen shots, showing this weird behavior.

132181132182

I hope someone can think of why this might be happening: something I haven't thought of or tried yet.

jeric_synergy
02-03-2016, 12:09 AM
Dynamic caches are stored automatically, no? If so, they must be somewhere. Can you do a time based search of your entire system to attempt to locate where they are being stored?

DrStrik9
02-03-2016, 11:33 AM
Dynamic caches are stored automatically, no? If so, they must be somewhere. Can you do a time based search of your entire system to attempt to locate where they are being stored?

Thanks for the suggestion. Unfortunately, there are no other related dynamics files on this system. Plus, dynamics are stored within the Content Directory, or "should" be.

The only other thing I didn't investigate before was the possibility that this scene (along with the one before it which also had strange dynamics problems) might be corrupted. So I rebuilt it from scratch.

Apparently, the two scenes before this one WERE corrupt, because this one works as expected.

Finally. Basically several days wasted, but victory at last.

jeric_synergy
02-03-2016, 06:28 PM
Better you than me, mate. ;) --Are they correctly storing in your Content Directory now?

DrStrik9
02-03-2016, 11:01 PM
Yes, but MDD is still a bit tricky. I can't use one MDD directory for multiple scenes. Each scene in a content directory MUST have its own MDD directory, even if the scenes are named differently. Otherwise the dynamics get confused and unworkable.

Things I will forget in a week. :+)

jeric_synergy
02-04-2016, 12:58 AM
Things I will forget in a week. :+)
Yes, why is it that we can't remember a couple hundred little 'gotchas' anyway? What's up with that? :devil:

That is truly sucktastic: why even HAVE a filename if the app is just going to get confused? Have you reported it? (Wondering if TPTB will confirm it as a bug.)

jwiede
02-05-2016, 12:28 PM
Have you reported it? (Wondering if TPTB will confirm it as a bug.)

DrStrik9, as Lino requested earlier, both scene files, related files, etc. need to be packaged up and sent to LW3DG as part of a bug report. Don't just post photos of the scene here, that isn't enough data for LW3DG to debug such issues. This situation definitely merits a bug report so it gets fixed. Even if the scene/content are commercial and private, it should still be possible to privately send LW3DG the content needed to debug, so please contact them if that's what is holding you back from filing.

DrStrik9
02-06-2016, 10:18 AM
I agree that this needs to be reported, but it's pretty difficult to package a scene when just opening it causes Layout to crash. I'm just not going to spend yet another two days attempting to recreate it for a third time, and have it corrupt itself again, sorry.

jboudreau
02-07-2016, 02:21 PM
I've been wrestling with this for three days. (2015.3) Bullet is being more that a little bit wonky. I spend almost 12 hours waiting for dynamics to be calculated, save the scene, note that the Dynamics is also saved.

Restart Layout, open scene, and it is once again calculating dynamics! WHY?

Also, after exporting MDD, NOTHING is there in the MDD directory.

This is VERY frustrating.


I've been playing around with dynamics most of the day today and noticed a couple of things that cause the dynamics to recalculate. If you have objects in your scene that are in the dynamics list even though they are shut off (which means they are not calculated or used in the simulation) they still cause the dynamics to recalculate. WHY? This makes no sense to me. For example if you have an object in the dynamics panel that is there but turned off. If you delete that object from the scene even though it has not effect on the bullet dynamic simulation it causes the scene to recalculate. If you remove the object from the list even though it's off it also causes the dyamics to recalculate. I'm not sure if this is how it's suppose to work, but if it is it really doesn't make sense to me

Lino, anyone can you please explain why this happens?

Thanks,
Jason

jeric_synergy
02-08-2016, 11:59 AM
Looks like some book-keeping issues. Assuming there's a flag on OFF/ON dynamics objects, when it's OFF recalculation should be skipped, of course.

Probably the easy (programming) path was taken and a RECALC order is initiated, rather than just going "oh, never mind", which would possibly take some coding. Or rather than re-ording data storage (matrices of points, for example) a RECALC is ordered, even though it winds up with the 'same' effect. There's probably a dozen other feasible reasons.

Surrealist.
02-09-2016, 02:50 PM
I agree that this needs to be reported, but it's pretty difficult to package a scene when just opening it causes Layout to crash. I'm just not going to spend yet another two days attempting to recreate it for a third time, and have it corrupt itself again, sorry.

You do realize that these guys can actually look at the scene file in text editor, right? All you have to do is send the scene and objects in the respective folders. I'd be willing to bet they can find the line or lines of code in the scene file that is making it crash and then open up the file. Once they set the directory all of the objects will load. No need to recreate everything again. :)

jeric_synergy
02-09-2016, 04:14 PM
Actually, NO Scene file should even be able to make the application crash: if it does, something is severely lacking in the scene parser.

It's like "a paragraph that drives you insane" -- can't exist, your mind rejects it. I call such files "poison files" (some forum member really objects to that), but the app should be robust enough to detect poison files and reject them. Better is to be able to partially load a poison file, and reject the rest of it without crashing, because that can save some poor user a lot of labor.

Surrealist.
02-10-2016, 01:09 AM
Yeah I am not sure how it all works behind the scenes. And you could have something there.

But all I know is both Maya and LightWave have an ASCII format for a reason. Maya it is binary by default. But you have the option to use ASCII which is obviously better in these circumstances. I have recovered more than one Maya ASCII file from crashing Maya, on open attempt, by identifying and deleting the line that was making it crash. I think it was in this thread or another that I posted a link to a tutorial on how to do that.

And the very reason it is set up this way is so that you don't have to spend 3 days rebuilding a scene. By contrast you can find the line (or lines) in less than an hour.

And as I mentioned before it would also go a long way in helping devs identify the problem.

And if you don't want to be bothered. At least send them the file. I don't think that is too much to ask.

DrStrik9
02-10-2016, 04:47 AM
Well, I really hate to report this, but now the same large-sim scene that crashes 2015.3 (and painstakingly rebuilt in 11.6.3) is now also crashing 11.6.3.

I've been rendering various portions of it for the last 30 or so hours. I finished rendering two of four parts (same sim, different cameras), and after the first two were finished rendering, the scene failed to respond with dynamics on or off. Dynamics is the problem: it just quit working. I did NOT re-save the scene, but rather simply restarted the computer, and restarted Layout 11.6.3, and now that scene crashes Layout 11.6.3 also upon opening the scene! :-(((

I had made NO scene changes. I only chose successive cameras, and rendered.

I have no idea what could be causing this problem with Bullet in BOTH 2015.3 and 11.6.3, but right now I'm just about ready to take up knitting instead of 3D.

Just call me the "scene destroyer." :-(

--

When I stop steaming over this (if I ever do), I will definitely send BOTH scenes in to LW3DG.

Kaptive
02-10-2016, 06:15 AM
: (
I hate hearing people have problems like this, I've had a few myself and it is so disheartening... so I feel your pain. Would loading from scene into a fresh one not solve it?..
I'd assume if you have to do that, then you'd have to recalculate the bullet physics again too... so not ideal, as you wont get the exact same results.
Beyond this simple fix, I'm afraid I have no further insight beyond what has been said already.
Out of interest, when you try to load the scene, have you taken note of what is being loaded when it crashes? Layout seems to load objects first, then images. I've had an image break a scene and cause crashes, even though it loaded in and worked fine first time. Changing its format and replacing it solved it.
Can you open a pre bullet version and package the scene up so that you are working on a seperate isolated version? Then copy the dynamics version of the scene (with the solve) over to the packaged folder and try to load it again. I know this might sound all a bit stupid, but sometimes this kind of thing can be solved by just shaking it all up a bit... stripping it back and isolating it away from the (what is probably a very full) multi project folder.

Anyway, just thoughts out the top of my head. Worth a shot if nothing else... but it is still very annoying that this kind of thing happens at all. As said above, it's only a text file, and some sort of scene file checker/fixer should be built into the load sequence (with a dialogue that pops up afterwards to tell you what broke and has not been loaded). Anyways, good luck! ...and keep positive if possible!

Surrealist.
02-10-2016, 10:38 AM
When I stop steaming over this (if I ever do), I will definitely send BOTH scenes in to LW3DG.

lol if you are anything like me you will. I mean you'll stop steaming. Nothing more frustrating to me than these kinds of issues. But go and kick a few things around, go out back chop some wood, whatever it takes, lol... but then come back and send the file to support.

DrStrik9
02-10-2016, 10:56 AM
lol if you are anything like me you will. I mean you'll stop steaming. Nothing more frustrating to me than these kinds of issues. But go and kick a few things around, go out back chop some wood, whatever it takes, lol... but then come back and send the file to support.

Yup. :+)

I sent in both corrupted scenes, (2015.3, 11.6.3) with a short horror story.

The crazy thing is that in 2015.3, I never even got a chance to try a second sim, with setting variations. But in 11.6.3, I did about a dozen variations, and then rendered for about a day. I just can't fathom what could have happened to the scene just by rendering. -- Unless in both cases, there is a slow leak somewhere.

In 2015.3, bullet is definitely wack. But it takes awhile apparently with a large sim, to get 11.6.3 to also go mad.

DrStrik9
02-10-2016, 11:36 AM
https://www.youtube.com/watch?v=gU1QWoV0EBY
https://www.youtube.com/watch?v=zTxekW52uzc
https://www.youtube.com/watch?v=IPayi38vQws
https://www.youtube.com/watch?v=-6Sl5CCxp3Q
https://www.youtube.com/watch?v=rsvxFkGqxZA

They must be able to keyframe the activation state. Wouldn’t that be nice.

My sim was only 31,500 “planks” (cubes), and it destroyed both 2015.3 and 11.6.3.

Surrealist.
02-10-2016, 11:51 AM
Scene destroyer! :D

DrStrik9
02-10-2016, 12:09 PM
Scene destroyer! :D

Done. :+)

DrStrik9
02-10-2016, 12:46 PM
I've had an image break a scene and cause crashes, even though it loaded in and worked fine first time. Changing its format and replacing it solved it.

Thanks for that. This idea resonates with me. I don't know why, but in this scene I used .png files for the UV maps. I just swapped them out in 2015.3 for .psd's of the same images (color, normal, specular), and am presently calculating the sim, which "seems" to be going a LOT faster than the previous fourteen hours (in 2015.3). It is presently at 80%, after only 30 minutes.

I'll let you know how that goes.

Dang, I can't BELIEVE I'm still working on this! Call me crazy.

Kaptive
02-10-2016, 02:24 PM
Anything that requires calculation is always the worst in programs because it is totally out of your hands.... normally leads to me staring at a progress bar until it is finished in (mild) nervous anticipation. Though, I'd maybe draw the line at watching it for 14 hours :)

Fingers crossed for you!

DrStrik9
02-10-2016, 03:29 PM
OK ...

It "may" be png's that caused the hellish problems. After swapping them out for psd's, the sim took 73 minutes to calc in 2015.3, as opposed to 14 HOURS before in 2015.3, and TWO hours in 11.6.3, both with png's.

Also, I first did the sim to frame 360, which took 41 minutes, and then did it to frame 480, which took 32 minutes ... apparently, the sim used the first 360 frames in its second calc (!!) -- this was not possible before. (That's where I got the 73 minutes total.)

I haven't tried changing dynamic settings and recalculating yet, but that's next ...

QUESTION: WHY does it take TWELVE MINUTES to save the scene? The scene itself is only 92 KB in size. All it has to do is REFER to the objects, pre-calculated sim, and images. It spends a full 12 minutes (on Mac) showing the spinning pizza of death, and then it saves. This just makes no sense to me.

Next: change dynamics settings and re-calc. If the scene survives this in 2015.3, then I'm almost 100% sure the issue was with the png's.

I've saved over the scene three times so far, with no bombs, so things are looking up. :+)

More to come ...

jboudreau
02-10-2016, 03:50 PM
OK ...

It "may" be png's that caused the hellish problems. After swapping them out for psd's, the sim took 73 minutes to calc in 2015.3, as opposed to 14 HOURS before in 2015.3, and TWO hours in 11.6.3, both with png's.

Also, I first did the sim to frame 360, which took 41 minutes, and then did it to frame 480, which took 32 minutes ... apparently, the sim used the first 360 frames in its second calc (!!) -- this was not possible before. (That's where I got the 73 minutes total.)

I haven't tried changing dynamic settings and recalculating yet, but that's next ....


Did you try running the simulation without texture images mapped on to your geometry. Then once the simulation is done you can texture your objects. Also as far as I know you can always stop the simulation anywhere during the sim, You can then run the simulation again and it will start simulating from where you left off. So for example in a 300 frame simulation, run it until 150, abort the simulation and then click on the 300 frame it will now calculate from 150-300. I'm not sure why it wasn't possible before it's always been possible to do this, I do it all the time.

It takes a while to save the scene because it's saving a dynacache file in your dynamics folder where your scene is located ( if you don't have a dynamics folder in your scene folder then it will save the files to where ever you have your custom path set to) These files are really big depending on how long and complex the simulation is. Go into your dynamics folder where your dynacache files are and see how big they are. I'm guessing they are gb's in size.

Glad to see you got things working

Hope this helps

Thanks,
Jason

DrStrik9
02-10-2016, 05:26 PM
Actually, the long saving time does make sense. This way, we can save successively-numbered scenes, and still have access to the dynamics. Thanks.

Kaptive
02-10-2016, 05:58 PM
OK ...

It "may" be png's that caused the hellish problems. After swapping them out for psd's, the sim took 73 minutes to calc in 2015.3, as opposed to 14 HOURS before in 2015.3, and TWO hours in 11.6.3, both with png's.

Also, I first did the sim to frame 360, which took 41 minutes, and then did it to frame 480, which took 32 minutes ... apparently, the sim used the first 360 frames in its second calc (!!) -- this was not possible before. (That's where I got the 73 minutes total.)

I haven't tried changing dynamic settings and recalculating yet, but that's next ...

QUESTION: WHY does it take TWELVE MINUTES to save the scene? The scene itself is only 92 KB in size. All it has to do is REFER to the objects, pre-calculated sim, and images. It spends a full 12 minutes (on Mac) showing the spinning pizza of death, and then it saves. This just makes no sense to me.

Next: change dynamics settings and re-calc. If the scene survives this in 2015.3, then I'm almost 100% sure the issue was with the png's.

I've saved over the scene three times so far, with no bombs, so things are looking up. :+)

More to come ...

That is great news!

Just regarding the save time, as jboudreau says, it's saving what is probably many Gbs of data. In fact, I think... actually, I just tested it so... for certain, if you save a scene increment (as you normally would as good habit!) the downside with a bullet scene, is that it will save an incremental bullet dynacache too! So, if your physics solve is as big as you say, then it is probably massive.. meaning that every time you save an increment, it'll be saving a Gig or two for bullet. So watch that Harddrive space! If you have plenty, then it's not really that important, but if you ran out mid save, then it'd probably corrupt and not tell you. Just a thought!!

Regarding image weirdness, it was pngs that caused me problems too. More specifically, 32bit pngs. I never actually had time to work out what it was that was wrong with it, as I was midway through an intense project. Just one of those weird quirks to remember when something is screwing up and nothing else seems to make sense.

Anyways, very happy that things are progressing for you :)

DrStrik9
02-10-2016, 06:07 PM
Wow, what a serious relief after a week and a half of pain. PNG's: one of those older image formats that throws the wrench in the works big time. Of course, LW3DG can't abandon that format or someone will want it for some reason, so it sits there as a nasty trap for people needing dynamics. I wonder if there's a dev solution for this. I sure hope so.

I added this info to my bug report.

jeric_synergy
02-10-2016, 07:52 PM
Wow, what a serious relief after a week and a half of pain. PNG's: one of those older image formats that throws the wrench in the works big time. Of course, LW3DG can't abandon that format or someone will want it for some reason, so it sits there as a nasty trap for people needing dynamics. I wonder if there's a dev solution for this. I sure hope so.

I added this info to my bug report.
Huh. I always assumed that every image format was loaded and immediately converted to some sort of internal data format, for consistency in handling. I guess I assumed wrong.

That would eliminate any format-specific bugs.

DrStrik9
02-10-2016, 08:50 PM
Well, maybe. I had never heard that, but it could be so. Still, if your assumption is correct, there could be something amiss with how png's are being converted in 2015.3.

I'm just feeling very lucky that Kaptive mentioned that my nightmare could be caused by an image format.

jbondreau's workflow suggestion of calculating the sim prior to texturing is also wise. Then when adding a buggy image type, when everything falls apart, at least you might assume the failure was related to texturing, and you'd still have the finished sim in an earlier-saved scene.

jeric_synergy
02-11-2016, 12:27 AM
You'd have to be clairvoyant to have anticipated THAT workflow.

DrStrik9
02-11-2016, 07:13 AM
You'd have to be clairvoyant to have anticipated THAT workflow.

Agreed, but still, it's a smart workflow for things like dynamics, which rely on stuff outside your direct control (such as sim calc). This brings up the whole issue of so many aspects of LW being "procedural" -- (you must do this, and then that, and then the other thing, in that precise order, or you will not get to where you need to go). I am always going back and doing things over from the start, because of something I could not have foreseen, unless I had already made that same mistake the day or week before, and hadn't forgotten it yet. :+)