PDA

View Full Version : Lightwave 2018.0.4 and memory issues



protogenes
05-07-2018, 03:35 PM
Lightwave 2018.0.4 is still crashing on my Mac, running out of application memory after rendering a few frames of animation.

I have 24 GB of Ram, 3.2 GHz Intel Core i5, OS Sierra 10.12.6

gerry_g
05-08-2018, 03:37 AM
must be more to it than just rendering, I loaded a .75K scene and did a turntable render over 240 frames, GI, Monte Carlo, and so on, memory did go from 3.5G to 7.5G but nothing that would crash it. Am on a Mac, same OS version as you and running LW 2018.0.4

protogenes
05-12-2018, 12:50 PM
must be more to it than just rendering, I loaded a .75K scene and did a turntable render over 240 frames, GI, Monte Carlo, and so on, memory did go from 3.5G to 7.5G but nothing that would crash it. Am on a Mac, same OS version as you and running LW 2018.0.4

This happens to me even without GI and motion blur, memory rises each frame until LW crashes.

gerry_g
05-13-2018, 01:54 PM
I looked again, first scene I tried was all hard modelled and only around 750,000 polys which took up 3Gb of memory rising to 7 at finish of 240 frames, second time I ran a subpatch model of around 4,000,000 polys and taking up around 5Gb at start, BUT, that rose to 31Gb by the time frame one was done, don't think tessellation at render time was problem as it had only doubled to 10Gb by the time the pre pass had finished, to me that sound like there is trouble here.

lertola2
05-14-2018, 06:21 AM
This is happing to me also. I sent in a bug report.

VideoRobb
05-21-2018, 09:39 AM
I'm having the same problem.

madno
05-22-2018, 12:58 AM
Same here without animation. Rendered a sub-d still multiple times. It comsumed a few hundred MB each time. Seems to be a fat memory leak.

3dworks
05-22-2018, 03:05 AM
i'm still not doing production on 2018 yet - but wanted to start this week. did this happen in 2018.0.3 as well? maybe i should stay with this version before using it for larger projects?

Shabazzy
05-22-2018, 05:22 AM
Are you guys all using Macs or does this affect Windows as well?

madno
05-24-2018, 12:51 AM
Possible cause of memory usage?

I assumed a memory leak, but think that assumption was wrong.

In my case (multiple still renders of same scene on Windows 10 machine, no animation) it seems to be the image previewer that needs the ram to store the previews in memory.

Render one (preview window open)
141838

Render ten (or so) (preview window open)
141837

Render ten again, this time preview window closed
141836

Note: Preview stores several passes, so it needs a lot of ram to hold them in memory.

Maybe that explains your issues as well?


EDIT:
Made some more renders and closed preview each time. Ram usage does not increase in this scenario.

CaptainMarlowe
05-24-2018, 03:32 AM
Interesting finding, thanks.

Skonk
05-24-2018, 06:18 AM
Are you leaving the preview window open between renders?

madno
05-24-2018, 08:54 AM
Yes, I leave the preview open - normally (64 GB installed to play with ;-).

If the window is selected I can hit "p" and get an advanced view where I can select two of the stored renders and compare them. I use it for example to check the effect of different sample settings. E.g. I can select a refl. pass of two of the renders and set a blend mode (I like the horizontal slider), to see how they differ. That only works if the window remains open during the renders (as said, the images are stored in ram not on disk).
By the way, that helped me a lot to get a feeling for the new 2018 render engine. For example: comparing passes of a high AA render vs. one with high individual samples made quickly clear that high AA is not the go to solution like it was in 2015 (well, sometimes it still is, but normally not).

Skonk
05-24-2018, 09:04 AM
The render preview stores every render you do until you close it so obviously the memory usage will increase every time you render and if you render an entire animation with the preview open you will end up with every single frame of the animation loaded into memory.

madno
05-24-2018, 09:13 AM
Correct. That's why I posted a second time to say I take back my assumption about a memory leak.
Well, after I got enogh coffee to start the "Initiate Brain Active Sequence".

jwiede
05-24-2018, 05:40 PM
Possible cause of memory usage?

I assumed a memory leak, but think that assumption was wrong.

In my case (multiple still renders of same scene on Windows 10 machine, no animation) it seems to be the image previewer that needs the ram to store the previews in memory.

...

Maybe that explains your issues as well?

Probably not, the OP's scenario (as well as that of subsequent animation-render posters) is quite different from the scenario you're describing.

They're citing failures during rendering of animation, where an rapidly-growing cost per frame eventually leads to memory exhaustion and failure. The frame-delta amounts they're citing (1's-10's of GB's) are also too large to reasonably blame on "normal" previewer operation.

Chuck
05-25-2018, 10:39 AM
Has anyone submitted bug reports with steps and content to duplicate? Please let me know the case numbers. Thank you!

jwiede
05-25-2018, 10:50 AM
Has anyone submitted bug reports with steps and content to duplicate? Please let me know the case numbers. Thank you!

Lemme know if not, and I'll attempt to replicate problem with demo, then file with supporting data.

To Protogenes & Gerry_G: For replication purposes, were your scenes largely focused around a single huge-poly-count object, many mid-poly-count objects, or...? Best if y'all can file for yourselves (and this is definitely worth filing on), but if you can provide packaged scene(s) replicating the issue, I'll gladly do the filing instead.

protogenes
05-25-2018, 06:04 PM
The render preview stores every render you do until you close it so obviously the memory usage will increase every time you render and if you render an entire animation with the preview open you will end up with every single frame of the animation loaded into memory.

Ive tried rendering with the image preview section of the render status window closed, but still the memory increases (if that is what you are referring to).

This was not an issue with LW before v2018, as I was able to render thousands of frames with no problem.

madno
05-26-2018, 04:05 AM
Hmm,

tried a quick animation on Windows 10 Prof / LW 2018.0.4. Seems so, no obvious memory leak exept from a little one once the render has finished. Maybe Mac related issue?

141850

Scene is attached (scene name is because I used it to report a caustics issue and it is 2018.0.4 not 03).

pmwhite
05-27-2018, 03:32 AM
Hi I'm Using a PC, but am also having stability problems, with some scenes. I was able to successfully open them with 2018.0.3, but my 2018.0.4 is crashing when I try to open them.

jwiede
05-28-2018, 12:27 PM
With an example from the OP, protogenes, I was able to reliably reproduce the memory growth problem (100% reliably), and it was trivial to reproduce as well. The fast memory usage growth is visible immediately and obvious, and there's no obvious justification for it.

protogenes
05-28-2018, 12:38 PM
With an example from the OP, protogenes, I was able to reliably reproduce the memory growth problem (100% reliably), and it was trivial to reproduce as well. The fast memory usage growth is visible immediately and obvious, and there's no obvious justification for it.

LW 2018 does not seem to have been tested for animation sequences before release, as this issue was in 2018.0.3 as well, the first version I used and noticed it in.

zamakli
06-05-2018, 03:24 AM
Using the new volumetric engine and primitives, you can specify physically based properties for your clouds and fog. These include, Scattering, Absorption, and Emission parameters, along with the use of powerful node networks to control all parameters.

jwiede
06-05-2018, 11:02 AM
LW 2018 does not seem to have been tested for animation sequences before release, as this issue was in 2018.0.3 as well, the first version I used and noticed it in.

Agreed, the problem is extremely obvious and visible during ANY sequence render.

jwiede
06-07-2018, 11:05 AM
Agreed, the problem is extremely obvious and visible during ANY sequence render.

Newtek, has this issue been root-caused and fixed? Is it at least in the process of being actively debugged?

It literally takes _nothing_ to reproduce. Just launch Layout and hit "Render Scene"/F10, and watch the memory consumption skyrocket (>6GB by frame 60 and still rising, confirmed in Activity Monitor). There's no rational justification for why an _empty scene_ should consume so many gigs of memory during rendering, and actual scenes' consumption growth rates are even more problematic.

Matt
06-07-2018, 12:37 PM
It's been identified as a Mac only issue, and has been fixed internally.

jwiede
06-07-2018, 01:42 PM
It's been identified as a Mac only issue, and has been fixed internally.

Thanks, very good to hear. Will it make it into the next update? Or is the fix further out?

Matt
06-07-2018, 01:51 PM
Thanks, very good to hear. Will it make it into the next update? Or is the fix further out?

Personally hoping for another patch, but it's not my call.

jwiede
06-07-2018, 02:35 PM
Personally hoping for another patch, but it's not my call.

No worries, understand how that goes. Hopefully TPTB authorize releasing another update with fix, current situ is quite problematic for Mac users rendering animation. Thanks again!

kolby
06-07-2018, 10:47 PM
It's been identified as a Mac only issue, and has been fixed internally.

I'm on Win7. Past week I rendered pretty simple animation. At the beginning system ram usage was 1.7GB, after 500 frames it was 7GB. So I'm afraid it is not just the Mac issue.

jwiede
06-07-2018, 11:30 PM
I'm on Win7. Past week I rendered pretty simple animation. At the beginning system ram usage was 1.7GB, after 500 frames it was 7GB. So I'm afraid it is not just the Mac issue.

Well, depending on what was going on, it might be a different (but similar) problem. Easiest way to tell is to simply launch Layout, then hit "Render Scene"/F10 (on the default, empty scene with default render settings), and allow the sequence to render. If the memory usage by frame 60 is around 6GB, it's suffering from the same problem on Win7 as what we're seeing on Mac.

It's worth noting that LW's memory usage normally grows when rendering animations, that in and of itself isn't remarkable. Run the same "empty scene" test in LW2015 (at least on Mac) and you'll see the usage continuously rise per frame throughout the render. This is because LW is retaining buffers for the output snapshots, etc. and is expected. What's remarkable in the current case is the _rate_ of memory use growth, which is much greater than can easily be accounted for by just normal buffer costs, and much greater than the rate of growth typical in LW2015.

kolby
06-08-2018, 09:16 AM
Well, depending on what was going on, it might be a different (but similar) problem. Easiest way to tell is to simply launch Layout, then hit "Render Scene"/F10 (on the default, empty scene with default render settings), and allow the sequence to render. If the memory usage by frame 60 is around 6GB, it's suffering from the same problem on Win7 as what we're seeing on Mac.

It's worth noting that LW's memory usage normally grows when rendering animations, that in and of itself isn't remarkable. Run the same "empty scene" test in LW2015 (at least on Mac) and you'll see the usage continuously rise per frame throughout the render. This is because LW is retaining buffers for the output snapshots, etc. and is expected. What's remarkable in the current case is the _rate_ of memory use growth, which is much greater than can easily be accounted for by just normal buffer costs, and much greater than the rate of growth typical in LW2015.

Sure, I noticed this in 2015 but the amount of memory spent was insignificant. Interestingly, once I hit Clear Scene, the amount of memory occupied by LW dropped from 7GB to 3.6GB while freshly fired Layout has only about 250MB. The only way how to get this memory back to system was to kill the Layout. So apparently not the same issue as with the Macs but similar one on Win7.

jwiede
06-08-2018, 01:24 PM
Sure, I noticed this in 2015 but the amount of memory spent was insignificant. Interestingly, once I hit Clear Scene, the amount of memory occupied by LW dropped from 7GB to 3.6GB while freshly fired Layout has only about 250MB. The only way how to get this memory back to system was to kill the Layout. So apparently not the same issue as with the Macs but similar one on Win7.

Definitely similar, could be same general cause, not enough info to judge. Sorry, I just haven't had time to poke around in Windows to see how similar the behavior there is to the Mac issue. I'll try to get a chance this weekend, and presuming I can, I should be in a better position to discuss.

Chuck
06-26-2018, 03:58 PM
Hi, Folks!

What are the case numbers from any problems reported here? I would like to deliver a nudge on those specific cases.

jwiede
06-26-2018, 06:19 PM
Hi, Folks!

What are the case numbers from any problems reported here? I would like to deliver a nudge on those specific cases.

Chuck, I wasn't the filer on the Mac issue, the person who first raised it mentioned doing that, believe they got replies and everything, I'll try to find the thread. I'm happy to file a dupe on the Mac if needed (I have data and screens).

I haven't had a chance to test for similar behavior under Windows yet (thus no Windows filing).

protogenes
07-02-2018, 11:28 AM
LWB-4007

Got this response May 29th:

"Yes, I believe that we have a handle on this - but more content can help test -

I will get this report assigned to the proper developer for assessment.

Thank you for your report.

Matt
07-02-2018, 04:54 PM
2018.0.5 is in your accounts

jwiede
07-02-2018, 08:04 PM
I was able to run the same "default scene" test I ran before, and with LW2018.0.5 (Mac) the memory growth was negligible across many frames (where the same scene before would have consumed gigs of additional memory) -- fix confirmed, and 2018.0.5 now seems even better than 2015.3 in that regard!

CaptainMarlowe
07-03-2018, 01:27 AM
I was able to run the same "default scene" test I ran before, and with LW2018.0.5 (Mac) the memory growth was negligible across many frames (where the same scene before would have consumed gigs of additional memory) -- fix confirmed, and 2018.0.5 now seems even better than 2015.3 in that regard!

Great news. I’ll update tonight when back home.

Matt
07-03-2018, 09:35 AM
BTW: There is a Mac crasher that is being addressed ASAP

CaptainMarlowe
07-03-2018, 11:04 AM
Thanks. I'll wait a bit before upgrading, then.

jwiede
07-03-2018, 01:51 PM
(withdrawn by author)

Matt
07-03-2018, 03:36 PM
Fixed.

jwiede
07-03-2018, 06:08 PM
Fixed.

Much appreciated, thanks Matt (and everyone else involved)!