Page 1 of 2 12 LastLast
Results 1 to 15 of 19

Thread: Enormous memory leak with image sequences?

  1. #1

    Enormous memory leak with image sequences?

    Having just read through the BSG VFX guys Emmy nomination submission, I decided to try out their image-sequences-on-hypervoxels-sprites technique, and I'm seeing massive memory usage in Lightwave, to the point that it locks up and won't finish the render, because it's used all my memory. Watching task manager as the render progresses, I can see it increases memory usage by tens of megabytes after every frame featuring the image sequences. The previous 160 frames are fine, but 20-25 frames of four image sequences, and the memory usage just carries on growing.

    I took two of the image sequences out, and sure enough, memory goes up at about half the rate.

    Anyone else seeing this? It's a bit of a serious problem. Perhaps it doesn't happen in the 64 bit version, or the BSG guys have much more than than my 2Gb RAM. Or both.

  2. #2
    ex-LightWave documentation BeeVee's Avatar
    Join Date
    Feb 2003
    Location
    Pessac
    Posts
    5,255
    What sort of images are you using? Avoid JPGs where possible and use 256-colour PNGs where you can. This will cut down on memory usage considerably. Optimized image use tutorial on LightWiki.

    B
    Ben Vost
    LightWave 3D Docs wiki
    AMD Threadripper 1950X, Windows 10 Pro 64-bit, 32GB RAM, nVidia GeForce GTX 1050Ti (4GB and 768 CUDA cores) and GTX 1080 (8GB and 2560 CUDA cores) driver version 456.71, 2x4K 23.5" monitors
    Dell Server, Windows 10 Pro, Intel Xeon E3-1220 @3.10 GHz, 8 GB RAM, Quadro K620
    Laptop with Intel i7, nVidia Quadro 2000Mw/ 2GB (377.83 and 192 CUDA cores), Windows 10 Professional 64-bit, 8GB RAM
    Mac Mini 2.26 GHz Core 2 Duo, 4 GB RAM, 10.10.3

  3. #3
    obfuscated SDK hacker Lightwolf's Avatar
    Join Date
    Feb 2003
    Location
    Stuttgart, Germany
    Posts
    13,703
    It could be that it uses that much memory. The BSG VFX team is afaik using 64-bit LW exclusively (at least for Layout and rendering).

    Cheers,
    Mike

  4. #4
    I'm using 24-bit TGAs and Quicktimes, but good point, I can convert them all to 8-bit PNGs no problem.

    I've no problem with it using a lot of memory, I'm more concerned with the fact that memory usage steadily increases, like it's loading in the new frames, and not ditching the old ones. In the scene, the number of individual image sequence frames that need to be rendered per fame goes up, then back down. And yet memory just goes up with every frame that has any hint of an image sequence in.

    I'll try the 8-bit solution, but it seems like there's something that needs fixing in Lightwave itself. In 25 frames, I'm a GB of memory down. Even with 8GB, you're still looking at less than 200 frames rendering at a time (assuming half a gig is being used for your scene and OS). It just seems, well, really bad.

  5. #5
    Ack, can't edit my post.

    I tried it without the Quicktimes, and the two 24-bit sequences themselves chewed up plenty of memory. I could get about 40 frames done before it crapped out.

    But yeah, I'll try 8bit PNGs. Thanks for the info/suggestions, guys. Is this a bug, or just an unavoidable side-effect?

  6. #6
    Super Member Nangleator's Avatar
    Join Date
    Sep 2007
    Location
    Massachusetts
    Posts
    2,142
    Set up a screamernet render. Just break your scene into smaller render chunks. Frames 1-5, then frames 6-10, etc.

    Perhaps memory will be cleared between scene loads.

    ()

  7. #7
    Okay, well, I switched to 8bit images, and it helped a lot. It still managed to build up 800Mb extra of used memory by the end of the render. So thanks for the advice, BeeVee, it definitely helped.

    However, if I could be permitted a moan (sorry):

    This still seems to me like a stupid memory leak. To take the situation above, if I quit lightwave, restart it, and render out any given frame, it manages it without the 800Mb overhead generated from the rendering (just taking up the usual 500Mb for the scene). So, clearly, that memory build-up is simply not needed. I'm going to take a guess here, and suggest that each frame isn't discarded from memory on subsequent renders, and new frames are loaded each time. The old frames stay in memory until I quit Lightwave.

    Come on, that's rubbish. I admit, I've never written a raytracing package, but if I'm right, this should be an easy one to fix. I can't see how the images I was using originally could account for that much memory use, unless they weren't being tidied from memory properly.

    I admit, I may have the wrong idea here, but the fact that quitting and restarting Lightwave (after rendering a scene with image sequences) allows me to render a frame from that same scene with over a GB less memory used, would seem like a memory leak issue. I don't expect a package as complex and long-term as Lightwave to have no memory leaks at all, but this seems like quite an obvious one to pin down.

  8. #8
    OK. Dumb question, but I have to ask. Are you using the image viewer or the render display when rendering?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	renderdisplay.jpg 
Views:	116 
Size:	20.2 KB 
ID:	61116  

  9. #9
    Quote Originally Posted by Surrealist. View Post
    OK. Dumb question, but I have to ask. Are you using the image viewer or the render display when rendering?
    Just the render preview. Why? I guess the image display would eat memory, but that still wouldn't explain why it only happens with image sequences.

  10. #10
    I have no idea. You could try and see what happens.

  11. #11
    Quote Originally Posted by Surrealist. View Post
    I have no idea. You could try and see what happens.
    Try what, turning off render preview? Trying that now, getting the same result. Good idea, though.

  12. #12
    Yeah. Had to ask. You never know. Just worth a shot

  13. #13
    Nothing ado about much Exception's Avatar
    Join Date
    Mar 2003
    Location
    Netherlands
    Posts
    5,560
    send in a bug report, if you will.
    It shouldn't use more memory after 100 frames than after 1. Except if you're using GI caching.

  14. #14
    Quote Originally Posted by Exception View Post
    send in a bug report, if you will.
    It shouldn't use more memory after 100 frames than after 1. Except if you're using GI caching.
    Happy to. I'm just... surprised that this really is a bug. I assumed there'd be plenty of other posts with people having run into this themselves, but... apparently not. Either it's just my machine, or I am, somehow, the first person to discover it.

  15. #15
    Nothing ado about much Exception's Avatar
    Join Date
    Mar 2003
    Location
    Netherlands
    Posts
    5,560
    Quote Originally Posted by DiscoBurgess View Post
    Happy to. I'm just... surprised that this really is a bug. I assumed there'd be plenty of other posts with people having run into this themselves, but... apparently not. Either it's just my machine, or I am, somehow, the first person to discover it.
    Have you tried it 9.5 beta? That would be useful to know.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •