PDA

View Full Version : Bloody rendering time



Beamtracer
06-17-2003, 10:25 PM
I'm making yet another blood animation. You know the type, with red blood platelet models floating around everywhere, controlled by Lightwave's particle fx. Not another one of those blood animations, I hear you say!!! They are almost as common as the astroid-through-space scenes!

The main drain on rendering is the particle effects, and moving 900 platelets around. What I'm finding interesting is the difference in rendering times between two very different machines machines.

350MHz Vs 1000Mhz

Two G4 Macs running OS X 10.2.4, LW7.5
The render times as follows:
350MHz = 199.8 seconds per frame
1000MHz = 84.9 seconds per frame

Now, the difference between their megahertz rating turns out to be roughly the same as the difference in rendering time.

(1000MHz is 2.8 times 350Mhz / 199.8 is 2.35 times 84.9)

You could predict this would be the speed difference. Except for one thing... the the 1GHz machine is a dual processor (2 x 1GHz).

Can I conclude from this that Lightwave's particle fx does not take advantage of dual processors?

The 350MHz machine is 3 or 4 years old, the lowest speed G4 Powermac ever made, and the bottom of the rage at the time. The dual 1GHz was purchased late last year and was the top of the Apple range at the time.

Ade
06-17-2003, 11:08 PM
Question is does particle fx on pc support dual cpu's? If so add this to the list of parity..

DaveW
06-17-2003, 11:16 PM
How much RAM is in both machines? And is it possible you have some other effect or plugin in there that is not taking advantage of multiple processors?

stib
06-17-2003, 11:37 PM
in the rendering options how many threads are you using? Or if you're using screamernet are you running two instances of LWSN on the dual processor machine? I find I get faster render times with my dual processor 800MHZ G4 if I set the number of threads to 4. If I watch the CPU monitor the level stays flat chat - as hard as it can go when I'm using four threads, whereas if I set the number of threads lower it occaisionally goes down. I'm a newbie but this seems to help so far.

Beamtracer
06-18-2003, 12:34 AM
Hi Dave...

RAM in 350MHz machine = 1.3gig
RAM in dual Gig machine = 2gig

Hah, you thought it would be the other way around, that lack of RAM may have slowed down the dual gig machine.


Stib:

350MHz machine set to 1 thread
dual Gig machine set to 4 threads

I got these times by trying the same render on the two different machines at different times. No screamernet. No other apps running.

The moral of all this? If you're thinking of buying a 1GHz+ machine, maybe you could save your money and buy a few second hand 350MHz boxes.

Zarathustra
06-18-2003, 08:33 AM
Up your threads to 8. Why? I dunno but it seems to render faster.

Last year, unhappy with what Apple unveiled, I took the $$$ I would have spent on a new box and bought:
1) G4 400 on Ebay
2) 1ghz upgrade card
3) Dual 1ghz upgrade card (for my existing dual 500)
4) RAM
5) KVMP Switchbox

I've found some things the single renders almost as fast as the dual, and some things the dual is much faster. Unfortunately, I haven't done much with particles since those purchases so I have no experience on that front to share.

Johnny
06-18-2003, 08:50 AM
Originally posted by Beamtracer
The moral of all this? If you're thinking of buying a 1GHz+ machine, maybe you could save your money and buy a few second hand 350MHz boxes.

wonder if there's another message here...don't expect LW for Mac to take advantage of Apple technology?

I get the feeling that you could have a Mac running at Warp 15, and still have LW puttering along at sub-light.

J

DaveW
06-18-2003, 11:11 AM
Originally posted by Beamtracer

RAM in 350MHz machine = 1.3gig
RAM in dual Gig machine = 2gig

Hah, you thought it would be the other way around, that lack of RAM may have slowed down the dual gig machine.


Actually, I thought they would have the same amount of RAM and that the scene might just be RAM dependent rather than CPU dependent. That still could be the case, it's hard to tell. If you really want to see if LW is using both processors, watch the CPU meter while rendering with 4 or 8 threads. I'd watch RAM usage as well to see how dependent the scene is on RAM.

mlinde
06-18-2003, 11:19 AM
My question is whether you are rendering in LW (via F10) or LWSN. If you are rendering via LWSN you need a separate LWSN node for each processor on your dual processor machine. LWSN does NOT utilize multiple processors, from what I can tell, and running multiple versions does NOT effect render times adversely.

Scazzino
06-18-2003, 01:16 PM
You can tell what's going on with these two items:

1) Open Applications/Utilities/CPU Monitor. When rendering on dual processors, both should be 100% full while chugging along... if not, you're not fully using both CPUs...

2) Download and run MemoryStick:
http://www.tidbits.com/matt/

In MemoryStick, turn ON Signal pageouts and then listen, if it starts beep, beep, beeping, indicating memory swapping to disk, then you need more RAM or need to lower the segment memory...

Also, I've found that Lightwave itself usually renders OK with multiple threads (though some plug-ins may not), but LWSN renders fastest with one instance running for each CPU on the machine, each instance running a single thread...

-MikeS

stib
06-18-2003, 06:25 PM
Originally posted by MikeS
Also, I've found that Lightwave itself usually renders OK with multiple threads (though some plug-ins may not), but LWSN renders fastest with one instance running for each CPU on the machine, each instance running a single thread...

-MikeS
Is there way to tell or better yet control how many threads are running on a LWSN node? Is it determined by the render options of the scene or do LWSN nodes always use 1 thread?

Beamtracer
06-18-2003, 10:04 PM
I'd be interested to find out more about Screamernet and dual processors also.

In the meantime, I rendered a difference scene (that had no particles) and got a similar result... the dual 1000MHz machine was about about 2.5 times faster than the ye olde single 350MHz machine. This was just compositing (with background and foreground images) and no 3D models. It did however have the Corona plug-in added, which took most of the rendering time.

So, I guess I can add the Corona plug-in to the list of things that don't recognize dual processors.

It's a pity that the port to OS X was done using Apple's 'Carbon' environment, rather than the 'Cocoa' environment. If they'd used Cocoa, then multiprocessing would automatically be there.

mlinde
06-18-2003, 10:39 PM
Originally posted by stib
Is there way to tell or better yet control how many threads are running on a LWSN node? Is it determined by the render options of the scene or do LWSN nodes always use 1 thread?

AFIK, LWSN utilizes the thread count which is stored in the scene file.

Julian Johnson
06-19-2003, 07:13 AM
The thread setting is stored in the Lightwave Layout 3 Prefs file and is independent of the scene file (like segment memory). The only way to 'control' the number of threads LWSN uses is to manipulate the number of threads in the saved prefs file directly before you launch LWSN (which can get a bit clunky). Some tests a while back revealed that if you're on a dual processor machine then it's best to run two instances of LWSN both set to 1 thread. If you're on a single processor machine then the higher the thread count the faster LWSN seems to go (mileage varies between 2/4/8 threads and it's all dependent upon what's going on in your scene - HVs are sometimes faster with 1 thread anyway, for example). It would be nice to have the option to store the thread setting in the scene file since you normally end up finding a specific thread setting that works fastest for specific scenes....many, many times I've dumped the thread setting back to 1 (if I'm baking, say) and then forgotten all about when I'm working on scenes that need to be set to 8.

Julian

Scazzino
06-19-2003, 08:23 AM
Is there way to tell or better yet control how many threads are running on a LWSN node? Is it determined by the render options of the scene or do LWSN nodes always use 1 thread?

Yes, as Julian mentioned both the memory and threads settings are stored in the Layout preferences file.

The preferences text file is named: "LightWave Layout 3 Prefs" and normally stored in your user/Library/Preferences folder (Mac OS X)

The setting lines look like this:
RenderThreads 1
DefaultSegmentMemory 300000000

You may edit them in a text editor or change them from inside Layout.

The memory setting is located in the Camera Properties (even though it's a global setting in the prefs and really has nothing to do with any particular camera) called Segment Memory Limit

The thread setting is located in the Render Options panel under Rendering.

Be sure to quit Layout to save the prefs, since there is currently no "save prefs" button or command (at least that I know about anyway) before runnnig LWSN, since LWSN only reads the prefs file on launch.

You may also store a separate prefs file on each machine you use LWSN on to configure each machine individually for memory and processors, as long as you tell LWSN where to find the file on each machine...

-MikeS

Zarathustra
06-19-2003, 08:31 AM
I never noticed before, but the thread setting isn't saved in the scene file? I thought it was.

mlinde
06-19-2003, 10:00 AM
Hmm I didn't realize that either.

And Julian, you should mention these little tasks you must perform everytime you work. These are the things that can be bunched up into an LScript to make them a one-key task. (hint-hint)

More on that later :)

Zarathustra
06-19-2003, 10:09 AM
Man, I'm slow. I have taken scenes to a single processor machine and when I opened it the render options were 1 thread (as oppossed to the 8 thread setting on my dual when I saved the file). I assumed it was because it was a single cpu machine, but of course you can set multithread on a single cpu. I never put 2 and 2 together.

stib
06-19-2003, 05:55 PM
Why can't newtek put a decent front end on all these things? what other professional level program has users hacking at the prefs and so on with a text editor to do everyday things like these? It's like they were too pooped after writing the software to remember to make a GUI. How difficult could a decent LWSN controller be? Sheesh I could write one in Applescript even (if I had the time (which I don't)).

DaveW
06-20-2003, 12:11 AM
Why can't newtek put a decent front end on all these things? what other professional level program has users hacking at the prefs and so on with a text editor to do everyday things like these? It's like they were too pooped after writing the software to remember to make a GUI. How difficult could a decent LWSN controller be? Sheesh I could write one in Applescript even (if I had the time (which I don't)).

You don't have to hack at the prefs to change the threads, it's in the render options.

As for screamernet controllers, I don't know why it's such a pain in the butt. They were attempting to fix this with 6.0 but it never went anywhere. It's really not *that* difficult to get it working, but the 3rd party solutions out there are a lot better.

toby
06-20-2003, 12:14 AM
back when I replaced my G3 300 with a G4 450dp I got very close to 3X render speed, same as the mhz increase, but that's also going from g3 to g4. I don't think you ever get a 1-1 increase in speed. You seem to be getting much less than that tho :confused:

I look forward to having a multi-ghz G5 running a 64-bit Cocoa Lightwave:cool:

stib
06-20-2003, 12:41 AM
[/QUOTE]
You don't have to hack at the prefs to change the threads, it's in the render options.

true, but you do to get LWSN up and running in the first place. I've just been trying to set up LWSN across a couple of macs and a gaggle of PCs. My wrists are getting sore from all the typing in the DOS prompt. Damn! I put that \ in the wrong place, now I have to satrt over..
puhleease..

Julian Johnson
06-20-2003, 11:26 AM
And Julian, you should mention these little tasks you must perform everytime you work. These are the things that can be bunched up into an LScript to make them a one-key task. (hint-hint) Will do! :).

Arnie Cachelin
06-20-2003, 11:34 AM
It's a pity that the port to OS X was done using Apple's 'Carbon' environment, rather than the 'Cocoa' environment. If they'd used Cocoa, then multiprocessing would automatically be there.

This is completely false. The multithreading of a complex application like LW is an extremely difficult task that constantly treads a fine balance between memory duplication, setup overhead and the raw possibility of coming up with algorithms that can be made parallel, so they can be computed by different threads, without depending on each other's results. At one extreme, 2 instances of LWSN would duplicate ALL the memory requirements of a scene, and have complete CPU utilization, one CPU per instance. It would also duplicate the loading time and pre-processing of images, meshes, etc. For scenes that are fast to render, but have huge files to load, this could actually be slower than using a single thread/app.
Within layout with multiple threads, things are also complex, since many processes simply must wait for other things to complete. Not everything can be made parallel. This issue is identical on Mac and PC, BTW. Threads in layout handle the different segments of the render, so making the segments smaller can help even out the burden on the CPUs. For example, if the scene has an open sky and a complex object in the bottom of the frame, and there are 2 segments, one thread will finish the easy part very quickly, then wait for the other to finish the frame. This is also why adding more threads than CPUs can be useful (although there is a fixed 'setup cost' per thread). The fastest rendering method, in terms of thread# and segment size depends on the frame being rendered, and so, cannot be generally specified, nor can a single measurement say much about the actual performance. Moreover, mac martyrs could take solace in the fact that these issues are platform independent, and should avoid malinformed whining about LW ignoring mac OS optimization. LW multithreading has been fine-tuned for more years than Maya and XSI have been available, combined. The posix threads architecture is not new to macs, is well understood, and is being used diligently by LW.

Beamtracer
06-20-2003, 05:20 PM
Hi Arnie. I'm not a programmer so I'll happily stand corrected on my comment that apps written in Apple's Cocoa environment would "automatically" gain multiprocessing ability. I must have read an article that glossed over how much work would be involved (in Cocoa).

Regarding RAM... just my personal opinion, but if there is a balance between multiprocessing rendering speed and RAM-hogging, I'd much prefer that Lightwave uses oodles of RAM in order to go faster. I mean, this week Apple is releasing new desktop computers with 8 gigs of RAM. No RAM shortages on this side of the computer fence.

In the meantime, I might try using two copies of LWSN on the same machine and see if it makes things faster. File loading and saving time is always a tiny fraction of the time it takes to render each frame. For everything that I do, the duplication of this would be insignificant.

PS... Arnie, I hope life is going well for you at Luxology. Hopefully Lux will let us all know what it's up to in the near future :)

Beamtracer
06-22-2003, 08:06 AM
In my example of a 350MHz G4 networked with a 2 x 1000MHz G4, can I assume that the fastest configuration for these machines would be to use the slower machine as the network controller, and run 2 nodes of Screamernet on the faster machine?

mlinde
06-22-2003, 11:34 AM
Originally posted by Beamtracer
In my example of a 350MHz G4 networked with a 2 x 1000MHz G4, can I assume that the fastest configuration for these machines would be to use the slower machine as the network controller, and run 2 nodes of Screamernet on the faster machine?

Best configuration would be to use the 2x as a controller AND run 2 nodes, as well as run a node on the 350.

DaveW
06-22-2003, 09:46 PM
Originally posted by stib


true, but you do to get LWSN up and running in the first place. I've just been trying to set up LWSN across a couple of macs and a gaggle of PCs. My wrists are getting sore from all the typing in the DOS prompt. Damn! I put that \ in the wrong place, now I have to satrt over..
puhleease..

Just edit one of the config files then copy over the others. If you keep a well organized renderfarm or use a good third party controller then conifg files shouldn't be a problem.

stib
06-22-2003, 10:00 PM
Originally posted by DaveW
Just edit one of the config files then copy over the others. If you keep a well organized renderfarm or use a good third party controller then conifg files shouldn't be a problem.
no, I'm talking about what you have to type in at the DOS prompt to start LWSN on a PC. Here's an example: C:\lightwave\programs\LWSN -2 -d"L:\Shared\Lightwave\Content" "z:\Screamernet\Scenes\FireScene.LWS" 1 600 1
is what I used to get my last render going. That's something like 80 characters, and get one letter wrong and you have to start again. And of course it always happens late in the day when I'm trying to set up an overnight render. OK I could set up a shortcut, but that always seemed to crash LWSN, so I gave up on it, and I had to change jobs around a bit, so I always ended up typing to DOS on the fly.

As for third party controllers - I've just convinced my boss to shell out 3 and a bit grand (aussie dollars) on this programme, you think he's going to be too pleased when I put my hand out again (especially when I'm trying to get a FCP4 upgrade outta him too)? All I'm saying is that Newtek should write a decent front end for LWSN, not leave it to 3rd parties.

mlinde
06-22-2003, 10:28 PM
stib, you need to learn the classic batch program.

Open a text editor on one of your PCs. Type your command string.
Save the file as "LWSN command.txt"
Browse to the file in Windows Explorer.
Change the extension to .bat
You should now have "LWSN command.bat"
Double click on this file. It should start up LWSN -and- begin rendering the scene.

You could also use my Launch Mode3 plugin/lscript to generate these files from LW. I think the last version on my website still generated the correct file for a windows PC but did not launch the LWSN node.
http://home.earthlink.net/~mlinde/lscript.html

stib
06-23-2003, 01:14 AM
Mlinde you rock! I thank you from the bottom of my carpal tunnel!