PDA

View Full Version : LWSN much slower than F9



kangonto
06-11-2009, 04:44 AM
Same scene, same settings, same LW configs, all cores rendering,
CPU 100%

LW9.6 - F9 Render: 26minutes.
LW9.6 - LWSN: 35minutes.

I think it's a BIG difference.

Same problems with 9.5. I hadn't detected such thing with 8.x (months ago with different scenes)

Everything seems to be OK so must be some kind of bug.

Just to give some clues:

- HD Instance 1.8.1
- Infinimap 1.06 (and 1.51)
- Perspective camera. Photoreal Motion Blur. AA13, AS0,045
- New Dome light
- MDD Pointer from Denis Pontonnier
- Some Hypervoxels

Too many 'exotic' things to play with, so please, if someone could give me any advice based on his personal experience (or not), it will be welcomed.

I have to deal with 10 render nodes. Rendering from inside LW could be a pain in the a**.

Please help.

THANKS!

kangonto
06-11-2009, 05:58 AM
First tests involving HD Instance and Hypervoxels.
I set the frame size to 50% to accelerate the process.

Deactivating HD Instance and Hypervoxels:
LW9.6: 1:27
LWSN: 1:30 No important difference

Activating HD Instance and deactivating Hypervoxels:
LW9.6: 6:08
LWSN: 7:09 A little longer

Deactivating HD Instance and activating Hypervoxels:
Lw9.6: 2:21
LWSN: 3:21 ever more longer (proportionally)

HD Instance has a great presence in the rendered image, but Hypervoxels' presence is minimal. But when hypervoxels are activated there is a big difference between LW and LWSN.

Does this test mean that the problem is in relation with Volumetrics?

kangonto
06-12-2009, 11:53 AM
Apparently this issue has generated a great passion here.

I have made another test. A different scene. No volumetrics at all, just cached Final Gather.

LW9.6: 7:39
LWSN: 8:06

Slighty longer render time. No volumetrics, no problem.

Nobody uses screamernet?

Scazzino
06-12-2009, 12:06 PM
One thing to keep in mind as well is that ScreamerNet needs to load the scene where it's already loaded in LW before hitting F9. I have scenes that take quite a long time to load. So make sure to count the F9 time to include launching and loading the scene, or make sure to not count that time in your ScreamerNet tests (if you haven't already).

I have noticed that HD Instance 1.8.2 renders single threaded in my tests here, which makes a huge difference on the 8-core Mac Pro. But that was both F9 and ScreamerNet... I'm currently working on some scenes with both HD-Instance and hypervoxels and using both F9/F10 and ScreamerNet. I'll take some notes and run some speed tests if I get a chance.

kangonto
06-12-2009, 03:19 PM
I took into account that loading time. In fact, just to make sure, i made those tests (LWSN) rendering two frames in a row and taking only the second frame time.

And talking about threads, i think HD Instance renders using multiple threads. At least CPU is at 100% from the beginning to the end. Note that i'm on windows (XP and Vista).

Please, make those tests. I'm alone here!

I'm rendering HD frames and the difference is huge at that resolution.

Thanks for answering

Natty99
06-15-2009, 02:36 AM
Hi,

Having exactly the same problems here, but luckily running with no 3rd party plugins, just LW64 (which may help simplify the problem).
A frame rendered on my pc as part of the network=2 hours 11 minutes. Loading the scene into lightwave and hitting f9=15 minutes.
The scene is using radiosity caching, but both the f9 render and the network render are targeting and using it so i cant see how that could be casuing a problem.

Is this a lightwave bug?? At the moment its nearly as fast to render the sequence on my own machine through lightwave rather than dumping it to the render queue (if only i didnt need to work on it :( )

Any help would be appreciated.

R

kangonto
06-15-2009, 03:27 AM
You shouldn't have a so big difference in render time, looks like you have a different problem. Apparently every node is rendering all frames, or using a single thread. Have you payed attention at what's rendering each node and the CPU usage?, it should be at 100% all the time.

Try first to render a single frame using LWSN with -3 switch. If render time is similar to that of LW's F9 then the problem is related to your render farm configuration.

Natty99
06-15-2009, 03:48 AM
Thanks for the reply.

Due to the complexity of the scene and lack of ram we can only use one thread on the network machines (woudlnt be able to load multiple copies of the scene for each core).
The CPU usage is at 100% throughout. The render controller grabs focus every 2 minutes or so for 20-30 seconds, which takes up one of the cores, but as you say that woudlnt count for the increase in render times.

Anyway as you say, may be different to your problem (sorry for hijacking the thread). We don't really have time for extended testing to identify the problem either, so we'll just have to drop the settings to account for the slower network renders.

Thanks for the suggestions anyway and good luck getting yours fixed.

3DGFXStudios
06-15-2009, 04:19 AM
I took into account that loading time. In fact, just to make sure, i made those tests (LWSN) rendering two frames in a row and taking only the second frame time.


Correct me if I'm wrong but when you render to frames lwsn loads the scene every time it starts rendering a new frame. That means that when you use radiosity it needs to start over every frame. So there is you time difference.

I use screamernet all the time but I didn't notice these problems.

3DGFXStudios
06-15-2009, 04:21 AM
Hi,

Having exactly the same problems here, but luckily running with no 3rd party plugins, just LW64 (which may help simplify the problem).
A frame rendered on my pc as part of the network=2 hours 11 minutes. Loading the scene into lightwave and hitting f9=15 minutes.
The scene is using radiosity caching, but both the f9 render and the network render are targeting and using it so i cant see how that could be casuing a problem.

Is this a lightwave bug?? At the moment its nearly as fast to render the sequence on my own machine through lightwave rather than dumping it to the render queue (if only i didnt need to work on it :( )

Any help would be appreciated.

R

Looks like lwsn fails to locate you cache file and does a full radiosity render instead.

Natty99
06-15-2009, 07:12 AM
Hi 3DGFXStudios,

Thanks for the reply, but I don't think thats the case. The renders came out fine, no flickering, and watching the lwsn 'dos box' it appears to be locating and loading the cache fine.

kangonto
06-15-2009, 11:36 AM
Correct me if I'm wrong but when you render to frames lwsn loads the scene every time it starts rendering a new frame. That means that when you use radiosity it needs to start over every frame. So there is you time difference.

I use screamernet all the time but I didn't notice these problems.

That's not the case. The first tests were done on a scene without radiosity at all. If you read above, the problem is more related to the use of volumetrics, although LWSN is slower (more or less) than LWF9 in any case.

The test with the radiosity scene was done using cache file and was perfectly loaded before render in both cases.

Everything is OK. Simply LWSN renders slower depending on the case.

Scazzino
06-15-2009, 01:13 PM
I'm currently working on some scenes with both HD-Instance and hypervoxels and using both F9/F10 and ScreamerNet. I'll take some notes and run some speed tests if I get a chance.

Those 9.6UB speed tests may need to wait. I've hit a memory leak with 9.6UB rendering (each frame takes more RAM than the previous, eventually running out of RAM) and may need to back down to 9.3.1UB instead, which doesn't seem to have the same leak on preliminary testing...

jwiede
06-18-2009, 02:17 AM
Scazzino, vewwwy eeeenteresting. ;)

I think I may have seen something similar in a recent render test. I had both lots of objects, and HDI instances. Were you by any chance using HDI in the scene where you saw the leak? I've not had a chance to retest without HDI yet, so was holding off posting about it, but then saw your post.

Also, how dramatically did the per-frame memory consumption increase in yours?

Mine was enough to see memory bleeding away on the activity monitor, and as this system has 6GB at the moment, that implies a fairly significant delta.

JBT27
08-29-2009, 10:02 AM
Just searching for this and came across this thread.

Our renders in lwsn.exe via BNR versus F9 are always longer - I didn't realise that lwsn reloads the scene for every frame though. But even then, we see network renders in the order of 2 mins longer than F9s on scenes that don't take 2 mins to load ..... sure, there will be network delays and all that, but even then scene loading should only be seconds longer.

No GI on these scenes either. Working on a scene now with HDInstance (2.0.5), where F9s are around 3 mins 20 secs, but on the network are 5 mins 30 secs.

I haven't tested it deliberately, but the longer times seem proportional to the F9, rather than a fixed extra amount. BNR reports loading at the start of a scene render, which kind of gives the impression it's loading the scene and then holding it on each client, but maybe not ..... if it is loading only once then I really don't get why these renders are so much longer. takes a big chunk out of the advantage of network rendering.

Julian.

jaxtone
08-29-2009, 10:58 AM
I hope you all reported this to the tech and support team... (even if I would be surprised if they didn´t ignore this problem instead of creating a solution to it.)

I guess this is one of the things that can be connected to Lightwave´s old platform and the energy from tech and support team are quite low since they have their hands full of CORE.

When reading "CORE news" it seems like there is still going to be two separate program modules at least in the first phase. CORE for modeling and Layout for animations. So even if this bug cause insanely stupid network calculation processes and is in fact connected to bad old failures in the Lightwave engine this doesn´t mean that it is acceptable to leave it as is if Layout is meant for any professional project out there.


Just searching for this and came across this thread.

Our renders in lwsn.exe via BNR versus F9 are always longer - I didn't realise that lwsn reloads the scene for every frame though. But even then, we see network renders in the order of 2 mins longer than F9s on scenes that don't take 2 mins to load ..... sure, there will be network delays and all that, but even then scene loading should only be seconds longer.

No GI on these scenes either. Working on a scene now with HDInstance (2.0.5), where F9s are around 3 mins 20 secs, but on the network are 5 mins 30 secs.

I haven't tested it deliberately, but the longer times seem proportional to the F9, rather than a fixed extra amount. BNR reports loading at the start of a scene render, which kind of gives the impression it's loading the scene and then holding it on each client, but maybe not ..... if it is loading only once then I really don't get why these renders are so much longer. takes a big chunk out of the advantage of network rendering.

Julian.

dwburman
09-02-2009, 07:01 PM
Do you have multiple layers in your objects? I've read that multiple layers in objects slows LWSN down. (buried in here about 2/3rds of the way down: http://www.battlestarvfx.com/bsg_tutorial_shotbreakoutandlighting.htm) If you separate your layers into separate layers, it may speed things up.

adk
09-02-2009, 09:17 PM
... interesting info folks, cheers. I'll have a play with this with BNR and plain old LWSN and see how I go.

JBT27
09-03-2009, 01:06 PM
Do you have multiple layers in your objects? I've read that multiple layers in objects slows LWSN down. (buried in here about 2/3rds of the way down: http://www.battlestarvfx.com/bsg_tutorial_shotbreakoutandlighting.htm) If you separate your layers into separate layers, it may speed things up.

That's interesting - thanks for pointing that out.

Looking at my current scene, which I am trying to trim down, I'm only seeing one object with multi-layers anyway, and it's a relatively small object. The layers are needed for animating around different pivot points, and there's a single rig in this scene. I don't know how rigged objects affect lwsn .....

But this is handy info - thanks again.

Julian.

Scazzino
09-03-2009, 01:30 PM
... I didn't realise that lwsn reloads the scene for every frame though. ...

No it only loads it once, as long as it's rendering subsequent frames... at least using OSX ScreamerNet Controller (http://www.catalystproductions.cc/screamernet/). When I mentioned ScreamerNet loading verses F9, I just meant at the beginning. F9 already has the scene loaded when you hit F9. ScreamerNet has to load it (for the first frame for each node).

JBT27
09-04-2009, 02:58 AM
No it only loads it once, as long as it's rendering subsequent frames... at least using OSX ScreamerNet Controller (http://www.catalystproductions.cc/screamernet/). When I mentioned ScreamerNet loading verses F9, I just meant at the beginning. F9 already has the scene loaded when you hit F9. ScreamerNet has to load it (for the first frame for each node).

That makes sense ..... I'd be surprised if it didn't work like this.

So the significantly longer render times via lwsn are very much down to other causes.

Case in point: a ball moving toward the camera over 250 frames. Simplistic but proves the point.

Average frame render time via F10 was 0.4 secs. Using BNR, it was 8 secs. Curiously, every third or fourth frame it only took 4 secs.

In F10, the animation took about 2 - 3 mins. In BNR, it's reporting about 30 mins - it's running at the moment, but clearly it's going to take a stupid amount longer than with F10. That's on the same machine.

More complex scenes don't show such a disproportionate difference, but they can still take two to three times longer (or more) than with F10, per frame.

It tends to negate the point of having a small render farm.

Julian.

papou
09-06-2009, 07:26 AM
OMG, i confirm LWSN is slower in rendering !!
I got a 20% slower in a very simple scene with nothing special in it, classic or perspective camera,on a 1 CPU machine.

Anybody have reported that major bug to newtek?!

JBT27
09-06-2009, 11:47 AM
20% slower I could live with - a 10 secs render in F9 but 12 secs or so in lwsn ..... not disastrous.

But my 0.4 secs in F9 versus 8 secs in lwsn, and the more complex scenes which can be anything up to 200% longer in render time, via lwsn ..... that is a real problem.

But with all the veterans using LW, and all the big studios, I can't help wondering how this isn't a known major bug ..... or is it? Has no-one noticed, or do they not care? With a render farm of dozens or hundreds of nodes, you're not going to fret about it too much, I guess.

With a recent scene, we got F9/F10 render times of 3 mins per frame. On lwsn, we got around 5 mins per frame. So on one machine, it would take 300 mins via F10. If I split the scene and have our two licenses rendering, it would take 150 minutes, obviously.

If I put the scene on the farm, which is 4 identical machines total, where the frame times are 5 mins, it would take 125 mins.

So doubling our render power has only given us a 20% advantage - 150 mins via F10 using 2 machines, versus 125 mins using all 4 via lwsn - 150/125 = 120%.

OK ..... it IS an advantage for sure, but disproportianate. And clearly lwsn does not have the performance of the main renderer. But I thought the renderer had been rewritten? So was lwsn much lower down on the list and didn't get the same attention?

Now, I haven't tried rendering via the 64 bit version - I've found quite a few scenes, identical ones, render faster in 32 bit than 64 bit, so unless I'm using alot of RAM, I don't bother with LW64.

I just tried another scene - average frame render time in F9 is 24 secs, but in BNR with lwsn it is 44 secs.

What I see and what I've reported here does not make sense. It is massively inconsistent, and although network rendering works and clearly there are always advantages to having it, in the case of LW, it seems actually quite crippled by this performance hit.

Julian.

arsad
09-06-2009, 01:12 PM
Well I've worked on a feature film recently and I haven't seen such slowdowns.
But for me it was difficult to extrapolate the render times as my Workstation
was a i940 with 12Giga Ram and the render nodes were Quad cores with only 8giga.
But the nodes were approximately 60 per cent slower, so I thought it was ok.

JBT27
09-07-2009, 09:39 AM
I found this thread from nearly three years ago:

http://www.newtek.com/forums/showthread.php?t=57321

Though on the BNR mailing list, Paul Lord is wondering if the LWSN 'Segment Memory' and 'Threads' are setup, without clearly explaining how that is done. I thought BNR and the clients all accessed copies of the current LW config files, where all this is anyway ..... it seems there is more to it than that.

If you look at that old thread, some of this seems related to such things.

Julian.

JBT27
09-07-2009, 03:29 PM
Just thinking about the Segment Memory and Threads setup thing, this is what Paul said:

I wonder if the LWSN 'segment memory' and 'threads' are setup - Check the Options->Configure Lightwave section.
You can check this when checking the LWSN Logs (Options->Preferences to enable)

I don't get that - they are both always setup, but I guess he means setup correctly ..... there's no clue as to what correctly actually is.

In BNR Options menu, there is no Configure Lightwave menu command - there is a Configure Platforms but it has no control over Segment Memory or Threads. Also the LWSN log files make no reference to Segment Memory or Threads.

However, lwsn uses a copy of the LW configs which gets copied to the lw9intel folder each time LW is quit.

The LW9.cfg does contain entries for Segment Memory and Threads. I don't get why Segment Memory is an issue in renders being slower via lwsn than F9/F10.

But for threads, there is one entry for RenderThreads and another for ScreamerNetCPUs. As I'm rendering with quad-cores, I've set both to 4 - does that sound reasonable? Previously, ScreamerNetCPUs was set to 8, but RenderThreads was 4 ..... I don't know what that would do, if anything, but clearly lwsn renders differently to the core program, and I wonder if this is what Paul means to look at. It's not what he's saying, but that doesn't make sense.

Wouldn't it be great if this stuff just worked?? :D

Julian.

JBT27
09-08-2009, 04:14 AM
Another update.

So having set RenderThreads and ScreamerNetCPUs to 4, in the config which lwsn accessess, I'm still getting a massive slowdown - the current scene renders at 5mins 15secs on F9, but 8mins 36secs on lwsn, on the self-same machines.

If you read the thread I mentioned, it looks like NT admitted they were tearing their hair out over this years ago, but never bothered to fix it ..... perhaps they just couldn't.

Not getting quick or straight answers from Liquid Dream on this one either, though it may not be anything to do with BNR ..... it may simply be that lwsn is semi-bust.

Dunno ..... depressing, and frustrating.

Julian.

JBT27
09-08-2009, 07:59 AM
..... 8mins 18secs with F9 ..... 18mins 12secs with lwsn .....

I know I'm going on about this, but seriously ..... is this really normal network rendering for LW???

And if anyone has the info to help me configure this properly (using BNR), even though I believe it is and I cannot see settings I've missed, I'd be very grateful.

Or, if this is a genuine bug, as per NT's comments from this thread:

http://www.newtek.com/forums/showthread.php?t=57321

I'd like to know so I can stop wasting time chasing something that cannot be fixed other than by NT.

Thing is that on that thread, people setting up raw Screamernet seemed to be getting a fix for this. Others bought into BNR and that fixed it for them. Still others were citing this as a known issue, which NT had apparently swept under the carpet. Don't know about that, but it's a confusing mess of opinions, with a fix seemingly setting RenderThreads appropriately on each render node.

Well, BNR is supposed to do all that via the single config file on the controller machine.

Each node is rendering identically slowly. Paul Lord suggested that segment memory and threads may not be setup, when oddly his BNR is supposed to do that automatically. I was directed to look at commands which don't exist and don't seem to have any bearing on segment memory and threads anyway.

Is it worth me setting up Screamernet itself to see if that makes any difference?

Sorry for the ranting, but seriously, this is way past funny (not that it ever was) :mad::mad:

Julian.

Lightwolf
09-08-2009, 08:18 AM
.
Is it worth me setting up Screamernet itself to see if that makes any difference?
You could try launching lwsn from the command line using the -3 option (launch it without any options for a list of them).

Then you'll know if it is indeed lwsn or BNR or the BNR config.

Cheers,
Mike

Natty99
09-08-2009, 08:24 AM
Hi JBT27,

Just posting to confirm we had this problem too on a recent production, with significantly slower network renders when compared to hitting f9. I posted here and received the usual advice and setups, configs and paths etc (which we'd already double checked to death).
We never did figure out why, and I'm hoping it's something that gets automatically fixed with the release of Core:/

Edit: we don't use BNR as our controller incase that helps.

JBT27
09-08-2009, 08:42 AM
You could try launching lwsn from the command line using the -3 option (launch it without any options for a list of them).

Then you'll know if it is indeed lwsn or BNR or the BNR config.

Cheers,
Mike

Mike - sorry if I'm being thick ..... OK, I am being thick :D ..... I don't understand what it is I'd be looking at here. So launch lwsn from the Windows command line? With the -3 option?

What would it show that would allow me to pin down what's what and why?

Julian.

Lightwolf
09-08-2009, 08:47 AM
Mike - sorry if I'm being thick ..... OK, I am being thick :D ..... I don't understand what it is I'd be looking at here. So launch lwsn from the Windows command line? With the -3 option?

What would it show that would allow me to pin down what's what and why?
Well, currently you don't know if it's an issue with the render controller (BNR) or lwsn itself, do you?

Take one out of the equation (BNR), run lwsn from the command line on your workstation (which will use your Layout configs) and see if it renders at the expected speed.
If it does then you can't blame lwsn.
If it doesn't, you can't blame BNR :D

Cheers,
Mike

JBT27
09-08-2009, 08:53 AM
Hi JBT27,

Just posting to confirm we had this problem too on a recent production, with significantly slower network renders when compared to hitting f9. I posted here and received the usual advice and setups, configs and paths etc (which we'd already double checked to death).
We never did figure out why, and I'm hoping it's something that gets automatically fixed with the release of Core:/

Edit: we don't use BNR as our controller incase that helps.

In a depressing way, that's very helpful!

I'm guessing from your comments though, that you don't see this with every LW job?

I think we do, to varying degrees - I cannot remember a network render with LW that didn't run slower than F9/F10, to be honest. Sometimes it's not a terrible problem, but right now it is, typically.

Not sure what to do right now - I just uninstalled BNR and will reinstall shortly, in case something got screwed somewhere. Just wading through Matt Gorner's Screamernet tutorial as well, in case I get the urge to try that.

NT deserve a sound kicking for having neglected this part of the renderer for so long ..... as to a viable network renderer in CORE ..... well, is an average human lifespan long enough?? :D

Julian.

JBT27
09-08-2009, 08:54 AM
Well, currently you don't know if it's an issue with the render controller (BNR) or lwsn itself, do you?

Take one out of the equation (BNR), run lwsn from the command line on your workstation (which will use your Layout configs) and see if it renders at the expected speed.
If it does then you can't blame lwsn.
If it doesn't, you can't blame BNR :D

Cheers,
Mike

Well, put like that, it makes complete sense!

Thanks! I'll report back when I've had a go!

Julian.

Lightwolf
09-08-2009, 09:39 AM
Well, put like that, it makes complete sense!
Standard debugging procedure... try to isolate the components first and check if one of them fails on its, as opposed to checking a complex system.

Been there, done that... and still do it ;)

Cheers,
Mike

JBT27
09-08-2009, 09:42 AM
Oh dear - the shame of it .....

OK - sorry Mike, but I've run lwsn.exe -3 from the command line, but all I get is window that opens quickly, then closes .....

Also, my setup is not configured for raw Screamernet so not sure how I render with it. Just tried to launch SN from Layout, which I did, but there's no Command Folder or any of that, so I don't get how I can test it.

Just assume I'm thick on this stuff ..... :D

Julian.

Lightwolf
09-08-2009, 09:52 AM
Oh dear - the shame of it .....

OK - sorry Mike, but I've run lwsn.exe -3 from the command line, but all I get is window that opens quickly, then closes .....
No you haven't. You double clicked on the lwsn icon from explorer.

Open a command line window (i.e. -> win-r -> enter "cmd" + enter).
Navigate to the drive where LW is (in my case it would be):
f:
cd "Program Files\NewTek\LightWave 3D 9\Programs"

(f: being the drive where lw is installed).

Then:
lwsn

You'll see the following:


LightWave AMD64 ScreamerNet Module (Build 1539)

Usage: lwsn -2 [-c<config file>] [-d<content dir>] [-l<log file>]
[-q] [-t<check interval>] <job file> <reply file>

or: lwsn -3 [-c<config file>] [-d<content dir>] [-l<log file>]
[-q] [-t<check interval>] <scene file> <first frame> <last frame> [<frame step>]


To render a scene you'd then type:

lwsn -3 g:\my_content\scenes\testscene.lws 1 10 1

The arguments in [] are optional. This will work is the scene is in the content directory that Layout used before you quit it...and it will also use the same config files.

The LW Screamernet Controller from DStorm might be an option as well (it basically does the same thing, but with a GUI).

Cheers,
Mike

Scazzino
09-08-2009, 09:52 AM
I found this thread from nearly three years ago:

http://www.newtek.com/forums/showthread.php?t=57321

Though on the BNR mailing list, Paul Lord is wondering if the LWSN 'Segment Memory' and 'Threads' are setup, without clearly explaining how that is done. I thought BNR and the clients all accessed copies of the current LW config files, where all this is anyway ..... it seems there is more to it than that.

If you look at that old thread, some of this seems related to such things.

Julian.

Here's some info on those settings...
Default Segment Memory (http://dreamlight.com/insights/10/config_files.html#segmentMemory) & Render Threads (http://dreamlight.com/insights/10/config_files.html#Multithreading)

They are config settings stored in the config file, not the scene file. Your nodes may use the same config as Layout, or they may be pointed to separate configs. It depends how you set it up when you launch the nodes...

JBT27
09-08-2009, 10:24 AM
No you haven't. You double clicked on the lwsn icon from explorer.

Open a command line window (i.e. -> win-r -> enter "cmd" + enter).
Navigate to the drive where LW is (in my case it would be):
f:
cd "Program Files\NewTek\LightWave 3D 9\Programs"

(f: being the drive where lw is installed).

Then:
lwsn

You'll see the following:


To render a scene you'd then type:

lwsn -3 g:\my_content\scenes\testscene.lws 1 10 1

The arguments in [] are optional. This will work is the scene is in the content directory that Layout used before you quit it...and it will also use the same config files.

The LW Screamernet Controller from DStorm might be an option as well (it basically does the same thing, but with a GUI).

Cheers,
Mike

Thanks very much.

Well, I got that to go through the motions, but ended up with ten .flx files which each took 9.5secs to render. Hmm .... the first ten frames take over 8mins via F9/F10, per frame, so something is not right - wish it was ..... 9.5 secs, that's what I call rendering :)

I did originally launch lwsn.exe from the command line ..... just gave it nothing to do, hence it closed I guess. Like I said - thick on this stuff :) But learning.

But now what to do, because I'm not seeing a valid test .....

The scene is set to save Targas as well.

Julian.

Lightwolf
09-08-2009, 10:49 AM
But now what to do, because I'm not seeing a valid test .....

Is it the lwsn from the same directory where your LW is installed? (on the workstation).
Is the config of your LightWave redirected?
When you closed Layout the last time, was the content directory set to match the one where your test scene is located?

Cheers,
Mike

JBT27
09-08-2009, 10:56 AM
[QUOTE=Lightwolf;923728]

Is it the lwsn from the same directory where your LW is installed? (on the workstation).

Yes.

Is the config of your LightWave redirected?

Yes, it's in a configs folder within the LW Programs folder - I didn't point lwsn to that .....

When you closed Layout the last time, was the content directory set to match the one where your test scene is located?

Yes.

So I should add the argument for where the configs are?

Julian.

Lightwolf
09-08-2009, 11:01 AM
So I should add the argument for where the configs are?

Yes do that. It's the same "-c" argument as in the shortcut that launches LW.

Cheers,
Mike

JBT27
09-08-2009, 11:15 AM
Right, now it's rendering more slowly, so I'll post back shortly - just rendering two frames - that should be enough to tell.

Thanks.

Julian.

Lightwolf
09-08-2009, 11:21 AM
Right, now it's rendering more slowly, so I'll post back shortly - just rendering two frames - that should be enough to tell.

Yea, disregard the first one as it includes the set-up time for loading and prepping the images (mip-maps!) and geometry.

Cheers,
Mike

JBT27
09-08-2009, 11:36 AM
Yea, disregard the first one as it includes the set-up time for loading and prepping the images (mip-maps!) and geometry.

Cheers,
Mike

Hmm ..... it might be that LWSN is the culprit - although the first frame has loading and processing times, it's just finished at 17m 47s, versus 8m 16s or so via F9/F10. I don't think loading et al took over 9m .....

Stay tuned for the second frame :)

Julian.

JBT27
09-08-2009, 11:59 AM
The second frame is 17m 46s, so a few seconds shorter than with BNR, but over double the render time with F9/F10.

This tallies with what we see fairly regularly on shorter renders, and indeed with forum posts from three and four years ago.

So is this the moment to concede that there is really nothing we can do about it, other than live with it?

If so, then the only options we have are to be aware of it, and when feasible, buy more render nodes to dampen the hit of NT's failure.

I'm slightly surprised Paul Lord didn't mention this when I raised the issue on the BNR mailing list - he is surely aware of the reports of slow LWSN rendering?

Julian.

Lightwolf
09-08-2009, 12:37 PM
The second frame is 17m 46s, so a few seconds shorter than with BNR, but over double the render time with F9/F10.

Wow, that is massive. What kind of CPU are you rendering on? any third party plugins?

I'll have to check here as well, I'm curious now - especially since I never noticed a slowdown like that.

If you can, send the content to NT with a bug report, and give them your system config as well.

Cheers,
Mike

JBT27
09-08-2009, 12:49 PM
Wow, that is massive. What kind of CPU are you rendering on? any third party plugins?

I'll have to check here as well, I'm curious now - especially since I never noticed a slowdown like that.

If you can, send the content to NT with a bug report, and give them your system config as well.

Cheers,
Mike

Mike - the cpu etc is in my sig - we have two of those, and the two render nodes are identical but with only 4Gb RAM and minus the graphics board of course.

This scene is running with HD Instance 2.0.7, but nothing else third party.

Notice also earlier in the thread, the test I did with a single sphere, plain colour, as basic as it gets ..... 0.4s/frame with F10, 8s/frame with BNR/LWSN.

Do you genuinely think something might be messed up in my install somewhere, configs, whatever?

This is a job in progress, and I am loathe to let the scene out generally, though NT are welcome to it and whatever support files they want, as I imagine they handle all such stuff confidentially ..... not that it's dodgy content ..... it's a bridge!

Julian.

Lightwolf
09-08-2009, 12:54 PM
Mike - the cpu etc is in my sig - we have two of those, and the two render nodes are identical but with only 4Gb RAM and minus the graphics board of course.
D'oh, sorry, I didn't see that... to busy copying command line options ;)


This scene is running with HD Instance 2.0.7, but nothing else third party.
Is that actually the latest one, the one that Graham release a few days ago?

Do you genuinely think something might be messed up in my install somewhere, configs, whatever?
At this moment I'm not thinking anything at all, I'd rather test myself first before I start to ponder again ;)


This is a job in progress, and I am loathe to let the scene out generally, though NT are welcome to it and whatever support files they want, as I imagine they handle all such stuff confidentially ..... not that it's dodgy content ..... it's a bridge!
Oh yeah, the idea is to just make it available for them. And they do handle the stuff careful if you tall them to do so.

Cheers,
Mike

JBT27
09-08-2009, 01:01 PM
Yes, HD Instance 2.0.7 is the latest, though I was seeing these slowdowns with 2.0.5 as well.

The sphere test was odd, and maybe a F9 render at 0.4s is not a good test for a network render ..... but even so.

I'm going to try some simpler scenes now, because I am badly freaked and panicked by this - we have alot of rendering to do in the coming weeks and this is the last thing I needed.

Julian.

JBT27
09-08-2009, 03:52 PM
Just another set of tests on the same scene, because these perhaps start to pin it down.

To varying degrees, our network renders with BNR/LWSN have always been longer per frame than F9/F10 ..... but not to this degree.

So, I've taken the same scene and done two more tests with it. I said it only had HD Instance 2.0.7, but in fact I forgot it also has a Maestro rigged character in it.

First, I stripped out the Maestro character and the HD Instance stuff, and rendered that. With F9 it was 2m 22s - with BNR/LWSN it was 2m 46s. Now we're talking.

With the Maestro rigged character back in, F9 was 2m 33s - with BNR/LWSN it was 3m 02s.

So LWSN is noticeably slower than F9/F10, but not to the degree that I've been seeing.

Which leaves only HD Instance 2.0.7 rendering the full scene at around 8m 16s by F9, but over 18m on BNR/LWSN.

I'll try to run some more tests, but it's looking in this case like it really might be HD Instance causing the trouble.

Apologies to NT for picking on LWSN without having pinned it down more tightly ..... though a little more parity on speed wouldn't go amiss :D

Julian.

Lightwolf
09-08-2009, 03:59 PM
I'll try to run some more tests, but it's looking in this case like it really might be HD Instance causing the trouble.
Hey, good to read.. and welcome to "The Procedure" ;)

Cheers,
Mike

JBT27
09-08-2009, 04:13 PM
Hey, good to read.. and welcome to "The Procedure" ;)

Cheers,
Mike

Thanks!

Yes, less rant and more procedure is something I need to learn :D Well, on some things anyway!

I'll email Graham tomorrow, unless he happens to pick up on this thread.

Also, as an aside, Package Scene fails with this scene when it contains the HD Instance stuff. It just hangs so I have to force quit.

Well anyway, a slightly easier mind to finish the day, so that can't be bad :)

Thanks to all for the help and suggestions.

Julian.

Lightwolf
09-08-2009, 04:26 PM
Thanks!

Yes, less rant and more procedure is something I need to learn :D Well, on some things anyway!
Hehe, well, a rant gets you attention :D

Thanks to all for the help and suggestions.
That's what were here for. And with a little luck you'll even get help that, erm, helps and suggestions that aren't silly ;)

Having said that, those can be fun ... :D

Cheers,
Mike

papou
09-08-2009, 04:46 PM
ok.. so LWSN is just little slower than F10 now... 20-25% slower...
when i was rendering a lot with screamernet, i can remember LWSN was a little bit faster...

JBT27
09-09-2009, 06:41 AM
ok.. so LWSN is just little slower than F10 now... 20-25% slower...
when i was rendering a lot with screamernet, i can remember LWSN was a little bit faster...

Well, in this case it looks alot like an HD Instance/LWSN issue - I've emailed Graham at Happy Digital.

But I'm not convinced there's any hard rule on how or why this is happening generally. Again, that thread from 2006 cites similar issues, though not with HD Instance - one guy yesterday reported a recent serious slowdown issue with it at their place.

We rendered a ton of animatics this morning - we got 4s via F9, but 10s via LWSN - the simpler the scene, to us anyway, the more disproportianate is the slowdown (as in the scene at 0.4s with F9, but 8s with LWSN), but then with renders of seconds you may not be that concerned about it anyway.

So the serious slowdown I was getting has been pinpointed to a third-party plugin, but there are plenty of people still reporting LWSN as slower, with sometimes massive slowdowns - 2 to 3 times slower.

But now you're saying you are gettiing faster rendering than F9/F10 sometimes .....

Well, I reckon one thing that comes out of all this is that LWSN isn't even close to consistent with F9/F10 rendering. Rewriting the renderer is great, but it needs the support of a predictable and efficient network renderer, and while I'm mithering about it, a multi-pass system. In many ways the job is only a third done.

Julian.

Hieron
09-09-2009, 07:13 AM
Got alot of network rendering of HDI coming up and did not factor in it taking 3x as long. Any news on this *greatly* appreciated.


Atm I'm 100% busy with rendering out a current job so can't test it myself untill the weekend. Will try and see if I can check this current job's speed this evening for ya. It has 0 plugins but rendertime of 10 to 30 minutes so it is a nice testcase for LW only.


ps: not sure if in your tests of quick rendertimes (4 sec etc) the writing of the files is included. Depending on your network it may give an extra slowdown..

Lightwolf
09-09-2009, 07:39 AM
I just did a quick test of a production scene on a dual Athlon X2 @2.4GHz, XP64 and LW 9.6 x64:
Layout: 3m24s (204.6s) - I set the viewport to schematic, show render in progress on @640x480
lwsn: 3m35s (215.0s)

Still not right imho. I do wonder if Layout is pushing the render threads at a different process priority though.

Cheers,
Mike

JBT27
09-09-2009, 07:44 AM
Got alot of network rendering of HDI coming up and did not factor in it taking 3x as long. Any news on this *greatly* appreciated.


Atm I'm 100% busy with rendering out a current job so can't test it myself untill the weekend. Will try and see if I can check this current job's speed this evening for ya. It has 0 plugins but rendertime of 10 to 30 minutes so it is a nice testcase for LW only.


ps: not sure if in your tests of quick rendertimes (4 sec etc) the writing of the files is included. Depending on your network it may give an extra slowdown..

Yes, fair point - on the apparently disproportionate short renders, saving the files might play a part, even with our gigabit network I guess shifting the files onto the controller machine will chew up a few seconds that wouldn't otherwise happen rendering with F10.

While I was testing yesterday, I also reduced the density of instances in HDI, but that didn't seriously reduce the bloated rendertime on LWSN. So if you've got a load of network rendering coming up with HDI, I really would run some basic tests up front to see if you can optimise the scenes.

To be honest, I would have just left our scene to render, but even with AA through the roof, we were getting serious aliasing on the foliage that made up most of the instances, even in the foreground. I'm not sure if that was simply too much fine detail, or if HDI is struggling generally. Looking carefully at the objects and the artefacting, and the AA levels I tried, I am stumped as to why I was still getting bad flicker.

That's another issue, but mentioning it in case it features in your upcoming work.

Julian.

JBT27
09-09-2009, 07:49 AM
I just did a quick test of a production scene on a dual Athlon X2 @2.4GHz, XP64 and LW 9.6 x64:
Layout: 3m24s (204.6s) - I set the viewport to schematic, show render in progress on @640x480
lwsn: 3m35s (215.0s)

Still not right imho. I do wonder if Layout is pushing the render threads at a different process priority though.

Cheers,
Mike

Well, I would regard that as a good day with LWSN/BNR here :D

What I'm uncertain of is how much the user can tweak network render settings to prevent this happening ..... which was part of my thinking yesterday - if LWSN is partly flawed in the code, there's a point where nothing can be done outside of NT.

Our universal experience is that LWSN always renders slower than Layout - the HDI escapade yesterday was an unusually vicious slowdown, but slowdowns of 50% are not uncommon, even on relatively simple and light scenes.

But BNR does set priorities doesn't it? That's one of the options in each Client node .....

Julian.

Lightwolf
09-09-2009, 07:53 AM
But BNR does set priorities doesn't it? That's one of the options in each Client node .....
Yup, and it's usually at "Below Normal" - which makes sense if you actually want to work on that machine.

Cheers,
Mike

Hieron
09-09-2009, 01:22 PM
To be honest, I would have just left our scene to render, but even with AA through the roof, we were getting serious aliasing on the foliage that made up most of the instances, even in the foreground. I'm not sure if that was simply too much fine detail, or if HDI is struggling generally. Looking carefully at the objects and the artefacting, and the AA levels I tried, I am stumped as to why I was still getting bad flicker.

That's another issue, but mentioning it in case it features in your upcoming work.

Julian.

thanks for the heads up.
Btw: if you have flickering noise in the animation, *alot* of that can be fixed by a temporal noise filter like Neat Video or De:noise in my experience.

Since the noise is changing per frame it can get rid of it pretty well in many cases. Perhaps I am stating something you already know.. nvm then! :)

JBT27
09-09-2009, 01:57 PM
thanks for the heads up.
Btw: if you have flickering noise in the animation, *alot* of that can be fixed by a temporal noise filter like Neat Video or De:noise in my experience.

Since the noise is changing per frame it can get rid of it pretty well in many cases. Perhaps I am stating something you already know.. nvm then! :)

No - didn't know that - thanks!

I like to AA the crap out of the scene normally to shift noise, but in this case it just wouldn't ..... that coupled with the abnormal render times and it's back to the drawing back, for the foliage anyway.

Shame - HDI did a great job on it and it looked just right. Maybe I'll see if these temporal noise filters fix this, and if so, put up with the render time :)

Julian.