PDA

View Full Version : The Network Rendering Curse



Chris Jones
08-25-2014, 11:56 PM
I have never had any success getting Lightwave to render over a network. At least not since my days in a game studio when there were a pair of IT guys dealing with the burden, and a brief test done with the help of a computer scientist on a university supercomputer.

I tried networking 2 computers when rendering my eye animation, but they wouldn't communicate with each other at all. I tried command line rendering on 1 PC... still no luck. Even back when I was rendering The Passenger I had a tech savvy guy hacking away for two full days as he tried to get LW to cooperate with the farm at a prominent animation/FX studio here in Melbourne. He gave up. I had to independently render it on 4 separate machines at home.

I tested Amleto a few months ago, and miraculously it did render something - except post processing wasn't working. No good.

Fast forward to now. I've been rendering an animation on one PC for the past month or two, but there's still a fair way to go so I reluctantly decided to give network rendering one more chance. I sent the scene to GarageFarm to test, and surprise surprise, the renders came out looking wrong. The DOF settings are out of whack when it renders on their farm compared to how it renders locally, so all they can do is render it on a single PC (which is what they're doing now).

As you can probably imagine I'm fairly exasperated by now. Has anyone heard of DOF rendering differently over a network before? Am I doomed to render anything I do in LW on a single computer for all eternity?

Thanks for any help.

(c:

sukardi
08-26-2014, 12:38 AM
Hi Chris,

It is hard to give suggestions without looking at the actual scene. However, since garagefarm is already on the case, I probably would not be able to add anything more to it ...

Matt
08-26-2014, 10:04 AM
Would love to help you get setup Chris, I've written docs on ScreamerNet rendering, quite old now, but still relevant, they may help if you've not seen them.

Video and PDF:
http://www.pixsim.co.uk/#lw_screamernet

If you have and still need help, get in touch.

Might have to look at picking back up my script that helped setup render nodes.

Cheers
Matt

erikals
08-26-2014, 10:53 AM
also see >
Native ScreamerNET setup

https://www.youtube.com/watch?v=r8zCZchifKg
https://www.youtube.com/watch?v=z_hVLhhcnCU
https://www.youtube.com/watch?v=cZTtjEPDRqk

by Mikael Burman "Cageman"

--------

sorry no, not sure why the Dof isn't rendering the right way.
you could send Mikael Burman a private message on YoutUbe, as i think he knows this stuff in-and-out...
edit: i sent him a message and link for you

Cageman
08-26-2014, 01:30 PM
It is hard to tell without actually having the content.

We've used over 300 machines at Massive when we rendered some large scenes out of LightWave without any such hickups as what you describe. I also have a modest 3 node "rendergarden" at home.

If you want me to troubleshoot your scene, PM me and I can setup an FTP where you can upload the content directly to my server and then I'll take a peak to see if something odd is going on.

erikals
08-26-2014, 03:20 PM
one thing i just thought of is that here at the forums we just tried DPont's Marker Pen.

and it turns out, when exiting LightWave, it doesn't save the changes.
...either that, or it doesn't load the changes when loading the scene

maybe it could have something to do with this... ? that the Dof settings are simply not saved / loaded ?

Chris Jones
08-26-2014, 08:54 PM
Thanks for the suggestions,

Much as I loathe the idea of attempting any kind of networking myself (since upgrading to Windows 8 from XP I can barely even achieve a simple file transfer between two computers), it looks like I'll have to try one last time to get LWSN to work. Denis Pontonnier sent me some clues there as well. If I can reproduce what's happening at Garagefarm I might be able to break the scene down into something testable, which would save having to hand around my past couple of years worth of work..

Thanks again!

(c:

Waves of light
11-12-2015, 11:04 AM
Chris,

Did you ever get to the bottom of this. I'm currently having problems with LWSN, Garage Farm and Rman Collection Marker Pen node (latest version). I'm just waiting for them to get back to me as it may be a old plugin issue (i.e. not up to date).

Cheers,
Ricky.

Scazzino
11-12-2015, 12:13 PM
If you're still having LWSN trouble, you can try the free lite version of DreamLight Constellation (http://dreamlight.com/shop/dreamlight-constellation-lightwave-3d-network-render-controller/) to configure and set up LWSN with a simple drag-and-drop interface that will take care of all the nitty-gritty details for you. That way you'll eliminate any potential syntax/path errors in your setup.

Waves of light
11-12-2015, 12:38 PM
If you're still having LWSN trouble, you can try the free lite version of DreamLight Constellation (http://dreamlight.com/shop/dreamlight-constellation-lightwave-3d-network-render-controller/) to configure and set up LWSN with a simple drag-and-drop interface that will take care of all the nitty-gritty details for you. That way you'll eliminate any potential syntax/path errors in your setup.

Thanks for that Scazzino. I will certainly have a look at that, once I've got the GarageFarm problem sorted.

To add to my testing (thanks to Probiner and Luis Lopes) I've found that if you have your 3rd party configs aways from the default config file C:\Program Files\NewTek\LightWave_2015.3\support\plugins it will not load them when you run LWSN via command prompt.

Having moving all my third party plugins to a separate folder within the above default folder and changing all the paths in the LWEXT2015.3-64.CFG file, LWSN has worked. But it's looking like Marker Pen loads, but doesn't work over LWSN. I'm continuing my checks.

JonW
11-12-2015, 12:49 PM
Have you tried setting up from a different computer.

I can almost set up Screamernet blindfolded from any of my PCs but I could never get it to work from my W5580. Any of the other computers are fine.

Matt's tutorial was a life saver!

Waves of light
11-12-2015, 04:26 PM
Thanks JonW. Right now my issue is trying to get Marker Pen to render over LWSN or another LW compatible renderer. I've tried setting just one layer only using Marker Pen (as my scene has two layers using it) but that hasn't helped. The guys at Garage Farm were helpful enough to allow me to log into their PC remotely and render the scene locally, but as soon as they sent it to the network, no displacement. We checked everything, up to date nodes, content directories setup correctly... but no. Locally, fine.. network, no.

Going to bake out the displacement and send that to the render farm.

Chris Jones
11-12-2015, 04:30 PM
Chris,

Did you ever get to the bottom of this. I'm currently having problems with LWSN, Garage Farm and Rman Collection Marker Pen node (latest version). I'm just waiting for them to get back to me as it may be a old plugin issue (i.e. not up to date).

Cheers,
Ricky.

Sadly no, I haven't attempted any network rendering since my aforementioned project. :/

Waves of light
11-12-2015, 04:34 PM
Thanks Chris.

Lol, it was my first venture into networking rendering too. It can be a little off-putting when things go wrong, but I always try to find out why things have gone pear-shaped. I've learnt some good points from the experience, especially from other helpful LWers and GarageFarm have been very open and helpful too.

I'll keep you posted as I get to the final renders.

Scazzino
11-12-2015, 05:10 PM
Here's some more information about setting up Screamernet, with tutorials (http://dreamlight.com/blog/mastering-lightwave-3d-screamernet-lwsn/), that may be useful.

Waves of light
11-12-2015, 05:12 PM
Here's some more information about setting up Screamernet, with tutorials (http://dreamlight.com/blog/mastering-lightwave-3d-screamernet-lwsn/), that may be useful.

Going to pump some time into this next week Scazzino, because it will be interesting to see if things work across your method.

ranhell
11-12-2015, 06:50 PM
This is a good thread for me. I'm in the process of setting up a second small ScreamerNet render farm with four Dell PowerEdge servers with 24 cores and 48 gigs each. I was so impressed with the performance of one server I had to buy three more. One of my renders that I did took an entire week to render. My first experimental setup with SreamerNet was pretty gruesome but I was able to set it up on the network no issue. My first render was a few frames that where black. Like any other person out there I got pissed what the hell this is junk. Instead of giving up I just read the errors in the command prompt and fixed them accordingly. Next render I was rendering my geometry oh ya, now I'm cooking next render was with fiberfx I could not get that f####r to work for ****. After several hours later I realized I never scanned my plugins directory to my lightwave sreamernet shortcut. Once I scanned my plugins to the screamernet lightwave everything worked. I'm not going to lie I'm new to screamernet but the headache and aggravation of setting it up is worth it. That one render that took me a week only took four hours on 24 cores seeing that for the first time was insane the way it just chewed through that scene was amazing. Again network rendering is totally worth it.

mummyman
11-12-2015, 07:09 PM
I use Royal Render for network rendering at my work. There were some issues with specific Newtek buffers / processes that just don't work via network rendering. So I stick to what I know works. I can test something if you'd like. I use LW 2015... but have been using it since 9.6 or so.

JonW
11-12-2015, 08:04 PM
Thanks JonW. Right now my issue is trying to get Marker Pen to render over LWSN or another LW compatible renderer. I've tried setting just one layer only using Marker Pen (as my scene has two layers using it) but that hasn't helped. The guys at Garage Farm were helpful enough to allow me to log into their PC remotely and render the scene locally, but as soon as they sent it to the network, no displacement. We checked everything, up to date nodes, content directories setup correctly... but no. Locally, fine.. network, no.

Going to bake out the displacement and send that to the render farm.

Any gaps in file names?

Years ago SN was spitting the dummy. It worked ok but could not find the content files. So I copied the folder with files to each computer and change the directory in each node. So each node was referring to its own computer for the files.

JonW
11-12-2015, 08:09 PM
Thanks Chris.

Lol, it was my first venture into networking rendering too. It can be a little off-putting when things go wrong, but I always try to find out why things have gone pear-shaped. I've learnt some good points from the experience, especially from other helpful LWers and GarageFarm have been very open and helpful too.

I'll keep you posted as I get to the final renders.

I found once you have used Screamernet I treat every job as a network rendering job. Naming every file properly. Put everything in correct folders at the start of every job. Set up images properly from the beginning. In other words one becomes a good administrator!

JonW
11-12-2015, 08:14 PM
This is a good thread for me. I'm in the process of setting up a second small ScreamerNet render farm with four Dell PowerEdge servers with 24 cores and 48 gigs each. I was so impressed with the performance of one server I had to buy three more. One of my renders that I did took an entire week to render. My first experimental setup with SreamerNet was pretty gruesome but I was able to set it up on the network no issue. My first render was a few frames that where black. Like any other person out there I got pissed what the hell this is junk. Instead of giving up I just read the errors in the command prompt and fixed them accordingly. Next render I was rendering my geometry oh ya, now I'm cooking next render was with fiberfx I could not get that f####r to work for ****. After several hours later I realized I never scanned my plugins directory to my lightwave sreamernet shortcut. Once I scanned my plugins to the screamernet lightwave everything worked. I'm not going to lie I'm new to screamernet but the headache and aggravation of setting it up is worth it. That one render that took me a week only took four hours on 24 cores seeing that for the first time was insane the way it just chewed through that scene was amazing. Again network rendering is totally worth it.

I think everyone should read Matt's tutorial. Even if you set up SN differently his tutorial explains why and what is happening behind the scenes. I would also suggest print off a copy and take notes if you are new to Screamernet. I still refer to my printed copy now and then.

paulhart
11-13-2015, 03:12 PM
No one has suggested Amelto?? I have used it for years in a small (3 stations network) and the developer has made updates recently. Donate to him and give it a try. Fairly easy to setup, easier than any ScreamerNet, at least for me. http://virtualcoder.co.uk/amleto/

Scazzino
11-13-2015, 08:52 PM
Any gaps in file names?

Years ago SN was spitting the dummy. It worked ok but could not find the content files. So I copied the folder with files to each computer and change the directory in each node. So each node was referring to its own computer for the files.

You can use free lite version of DreamLight Constellation and DLI_SNUB-Launcher (http://dreamlight.com/shop/dreamlight-constellation-lightwave-3d-network-render-controller/) to simply drag-and-drop the config folder, content folder, scene file and output folders to the GUI and it will properly configure all the command lines for you, wrapping the paths in quotes to allow spaces, etc. Even if you don't use them for the rendering you can still use them as an aid to validate and create your command lines to ensure proper syntax. They even perform other checks for you such as testing for proper read/write permissions, proper content folders, etc. and indicate success or failure with red/green color coding. Same drag-and-drop GUI on both Mac and Windows.

Chris Jones
11-13-2015, 09:30 PM
No one has suggested Amelto?? I have used it for years in a small (3 stations network) and the developer has made updates recently. Donate to him and give it a try. Fairly easy to setup, easier than any ScreamerNet, at least for me. http://virtualcoder.co.uk/amleto/

Apparently it doesn't work with post processing according to my original post, although that was a year and a half ago...