PDA

View Full Version : Trying to figure out network rendering



Sebasvideo
07-30-2013, 06:44 PM
I'm trying to figure out network rendering in Lightwave and as far as I can tell this is in the LW 10 manual and not in the addendums. But one thing I read there worries me: "Each will render a frame from the animation until the scene is rendered." Does that mean that in Lightwave I cannot use network render for just one frame, it has to be a scene and each computer renders a full frame? I'm used to Modo where each frame is rendered by all available computers.

But regardless of that, how does it work? It seems to me that the manual assumes I already know some of this stuff. I assume I have to install LW in all computers, and then what? In Modo I have the shortcut to it with the /slave modifier so it starts in slave mode and when I press F9 all nodes (well, my three computers) start rendering. I have to start each slave with that modifier so the master knows it's a slave. It appears to be more complicated in Lightwave, but it may be that I'm totally unfamiliar with it. It will be a real letdown if it renders one full frame per computer instead of all computers contributing to the render.

Phil
07-30-2013, 08:40 PM
Yes. Each node is tasked with a single frame. LW does not do buckets. There are third party scripts/plugins to slice up frames and to distribute those to nodes. Some renderfarm controller software for LW also enables this.

Note that this slicing up may cause issues for pixel filters and post-process effects, so you'll need to experiment.

As for getting ScreamerNet set up, that's somewhat dependent on your operating systems. Presumably, you are running Windows. There are various guides out there to get it running; some reference older configuration file names and program locations, but common sense should help you adjust these as you go.

Try http://www.pixsim.co.uk/downloads/screamernet_tutorial.pdf for starters - it seemed relatively complete, but you'll need to (as mentioned) be aware that the config files and program locations will be different (e.g. LWEXT11.CFG, LW11.CFG, etc. rather than LW3.CFG; LWSN.exe now resides in the bin folder inside the LW folder). Note also that LW 11.0 onwards *require* that you install LW on each node in order to set up the libraries needed by LW to run. You can, though, then run LWSN.exe from a common network location if you desire.

tcoursey
07-30-2013, 08:46 PM
I'm trying to figure out network rendering in Lightwave and as far as I can tell this is in the LW 10 manual and not in the addendums. But one thing I read there worries me: "Each will render a frame from the animation until the scene is rendered." Does that mean that in Lightwave I cannot use network render for just one frame, it has to be a scene and each computer renders a full frame? I'm used to Modo where each frame is rendered by all available computers.

But regardless of that, how does it work? It seems to me that the manual assumes I already know some of this stuff. I assume I have to install LW in all computers, and then what? In Modo I have the shortcut to it with the /slave modifier so it starts in slave mode and when I press F9 all nodes (well, my three computers) start rendering. I have to start each slave with that modifier so the master knows it's a slave. It appears to be more complicated in Lightwave, but it may be that I'm totally unfamiliar with it. It will be a real letdown if it renders one full frame per computer instead of all computers contributing to the render.

As stated already, LW does not have bucket rendering. I tried Modo with network rendering (slave mode) and it was cool to have an F9 type render use all of my computers in the office! pretty sweet.

We use BNR Butterfly net render to control our network. There are others (I haven't stayed up on them though) The controllers can help slice up the frames horizontally or vertically, but not buckets (that I know of).

I have not used the internal screamernet controller since maybe 6 or so. I don't think Newtek (LW3DG) has done much since either. They depend on 3rd party controllers to fill the gap. Which they have. They talk about the unlimited nodes (which is great) but don't have a robust controller to handle the distribution... :(

Good luck.

Sebasvideo
07-30-2013, 09:19 PM
That kind of sucks coming from Modo. I'm not doing professional work yet, but in learning 3D I download some free models and experiment with textures and lights to make them as realistic as possible, and that takes a lot of rendering time even for one frame. Render buckets are great for that reason, Modo calculates the power of each node dynamically and gives me the best possible render time. If I got a render manager that slices according to a certain percentage it would probably take longer unless it calculates the slicing as it renders rather than set a specific amount before render. But, BNR Butterfly is $180 for the cheapest version that supports slicing (the PE version doesn't).

I hope they include buckets at some point in the future. I'm sure that Lightwave has some great features to complement Modo, that's why I got it.

nickdigital
07-30-2013, 09:38 PM
Rendering a still across multiple nodes does sound useful. But doesn't VPR negate the need for that, a least for quick feedback/test renders?

Lewis
07-31-2013, 12:47 AM
Amleto (now free) supports slicing and it's setup is very very easy (although it's copying all project files to each node so it can take some time to start rendering first time).

so check that for a start - http://virtualcoder.co.uk/amleto/

Sebasvideo
07-31-2013, 08:55 AM
Rendering a still across multiple nodes does sound useful. But doesn't VPR negate the need for that, a least for quick feedback/test renders?

Quick feedback, yes. But when you want to render something that is fairly heavy and would take 40 minutes to render one frame without a lot of grain, the way Modo does it is really useful. While my two other computers are rather slow compared to my main one, at least they shave off a few minutes.

nickdigital
07-31-2013, 09:27 AM
Right, I can definitely see the benefit of that. There are ways to slice an image across multiple nodes but LightWave's renderer on its own doesn't do that as it's not a bucket renderer.

paulhart
07-31-2013, 11:49 AM
As Lewis said, take a look at Amleto. I have been a user from the paid days, continued on to now, always found it useful and relatively easy to setup. Problems have always been "user error" and relatively easy to fix. There are other threads on forum posting process.

Sebasvideo
08-03-2013, 01:45 PM
Amleto (now free) supports slicing and it's setup is very very easy (although it's copying all project files to each node so it can take some time to start rendering first time).

so check that for a start - http://virtualcoder.co.uk/amleto/

If you use this then you don't have to setup Screamernet at all? I started reading the manual on it the other day and I was so disappointed. It shows screenshots from Windows 2000 (a 13 yr. old OS), Lightwave 8 (a 9 yr. old version) and the whole process is so complicated you have to wonder if they actually gave out prizes for making it as complicated as possible. Coming from Modo where setting up network rendering is a matter of a few clicks once you have it installed on all nodes, this is really frustrating.

Lewis
08-03-2013, 01:49 PM
Yep that's correct, if you use Amleto you don't need to setup scremanernet at all, just point out Amleto which Layout configs you use (pick folder) and install server/clients on each node and that's it, you can pause, resume, stop nodes, add more nodes during rendering process, render split image across multiple machines, etc, etc.. Basically all what should proper native contorler do but it's not in LW sadly so we use mostly 3rd party :(.

paulhart
08-03-2013, 02:21 PM
Lewis is "spot" on. Amleto works as expected. Any errors, have always been "user related" and remedied by closer following of the directions. Way simpler than ScreamerNet. You can set up the Master on your current station, create one or more clients on the same station, (RAM and processor dependent) and setup clients on other stations. I highly recommend try running TightVNC on the main station, to establish clear open links to the other stations, working out the kinks of talking to your own machines. It also helps you from running down the hallway to problem solve. If TighVNC runs clean, Amleto is going to work fairly easily. You are lucky on modo. I have had it from the beginning and have yet to get the network rendering to work, try as I might, with each new version.

Sebasvideo
08-03-2013, 04:47 PM
You are lucky on modo. I have had it from the beginning and have yet to get the network rendering to work, try as I might, with each new version.

It's true that many people seem to have problems with Modo's network rendering, and I was one of them until I realized that the problem was a feature Time Warner Cable has for which if you mistype an URL it takes you to their own landing page. That seemingly inane feature was causing Apple's Bonjour (which Modo uses for its networking) to mess up and not recognize the slaves. Once I turned that feature off, it was all smooth sailing, so you might want to check if that if your ISP is TWC or even if it's another company. The easiest way to test that is to disconnect your cable model from your router. If it works fine after that, then chances are it's an ISP related problem.

I'll give Amleto a try and see what it does.

Qexit
08-03-2013, 05:54 PM
If you use this then you don't have to setup Screamernet at all? I started reading the manual on it the other day and I was so disappointed. It shows screenshots from Windows 2000 (a 13 yr. old OS), Lightwave 8 (a 9 yr. old version) and the whole process is so complicated you have to wonder if they actually gave out prizes for making it as complicated as possible. Coming from Modo where setting up network rendering is a matter of a few clicks once you have it installed on all nodes, this is really frustrating.This brings back some old memories. Way back in October-December 2000, I seemed to spend all my time sending people copies of the instructions for setting up ScreamerNet compiled by Stu Aitken and then helping them sort out any resultant problems. Do these still resemble the instructions in the current manual ?:

12. steps to succesful network rendering:

1. setup lightwave on a shared folder visible to the whole network. if
mapped make sure its mapped to the same letter on each machine - I'd
recommend ditching drive mapping and just using unc names eg:

\\server\lightwave\ instead of z:\lightwave\

(obviously these are example names - please substitute with you're
actual network reality)

you only need the programs, plugins and command folders - make a config
directory there as well

2. copy you're configs into the config directory

3.open up lw3.cfg and change the command directory line to point at
you're new lightwave folder eg:

CommandDirectory \\server\lightwave\command\

4.open up lwext3.cfg in wordpad and make sure ALL plugin references
point to the correct locations on the networked lightwave directory -ie
they should all look like:

{ Entry
Class "AnimLoaderHandler"
Name "QuickTime"
Module "\\\\server\\lightwave\\plugins\\input-output\\QTTools.p"
}

5. go to the programs directory and make a shortcut to lwsn.exe - enter
the properties to this shortcut and set it to the following:

\\server\lightwave\programs\LWSN.exe -2 -c\\server\\lightwave\config
\\server\lightwave\command\job1 \\server\lightwave\command\ack1

6. to make a machine a node simply drag a copy of this shortcut onto the
desktop, then enter its properties and set the two ID tags to something
unique (ie the bits where its says job1 and ack1 are changed to job2 and
ack2 or whatever)

7. thats it - easy - double click to start node.

8. changing the content dir entails changing the content dir line in the
lw3.cfg file in \\server\lightwave\config\ then saving it.

9. make another shortcut, this time to lightwav.exe on
\\server\lightwave\programs

10. change its command line in the properties tab to:

\\server\lightwave\programs\LightWav.exe -0 -c\\server\lightwave\config

11. open up the license key file in \\server\lightwave\programs\ and add
ALL the license key codes for each registered copy of lightwave you
have, on a seperate line each, then save the file.

12. drag this shortcut onto every desktop where you might want to
control screamernet from - ONLY use that shortcut to launch lightwave to
control the nodes

tada - instant screamer net - if you're smart you'll just run lightwave
from the network full stop, makes updating configs, plugins, etc across
a network dead easy.

Those were the days :D

Sebasvideo
08-03-2013, 06:40 PM
Thanks Kevin, yes, it seems about the same although I didn't compare all of it, but well, if Almeto works I don't even care about Screamernet. I think the name comes from the screams of frustration when you start reading all the steps to set it up, and I think the founders of Luxology said, well, Lightwave has the most needlessly difficult and frustrating possible method for network rendering, so let's make a point of making the easiest possible, and they did.

Qexit
08-03-2013, 06:53 PM
Newtek did have a go at creating a Screamernet controller of their own called StealthNet but it never worked very well and it was easier to just go back to ScreamerNet. Another third party solution that passed away was called LightNet and there was also one called TequilaScream. I eventually opted for ButterflyNet but I haven't kept my license upto date on that one, so I'm not sure where it is up to these days. I only have a pair of PCs, so haven't really done a great deal with network rendering for a while.

Qexit
08-03-2013, 06:59 PM
I think the founders of Luxology said, well, Lightwave has the most needlessly difficult and frustrating possible method for network rendering, so let's make a point of making the easiest possible, and they did.The jury is still out on that one. Modo's network rendering is still very hit and miss, even though it is very simple to set up. The problems stem from their use of Bonjour which has a lot of problems which can result in PCs/nodes simply not being visible no matter what you do. For some people it works like a dream straight off, for others it never works no matter what they do. Luxology are aware of the problems and have said they will be addressed at some point. This may sound familiar :-)

Phil
08-03-2013, 07:11 PM
Newtek did have a go at creating a Screamernet controller of their own called StealthNet but it never worked very well and it was easier to just go back to ScreamerNet. Another third party solution that passed away was called LightNet and there was also one called TequilaScream. I eventually opted for ButterflyNet but I haven't kept my license upto date on that one, so I'm not sure where it is up to these days. I only have a pair of PCs, so haven't really done a great deal with network rendering for a while.

LightNet is still around : http://www.joejustice.org/lightwave/lightnet/index.html

And there's a 'cloudy' variant, as is all the rage these days :

http://www.joejustice.org/lightnet-cloud

Sebasvideo
08-03-2013, 07:15 PM
The jury is still out on that one. Modo's network rendering is still very hit and miss, even though it is very simple to set up. The problems stem from their use of Bonjour which has a lot of problems which can result in PCs/nodes simply not being visible no matter what you do. For some people it works like a dream straight off, for others it never works no matter what they do. Luxology are aware of the problems and have said they will be addressed at some point. This may sound familiar :-)

I know, but I would bet money I don't have that most of the people have the problem I had, their ISP having some pointless feature that messes up Bonjour unless said feature is disabled. Before I found that out recently, I was among those who hated Modo's network rendering, after changing that it recognizes my two slave PCs each and every time. Of course I'm not arrogant enough to say that if it worked for me it will work for everyone, but since TWC is a very popular ISP I bet many people have it and is not aware of it, even though I posted about this in the Lux forums, but of course not everybody read it. Then it's logical to think that Comcast and other ISPs have the same feature and is also messing up their customer's networks, so I'm sure that many Modo users suffer from the same problem and don't even know it.

However, they said they intend to fix it completely during this release cycle, not wait until 801 or 901 to fix it, and we're told they're working really hard on it. As for Lightwave, I just hope that they realize they don't have a good network rendering module and come up with something better, with buckets or whatever is needed to render one frame across multiple nodes, and far easier to setup. If they want to make it even easier than Modo they could release an installer for only the slaves, which installs only the files needed for network rendering, and sets up an icon that launches the process with a small window that shows the progress and other related information. I don't think it's too much to ask for, I don't expect the kind of network management software a professional render farm would use, I expect something basic that works and is easy to setup.

Qexit
08-04-2013, 03:36 AM
LightNet is still around : http://www.joejustice.org/lightwave/lightnet/index.html

And there's a 'cloudy' variant, as is all the rage these days :

http://www.joejustice.org/lightnet-cloudThat's for the links Phil. I didn't realise Joe Justice was still keeping LightNet afloat. Kudos to him for persisting. Interesting to see he was a couple of years ahead of the field by introducing a cloud version, though not something I will be trying out.

Qexit
08-04-2013, 03:55 AM
I know, but I would bet money I don't have that most of the people have the problem I had, their ISP having some pointless feature that messes up Bonjour unless said feature is disabled. Before I found that out recently, I was among those who hated Modo's network rendering, after changing that it recognizes my two slave PCs each and every time. Of course I'm not arrogant enough to say that if it worked for me it will work for everyone, but since TWC is a very popular ISP I bet many people have it and is not aware of it, even though I posted about this in the Lux forums, but of course not everybody read it. Then it's logical to think that Comcast and other ISPs have the same feature and is also messing up their customer's networks, so I'm sure that many Modo users suffer from the same problem and don't even know it.

However, they said they intend to fix it completely during this release cycle, not wait until 801 or 901 to fix it, and we're told they're working really hard on it. As for Lightwave, I just hope that they realize they don't have a good network rendering module and come up with something better, with buckets or whatever is needed to render one frame across multiple nodes, and far easier to setup. If they want to make it even easier than Modo they could release an installer for only the slaves, which installs only the files needed for network rendering, and sets up an icon that launches the process with a small window that shows the progress and other related information. I don't think it's too much to ask for, I don't expect the kind of network management software a professional render farm would use, I expect something basic that works and is easy to setup.I have both Modo (I have been using it from 102 through to 701) and Lightwave (every freestanding version from the Amiga 3.0/3.1 through to the latest 11.6 release for the PC) and do prefer Modo's network rendering setup. Although I've been pretty inactive with both for the last 12-18 months, I do keep up to date with activities through their respective forums and websites. I very much doubt if Newtek will be changing their renderer to be a bucket system any time soon as I'm pretty sure that would involve a complete rewrite. As the quality of Lightwave's renderer is one of its main selling points it would be foolish of them to jeapardise that by ignoring the old maxim of 'it aint broke, don't fix it'. Both products have areas that would benefit from being addressed in their current cycle of releases. The one they have in common is network rendering. So I hope both companies put in the time and effort to sort things out before their next full point releases as it would be welcomed by a lot of their users.

dsol
08-04-2013, 08:17 AM
There's waaaaay more important things to fix in LW before they worry about changing to a bucket-based renderer. Of course, they might be able to support single-frame multi-node rendering by splitting scanlines across multiple machines. But compared to the other work needed in LW (new geometry engine, unification, modifiers, referencing, full working undo system etc.), it doesn't seem quite as urgent to me.

Though this does beg the question - are there any 3rd party render engines for LW that do support multi-machine single frame renders (KRAY, Octance, Arnold etc.)?

DaveyWaver
04-05-2014, 06:10 PM
Hi Lewis,

May I ask, have you had any luck using Amleto with wifi connections ?

I can only connect two machines together via eithernet connection. Any machines trying to connect via wifi don't succeed.

Have you had similar problems ?

DaveyWaver

Sebasvideo
04-06-2014, 01:30 PM
Hi Lewis,

May I ask, have you had any luck using Amleto with wifi connections ?

I can only connect two machines together via eithernet connection. Any machines trying to connect via wifi don't succeed.

Have you had similar problems ?

DaveyWaver

I have four machines on ethernet and one laptop using wi-fi, all five work perfect with Amleto. Of course the laptop is much slower because it's a laptop, and it takes longer to copy the content folders because it's connected at 300 Mbps while the others are at 1 GB.

Lewis
04-06-2014, 02:10 PM
I'm usin LAN coonection torugh Gigabit Switch so i can't say i ever tried WiFi connection, sorry :(.