Page 1 of 2 12 LastLast
Results 1 to 15 of 20

Thread: Render Farm and Nodes

  1. #1
    Registered User
    Join Date
    Jul 2011
    Location
    Chicago
    Posts
    7

    Render Farm and Nodes

    Hello Fellow Lightwavers, I'm New to starting out threads and I hope I can get some helpful answers for what I'm about to ask...


    Question One : I have Been searching for many ways to go about building a small networked render farm for Lightwave, and I would Like to Know if anyone has such a set up and How can I go about putting one together. I am a PC user with AMD Processors... I am set to what I want to use but I just want to make sure that I invest on the required Hardware to set up a Render Farm for Five PCs. Any Help would Be greatly Appreciated and I would Like to thank everyone who responds ahead of time! 8),

    Question Two : I have Lightwave 10 Educational Version but Im going to upgrade to the commercial version with 11. Does Lightwave 11 have unlimited nodes for rendering and is Newtek still using screamernet or have they updated their render farm application? Again I thank you all for your answers and your Help!



    Sincerely, Faithful Lightwave User
    from the moment I got
    my hands on Inside
    Lightwave 9

  2. #2
    Super Member JonW's Avatar
    Join Date
    Jul 2007
    Location
    Sydney Australia
    Posts
    2,235
    There is quite a bit in the SN section. It's especially worthwhile reading Matt's tutorial on setting up SN, & it's worth more than that as it gives you a clear understanding of your network & SN. http://forums.newtek.com/showthread.php?t=26572

    Remember-no-gaps-in_any_file_names! Most of us learn the hard way!

    LW does not have unlimited nodes, so if you need more than a thousand nodes you will need another licence/s. (& a much better connection to the mains power!)

    If your only purpose for the other 4 computers is render nodes, build the cheapest box you can possibly put together, eg built in graphics to set up the computer. If you are using a computer with a graphics card you can take it out once you have set up LW/SN as it doesn't need one & they only waste electricity.
    Last edited by JonW; 05-17-2012 at 03:00 PM.
    Procrastination, mankind's greatest labour saving device!

    W5580 x 2 24GB, Mac Mini, Spyder3Elite, Dulux 30gg 83/006 72/008 grey room,
    XeroxC2255, UPS EvolutionS 3kw+2xEXB

  3. #3
    Registered User
    Join Date
    Jul 2011
    Location
    Chicago
    Posts
    7
    Thanks for the advice JonW, I greatly appreciate your answer!

  4. #4
    Animated Fool paulhart's Avatar
    Join Date
    Mar 2003
    Location
    San Diego, CA
    Posts
    758
    Please take a good look at Amleto. It is well maintained, recently updated, and no cost, except if you want to donate?? Works extremely well, transfers all of the files needed without effort and the set up is not as difficult as "Screamer-----Net" aptly named, me thinks... There is a thread on this forum.

    Also, download and install TightVNC (OpenSource) If it is working fine, you know that you have all of the network settings correct, and it allows me(you) to monitor each of the other stations from a central system. I enjoy the ease.
    Last edited by paulhart; 06-07-2012 at 08:52 PM. Reason: Further info
    An eye for an eye, and the whole world goes blind... Gandhi/MLKing
    Greater consciousness is worth the effort. Be aware.... so sayest the Animated Fool ~ HARTWORKS www.hartworks.net

  5. #5
    LightWave documentation BeeVee's Avatar
    Join Date
    Feb 2003
    Location
    Pessac
    Posts
    5,060
    I can thoroughly second PaulHart's recommendation of Amleto. At the art school I teach at we have 10 machines that are no longer used for 3D even though four years ago they were very powerful. Using Amleto means you don't even need to install LightWave on machines other than the host, it's brilliant. Thanks for your recommendation of TightVNC Paul, I need to replace TeamViewer since we have no budget to buy the commercial version.

    @JonW: I'm pretty sure it is unlimited nodes actually. The truth of the matter is of course that controlling that many machines is going to be difficult

    B
    Ben Vost - NewTek LightWave 3D development
    LightWave 3D Trial Edition
    AMD Threadripper 1950X, Windows 10 Pro 64-bit, 32GB RAM, nVidia GeForce GTX 1050Ti (4GB and 768 CUDA cores) and GTX 1080 (8GB and 2560 CUDA cores) driver version 430.86
    AMD FX8350 4.2 GHz, Windows 7 SP1 Home Premium 64-bit, 16GB RAM, nVidia GeForce GTX 1050Ti (416.34, 4GB and 768 CUDA cores)
    Dell Server, Windows 10 Pro, Intel Xeon E3-1220 @3.10 GHz, 8 GB RAM, Quadro K620
    Laptop with Intel i7, nVidia Quadro 2000Mw/ 2GB (377.83 and 192 CUDA cores), Windows 10 Professional 64-bit, 8GB RAM
    Mac Mini 2.26 GHz Core 2 Duo, 4 GB RAM, 10.10.3

  6. #6
    Super Member dwburman's Avatar
    Join Date
    Feb 2003
    Location
    Washington State, USA
    Posts
    2,880
    Newtek has not updated the features of its render controller for LW, but there are others out there. They all use the same lwsn.exe file that the built-in controller (ScreamerNet) uses.

    When network rendering here's what you need to be careful about (probably in Matt's guide):

    File Paths: All the files (images, objects, .mdd files, etc) need to be accessible to all the render nodes. The best way to ensure that is to have all your assets in the same folder and set the Content Directory to that folder. I like to have a separate content directory for each project (sometimes for each scene). It's easy to grab an image texture from somewhere else on your hard drive and forget to put in your content directory/networked drive.

    You will not get a nice big dialog box saying that "x file cannot be found. Do you want to continue?" You will either get

    This goes the same for setting the location to save the frames to. I think some things (some savers from the image processing section or some mdd loaders) may not use relative paths. If you come across any of those, be sure to use a network path and not a local path.

    Plugins: Your plugins have to all be in a networked location, or copied to all computers in the same location. You may have to go to the plugin editor, locate the plugin through the network path ("\\mycomputer\...\plugins" instead of "c:\...\plugins") and then quit LW so the network path is saved in the configs. You could also just open the config file (the one with "ext" in the name) in a text editor and do a search/replace on the file paths.

    If you've copied the plugin files to all the computers you don't have to worry about the network path step, just keeping things synced.

    Usually, any problem I've had with a network render has been from one of those two things, and they are essentially the same problem.

    Once a render farm is set up, it's usually fairly smooth sailing. Troubleshoot can be a long and painful process so it's best to set up a render farm before you need it. You can waste a lot of time trying to set up a network render at the last minute.
    [mr] Dana W. Burman | http://dwburman.com
    Motion Graphics for sale at Pond5 Stock Footage
    Check out my tutorials at Liberty3d.com & my YouTube channel!

  7. #7
    I've run render farms for companies like darkside, Pixomondo and now Rhythm and Hues, so I think I can say I have some experience with this stuff.

    It's been a LONG time since I looked at anything but Butterfly Net Render or Deadline, and I still maintain they are the best in their respective classes.

    In fairness, it has to be said that Amleto is priced at $75 for a 5-node setup.

    BNR Pro will cost you $125 for a 5-node setup and you can add additional nodes as required at reasonable cost. It will also support additional software such as Fusion and even (ugh!) other 3D packages.

    If you're going to get REALLY serious, Deadline starts at about $15,000...

    Network rendering is NOT rocket science, it's about being methodical. Get one machine set up right and you are 90% of the way there. Lots more on my site at www.simon-coombs.com, although it is due for an update...

  8. #8
    Animated Fool paulhart's Avatar
    Join Date
    Mar 2003
    Location
    San Diego, CA
    Posts
    758
    I agree that Butterfly is a good choice and well suited for the larger network setup running different apps. Need to disagee about the pricing on Amleto. It is FREE. I originally bought the licensed version and then shifted over to the free version as Steve Moody updated the code to C#, added some features and released it to the community. He was one of the original developers and did not want to see it "waste away" when his colleagues stopped commercial development. Much thanks to Steve Moody!! Try it out. Once I was sure that TightVNC could talk to each of my nodes so that I knew that the network settings were correct, I installed the Server and Client on the main station and the Client on each of the nodes. I set a \temp directory locally on each node for it's use and a shared network visible folder for the complete project of the moment. Amleto copies the complete folder to the local stations automatically, so I keep it "lean" without multiple versions of files or construction assets. Amleto also copies over your configs, programs and your plugins to the local station to speed up the local stations. It only does this at start up unless you have made changes. After that it renders the frames to the \temp folder on the local machine, then uploads and renames them to the shared folder when each completes it set of frames. It really does work very smoothly and I continue to appreciate Steve Moody's contribution to the community.

    [url] http://virtualcoder.co.uk/amleto/ [\url]
    An eye for an eye, and the whole world goes blind... Gandhi/MLKing
    Greater consciousness is worth the effort. Be aware.... so sayest the Animated Fool ~ HARTWORKS www.hartworks.net

  9. #9
    Registered User
    Join Date
    Jul 2011
    Location
    Chicago
    Posts
    7

    Wow

    Thank you all For Being so generous with all the advice, I Hardly get a chance to jUmp on to read, But Im glad you are all awesome and Helpful, Thank you all Again, I have Much reading to Do!

  10. #10
    Registered User
    Join Date
    Jul 2011
    Location
    Chicago
    Posts
    7

    Hello Again

    Okay, So this is what I have as my set up as I am Trying to get to 4 Machines before windows 8 comes out

    PC 1: HP Pav. Elite HPE 500 y with Lightwave Installed

    : Windows 7 Home Premium
    : AMD Phenom II x6 1045T Processor 2.7 GHz
    : 64 bit (Obviously)
    : 8 GB DDR3- 1333 PC3 10600 MB/sec Installed Memory
    : AMD Radeon HD 6770 Video Card
    : Corsair GS600 Power Supply
    : 1.5 SATA 5400 rpm HD

    PC 2: HP Pav. HPE H8 - 1214

    : Windows 7 Home Premium
    : AMD FX-6100 Six core Processor 3.3 GHz
    : 64 bit
    : 10 GB DDR3- 1333 PC3 10600 MB/sec Installed Memory
    : AMD Radeon HD 7450 Video Card (came with as default)
    : HP 300 watt Power Supply
    : 2TB SATA 5400 rpm HD


    So these are my babies that I want to use to start my render farm and I also will be Upgrading the Memory to 16GB and Hope to Get Two More PCs with AMD Processors.

    I have them LAN together with a Crossover cable but I also have a wireless router and I was wondering what would be best as to how I should have screamernet o any other type of render engine I will decide to use in the future.

    Maxwell render has always been a Choice I felt I would lean to as soon as I had all 4 PCs


    Please advise as How I should Go about having my setup, because as I work on my Modeling, Texturing and Lighting, I would Like to have this out of the way, My Goal is to Build My portfolio and Have a Short made.

    P.S. Has anyone attended the Pixar Masterclass before ?

  11. #11
    Registered User
    Join Date
    Jul 2011
    Location
    Chicago
    Posts
    7
    Thank you all in Advance
    HP Elite HPE 500y, Windows 7 64bit, AMD Phenom II x6 1045T 2.7 GHz, 8 GB Ram, AMD Radeon HD 6770
    HP HPE H8 1214, Windows 7 64bit, AMD FX 6100 Six core Processor 3.3 GHz, 10 GB Ram, AMD Radeon HD 7450

  12. #12
    Registered User gravin's Avatar
    Join Date
    Dec 2011
    Location
    CNY
    Posts
    60
    I don't want to hijack this thread but I figure this would be a related bit of info for anyone looking at setting up there own farm. What does BNR consider a "node" to be? I know some managers do this by cpu, others by core or thread and a few by machine. For example Squidnet counts each core or thread as a node but I believe Deadline counts each machine as a node no matter how many thread or cores they are capable of.

  13. #13
    A node is one rendering task.

    So if you had a machine with, say, eight Xeon cores, you could set it up to be one, two, four or eight nodes.

    The optimum choice would depend on memory and lots of other factors.

    Simon

  14. #14
    One Man Army Andy Meyer's Avatar
    Join Date
    Dec 2009
    Location
    Lucern, Switzerland
    Posts
    1,200
    Dude, crossover lan cable?
    ok, thats a start and it will work with two machines...

    to connect many devices to each other you need a lan switch. this is the center of the network. connect every pc to the switch with a lan cable. also connect your wireless router or your internet router/firewall to the switch. all devices connected to the switch can talk to each other. you can even connect a second switch to your main switch to expand overall connections (stacking).
    i think for your plans an 8port gigabit switch will be good enought (4 nodes, 1 workstation, 3 spar). 50$-100$ for e cheap one. i love the netgear pro series.
    one thing: there are many fanless/noiseless good 8port switches. dont buy a rackmount/server switch, they can be loud!
    my website: www.andymeyer.ch
    Supermicro 7047A-T/X9DAi: dual E5-2687 8C/16T @3.1ghz | 64GB Ram | nVidia GTX680-4gb (301.42) | Win7 64 Bit (SP1) |
    display 30"+30"+22" | 13 rendernodes, 54 cores

  15. #15
    As far as what a node is, what I had done was use hyperthreading on my CPUs so a dual core machine could act as 4 nodes. At least in BNR. I believe it's the same with Amletto.
    ~~Phil Nolan
    www.philnolan3d.com - Twitter - YouTube
    Win 10 Home x64, Core i7 3.4 GHz, GeForce GT 640, 16GB RAM
    Laptop: Win10 x64, Core I7 8th gen, 1.8 GHz, 16GB RAM

Page 1 of 2 12 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •