Page 1 of 2 12 LastLast
Results 1 to 15 of 29

Thread: S-L-O-W Screamernet (LW9)

  1. #1
    Curmudgeon in Training Ma3rk's Avatar
    Join Date
    Mar 2003
    Location
    Near Beaverton, OR
    Posts
    1,392

    S-L-O-W Screamernet (LW9)

    Hey All,

    I posted this on the LW newsgroup but haven't had much response so thought I'd try here.

    I finally had some time last weekend to bite the bullet and get ScreamerNet working for LW9. I even did a fresh install to remove spaces from LW9ís install path. I pretty much just followed the pdf file made changes to the .bat files, and it fired up the first try. WooHoo!

    I got all four nodes going and tried a simple test scene from the CDís content and it rendered quite speedily.

    So, decided to throw a scene Iím currently working on and all nodes hung. Hrrumph!

    Found some notes regarding Spreadsheet.p being finicky with SN so removed that from the scene and tried again. That did the trick, all nodes load the scene and begin rendering.

    But then I notice the render times per frame. An F9 render was taking 2 min, 55 secs. Each node of SN is taking around 11-12 mins. Per frame.

    Node 1: XP, AMD Athlon 64, 3200+ Mhz, 2Gb Ram

    Node 2 & 3: Win2000, Dual Intel P-III 700 Mhz, 1 Gb Ram

    Node 4: XP, AMD Athlon, 2800+ Mhz, 1Gb Ram


    Thereís a message on the Newtek Discussion forums

    http://vbulletin.newtek.com/showthre...eferrerid=3153


    In particular quoting SCS5:

    ďIf you look at the task manager on ANY OTHER PC on the render farm, it is using virtually NO RAM, OR PROCESSOR RESOURCES! While the host PCís Processor, & Ram usage are buried! And render times on the Render Nodes was sometimes 3 or 4 times slower than rendering the same project on a single pc!Ē

    This is exactly what Iím experiencing, but doesnít appear to be universal either.


    So, whatís up Docís? Where do I even begin to track this down?

    Thanks,

    Ma3rk

  2. #2
    Not sure if I can completely answer your problems, but here goes:

    Is your host machine (the one you hit F9 to do your test renders) a dual processor? If so, that render time will be faster than your render nodes. The nodes are rendering with only one processor when they do a frame.

    If I hit F9 on the machine that I work on ( a dual 3.4 ghz xeon), I get a certain time per frame. I am set to 2 threads (or more depending on what I'm doing) in Lightwave when I hit F9. My render nodes take between 2 to 3 times longer per frame to render because they are not multi-threaded when screamernet renders. They are also slower processors (3 ghz).

    A more accurate test render would be to set Lightwave to 1 thread, then do an F9. Your time will be longer and perhaps closer to the time of your screamernet nodes.

    Also, the main thing that I needed to do to get screamernet to work was to put the " " marks around the path names. Screamernet would not work for me until I did this.

    For instance: "C:\Program Files\NewTek\LightWave 3D 9\Programs" LWSN.exe -2

    instead of: C:\Program Files\NewTek\LightWave 3D 9\Programs
    LWSN.exe -2

    Hope this helps.

  3. #3
    Curmudgeon in Training Ma3rk's Avatar
    Join Date
    Mar 2003
    Location
    Near Beaverton, OR
    Posts
    1,392
    The basic problem is even though SN is working, it's considerable slower than previous experience.

    The host machine is a single CPU AMD 64 Athlon. Lightwave is set to Single Thread.

    I did a clean install removing all spaces in the base Lightwave folder. It's odd that the default installation adds spaces.

    Mark

  4. #4
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Ma3rk,
    I just wanted to add, I have this problem aswell..1min per frame for 1 pc @ 60 frames =1 hour..5 pc's through screamernet also took 1 hour...Whats going on..
    Your not alone...

    Host =2.8ghz cpu
    pc 2-5 =2.4ghz cpu

    Ivan.

  5. #5
    You guys should use a different set up for this. Or at least give it a try...

    Try using a bat file like below for your LWSN start up.

    ## Launch a Screamernet node
    ## Edit the directory paths to suit your system
    echo init x:\commands\job8
    LWSN.exe -2 -dx:\ -cx:\LW9-64bit\Programs\LW9-64.cfg x:\commands\job8 x:\commands\ack8

    I find if you use a config like above, and have each machine run lightwave.exe from that same executable directory from the shared content drive (in my world that's x:\) and then scan for plug-ins from the location x:\LW9-64bit\ directory and then close out. Don't forget to set your content directory!
    Your host machine should also run the same executable for lightwav.exe from that directory, and do the same thing, open up scan plugs and then close.

    copy your "host" machine's config file, and paste it to the location suggested in the LSWN bat file example above.

    Make sure they can all see X:\ and then run those bat files (screamerA.bat, ScreamerB.bat, ScreamerC.bat for each machine A, B,C)

    Whatever you have for your host system RenderThreads=whatevernumber will be carried over to the nodes correctly this way when you do the copy paste from your host configs. Now when I say the host configs I mean the one you have in your documents and settings folder on the host box.

    Give that a go and see if your problem goes away. If not, try copying the config from your host machine into the same directory for documents and settings on each of the nodes.

    I will tell you however the problem you are experiencing is something I have never heard of or experienced myself unless a node is in "single threadmode".
    DJ Lithium | KAT! =^..^=

    Black Tiger Recordings
    http://www.blacktigerrecordings.com
    Founder and Citizen of Liberty3d.com
    www.liberty3d.com

  6. #6
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Hey djlithium,

    I'll make a little video of my setup and post a link, so you can check it out..If thats cool, much easier to show you then try to explain...Hopefully you can tell me if I have done something wrong.

    Thanks ,
    Ivan.

  7. #7
    Ok kewl.
    Keep me posted.
    I don't run LWSN raw usually but even through smedge/deadline the same rules apply. A system will not multi-thread on its own unless given the correct command to do so on start up and the way to do that is to ensure its pulling the same config settings either set locally on each box or from a specific file. Of course that is obvious, but I know people get frustrated with Screamernet at times because the obvious seems anything but when troubleshooting the reasons for it working here and not working there when both systems are apparently using the same configs.

    Fire me off a link with your video when you have it and I will for sure take a look.

    email me direct at [email protected] (remove the nospam_) and that's me.
    DJ Lithium | KAT! =^..^=

    Black Tiger Recordings
    http://www.blacktigerrecordings.com
    Founder and Citizen of Liberty3d.com
    www.liberty3d.com

  8. #8
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Hey djlithium...Your a champion...I'll do that!

    Before I do, can you shed some light on what you mean about multi-thread..Has that something to do with multiple cpu's on the same pc? or muliple pc's rendering the scene..It kind of sounds like this MAY be my problem..not sure. My pc's are single cpu's...
    My SN lightwave setup has the configs saved in their respective folder (screamernet_config etc) and they are shared folders.

    Ivan.

  9. #9
    Well that depends on the processor type.
    Intel CPUs such as the xeons show up as "two cpus" in Windows even if there is one, if the model of the CPU supports "hyperthreading". In a dual Xeon system, it shows up as 4 CPUs in your task manager peformance monitoring area.

    Let's say you have a single hyperthreading technology CPU running LWSN. If your LWSN config (not the screamer.bat file!) file is not set to "multithread" it may not consume more than 50% of the processing capability of the processor because its only "seeing" the first "virtual" for lack of a better word, CPU. The affinity for LWSN may be set to use both CPU 0 and CPU 1 but that just means that the it will spread that work in a single thread process across those two virutal CPUs. This of course may not be the real technical explaination as to what is going on but I am trying to keep it simple here for explaination purposes and because its 3AM.

    Regardless of what its doing, the CPU is not being forced to engage its 'hyperthreading' capability and consume 100% of the total system CPU useage because of this virtual split. Set it to anything more than 1 thread, and it forces the CPU to engage this technology and use the full capabilities of the processor. At least that is what happens on our systems now with multi-threading fixed in LWSN 9.0 build 998 (release, although this was fixed early on in beta for 9.0 - thanks again newtek for fixing this! AMAZING! FAST STUPID SPEED and all the ram in the system could be used on a single LWSN job).

    This can be demonstrated by using a dual CPU system with hyper threaded CPUs from Intel. If you want, you could set LWSN to use 2 threads on such a box and with the intel processors it will use two of those virtual CPUs listed in the task manager. Set it to 4 and it will use all four CPUs. Or, launch one LWSN with single multi thread and it will consume 25% of the total resources (4 divided 4 = 25% or one CPU) Interestingly enough on a dual cpu opteron systems they (the processors) show up as simply two CPUs in the task manager instead of four even though each processor has AMDs version of hyperthreading technology.
    On your xeon box if you have one or a P4 I guess, launch another LWSN and it will grab a hyperthread and consume another virtual CPU, for a combined amount of 50% of total CPU power. Launch another single thread LWSN and it will go to 75%. Add another, and you get 100%

    With multi-threading turned on in a single LWSN on a dual xeon box it should consume 100% of the CPU power now, where as it didn't before in pre 9.0 version. So, if you are not seeing 100% of the CPU being consumed it means that the config file that LWSN is reading on launch is not set to RenderThreads=higher than one. So again, that is why I use the set up I recommended before with the config file switch, and just to be a jerk about it I launch LW itself on each machine (you can do this and set it without a dongle, LW will save your settings in discovery mode) and check the render threads in the renderglobals and set it to 4, and set my segment memory limit to where I want it as well. What this does is it makes sure that if for some reason the LWSN is pulling its config from the local lightwave config file that is present in the documents and settings folder for that systems user you are running LW or LWSN on, matches your intended settings, just in case it can read the common config file on the shared directory. Try this! If you see plug ins failing to load because it can't find them, make sure that systems local lw configs are set to pull from the shared directory, make sure the content directory is set the same across every machine as well. Then pick one box where you set in the renderglobals the render threads in layout itself to be 1 thread instead of 4 and run a LWSN job across all of them and compare the CPU consumption on each one. I think what happens when LWSN can figure out what the shared config file is or if the config file cannot be read properly it defaults to its own local config or a default internal to LWSN setting of 1 thread. Based on my experience, this holds true.

    Remember the place for render threads is in a different config file than say where your plug ins are located. PLug ins may work on all of your systems fine, but if one isn't multi-threading correctly or has a low segment memory limit (like the default 32mb) you will see this immediately as being reported as "rendering segment 1/3" on 1280x720 resolution frames or in the task manager processor consumption meter as being 25% or 50% depending on the real CPU count and hyperthreading capability.

    Again opterons don't have this problem, nor do athalons except when you get into multi core CPUs. A dual core, single CPU system will list two CPUs on an AMD X2 based box. I think thats just because AMD has a better way of doing things frankly.
    Also don't be afraid to run multi-thread for things like HVs and even weird plugs. I have yet to run into problems with multi threading shots with hypervoxles in them or third party plug-ins like Dynamite or EXRTrader. I think that its reasonably safe to say that for LW9, you can multithread all you want and it won't go wonky on you. Even if a plug-in doesn't support multi-threading I am guessing the worst thing that will happens is that while that plug in process isbeing called by LWSN's rendering engine, your performance will drop to a single thread worth of CPU power.

    Also you should know that during object load, and some moving poly operations that the CPU may not run at 100% and might drop to "single thread" peformance levels. This is because 1. some of these operations either cannot be optimized for multithreading (how to you multi thread loading an object???) or because it happens so fast in succession over several objects being moved in the scene that the CPU never gets a chance to "kick over" pass the single thread step or at least appears not to do so because its moved on to the next object and starts from zero again by the time windows has caught up with reporting that to you visually. Word of the wise, and I am having a ***** of a time getting people on board at work with this... BUILD YOUR OBJECTS INTO SINGLE LAYERS IF POSSIBLE or as few as you possibly can, because if you have a tonne of objects with single layers you are going to never get past that 25% consumption rate on a dual xeon box showing up as 4 CPUs because that process is not optimized, or can never kick in to take advantage of it (I think its the previous reason). YOu may find that the rendering process itself takes less time than the actual "moving objects/loading objects" portion. In some extreme cases on BSG, upwards in the area of 90% of the time spent "rendering a frame" is consumed by "moving objects". This isn't newtek's fault really, I think that kind of thing to optimize would be a serious challenge, it really comes down to how your scenes and objects are constructed. This isn't 1996 kids. This is 2006, and LW9 is king for speed so don't build your assemblies and objects as if you have the same kinds of object size limits from legacy versions of LW running on computers that barely qualify for use in 3rd world countries. That's not a problem any more (And hasn't been for a long time) so please put those Babylon 5 VFX production methods and mentality off to the side and under a desk.
    DJ Lithium | KAT! =^..^=

    Black Tiger Recordings
    http://www.blacktigerrecordings.com
    Founder and Citizen of Liberty3d.com
    www.liberty3d.com

  10. #10
    Sorry, this part "BUILD YOUR OBJECTS INTO SINGLE LAYERS IF POSSIBLE or as few as you possibly can, because if you have a tonne of objects with single layers you are going to never get past that 25% consumption rate on a dual xeon box showing up as 4 CPUs because that process is not optimized" should read as follows

    "BUILD YOUR OBJECTS INTO SINGLE LAYERS IF POSSIBLE or as few as you possibly can, because if you have a tonne of objects with A TONNE OF layers you are going to never get past that 25% consumption rate on a dual xeon box showing up as 4 CPUs because that process is not optimized"
    DJ Lithium | KAT! =^..^=

    Black Tiger Recordings
    http://www.blacktigerrecordings.com
    Founder and Citizen of Liberty3d.com
    www.liberty3d.com

  11. #11
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Um...WOW!,
    Thats awsome mate!..I have to say, I'm not that smart and a fair bit of that went over my head..I'm going to have to read it a few more times, but atleast this way it gives me areas to research...
    I have done a few more tests and can definatly say only one nodes is rendering at a time..

    You have done more than enough..Thank you so much!...Now go to sleep.hehe

    I'll read it a few more times and I'll get back to you..

    Thanks again!!!
    Ivan.

  12. #12
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Um...WOW!,
    Thats awsome mate!..I have to say, I'm not that smart and a fair bit of that went over my head..I'm going to have to read it a few more times, but atleast this way it gives me areas to research...
    I have done a few more tests and can definatly say only one nodes is rendering at a time..
    The thing is..when you say if the LWSN config is not set mulithread etc these are the things that get confusing..That is knowing how or where to do it..

    You have done more than enough..Thank you so much!...Now go to sleep.hehe

    I'll read it a few more times and I'll get back to you..

    Thanks again!!!
    Ivan.

  13. #13
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Hi all, me again.

    I had no luck with SN..So I opted for an easy way out..I purchased "ButterflyNet Render3..personal edition 5 nodes for Lightwave" $45..Took me ages to set it up (well, not the software thats very easy..you use a wizard) but the mapping of my nodes..I have no idea about networks and mapping drives, but all the pc's have to be mapped to the same shared folder...who knew?

    Anyhoo, its all working a treat..my one hour render with one pc is now down to 15 mins with 5.

    Thanks to all that tried to help with SN.

    If anyone decides to get BNR..I'm happy to help with setting it up.

    Ivan.

  14. #14
    Curmudgeon in Training Ma3rk's Avatar
    Join Date
    Mar 2003
    Location
    Near Beaverton, OR
    Posts
    1,392
    Meanwhile...

    I'm finally getting some time to check back in on this thread. That pesky having-to-work thing keeps getting in the way.

    Anyway, WOW. I've saved out all the above replys to study over. I'll have some more specific questions a bit later I'm sure, particularly for DJ.

    But Nackers last reply answered & raised some questions. Might not a 3rd party utility be the easiest/quickest way to go? I've considered that, but thought they still relied on or otherwise required LWSN to be up & flying straight to begin with.

    Thanks,

    M.

  15. #15
    Member Ivan_B's Avatar
    Join Date
    Jan 2006
    Location
    Australia
    Posts
    292
    Hey Ma3rk,

    "I've considered that, but thought they still relied on or otherwise required LWSN to be up & flying straight to begin with."

    The BNR uses lightwave and all its files, however screamernet doesn't need to be setup at all, infact I deleted all my node .bat files, removed all the extra folders (screamer_config etc)..

    Bnr copies all the LW configs and plugins to its own folder, along with the LW programs folder, these folders along with your content folder are shared and mapped to a virtual drive for all the nodes to map to aswell..Point the nodes to the host IP, and away you go...(The mapping part stumped me a bit..never used it before) appart from that hickup, everything else is really easy.

    I suppose the only drawback is, I have a limit of 5 nodes, cos I opted for the cheap way (I only have 5 pc's anyway), but for a few more $ you can get a lot more nodes.

    Anyway, just another option.

    Ivan.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •