PDA

View Full Version : my screamernet requests



eblu
02-13-2003, 11:09 AM
my wish list for LW screamernet:
I am assuming a complete re-write, and i am making a pie-in-the-sky list of what i want in screamernet, feel free to chime in...

1. screamernet now has a messaging system, it can issue commands without writing to files. in this fashion we get away from file permission and other legacy problems.

2. A rendering network is considered a pool instead of a two-level hierarchy, with no limit to the amount of clients or Hosts. In fact a netwrk could easily have multiple hosts running renders at one time, or a particular client could belong to any number of rendering networks.

3. Screamernet is now a separate aplication from lightwave that enacts both the host and the client functions of rendering. This enables users to use Lightwave while screamernet is rendering.

4. screamernet maintains 2 lists, which can be edited by the user. one is a client list, a list of all participating clients, and the other is a list of hosts. The hosts list if empty means that the client will take any render commands.

5. a screamernet Host supports 2 ways of distributing the files for rendering. it maintains a "safe area" a locally accessible directory where it copies the files neccessary to render the job, and the directory is assumed to be shared with the clients (the hosts/client lists maintain paths to thses directories). The second file sharing protocol is built upon ftp. at the beginning of the render, the host ftps all neccessary files to each client. each client and host can be configured to do both and prefer one method over another. In this way... clients that need all thefiles locally can have them in the same render that clients access the locally accessible directory.

6. on the mac and Linux platforms... the rendering engine is a command line application. The host/client controller runs as a windowed application. The host/client app handles all of the details neccessary to get the render going, and then issues a command to the render engine.

7. on the mac the host/client app supports applescript... it enables at least 3 commands...
send command (allows applescript to automate hosting duties)
add client (allows remote adding of a client to a host)
add host (allows remote adding of a host to a client)

8. I propose a change to the way Lightwave finds files. instead of having a content directory hierchy, the path to each item is stored in the Scene file... This is the Right way to do it, and it makes handling batch rendering easier.



a typical render would go like this:
a. user saves the file for rendering
b. launches the host/client app
c. loads the scene file
d. hits render
e. host/client app copies the files to its locally accessible directory.
f. host/cient app sends an "are u busy?" message to each client in its list, and becomes a host.
g. the clients that can take part send an "all clear" message
h. the host then negotiates with each client in turn, and either sends files via ftp, or agrees with the client that it can indeed get to the files in the locally accessible directory, if it cannot see all of the the files this is reflected in the interface. and pauses the render so that the user can address this issue, by either fixing the access of the client, or setting it to accept ftps
i. the host then sends a message to All of its clients (even the ones Not participating) that it will begin rendering
j. if at any time during the render the non-participant clients can participate they will send an "i can help now " message
k. the host doles out each frame to a client, keeping track of wos doing what, and how long its taking, optimising it batting order to get the fastest render possible. when each client finishes rendering the frame, it sends the frame back (ftp) or saves it to the locally accessible directory, then it sends a "next" request message to the host.
l. when the render is done, the host sends a "redner done" message to All of it clients. the clients that have been ftp'd files delete the render files.

Matt
02-13-2003, 12:34 PM
This is old work now but I thought it might be interesting for those that haven't seen it already.

How I'd like ScreamerNet to work . . .

Host Setup

http://www.creactive-design.co.uk/misc_images/host.jpg

Node Setup

http://www.creactive-design.co.uk/misc_images/node.jpg

Monitoring . . .

http://www.creactive-design.co.uk/misc_images/monitor.jpg

Cheers
Matt

Lightwolf
02-13-2003, 12:49 PM
Hey, let me chime in here:

1) Network accelerated rendering from within LW on single frames/previews (like Real4D, FinalRender and mentalray).

2) 1) assumes some kind of scanline or bucket rendering algorithm, which should improve load balancing on multiple CPU systems as well.

3) Smart, cacheing screamer nodes that support on demand, lazy loading (i.e. objects and images are only loaded when they are needed in a scene, if the image loading was tile based, we could render out huge texture maps.

4) Standalone Queue client.

5) "Send to Renderfarm" Script for the current scene.

6) Please leave the content directories as they are... Storing paths within scenes is a no no on networked machines, since you have to make sure that al machine use the same drive names, unless you use UNC file names, and in that case the current system is fine enough.

7) Extensive stats and logfiles (for charging clients, also ETA calculations for the scenes, so that you can appease the director/producer....)

8) Image previews of the rendered files from within the client.

9) A screensaver that shows the status of the rendering nodes on the local machine :) (Very neat for renderfarms with an auto-switching kvm switch, one glance and you know that's happening).

10) extension to 8, creation of preview .avi/.qt and e-mail option + copy option for files or animations (would be great to get an e-mail of every rendered scene, as well as automatically copy a finished animation over to the editing system, great for over-night jobs with a tight deadline).

...oh well, I wish :)

Cheers,

Mike