PDA

View Full Version : FReq: Distributed F9 renders



jeric_synergy
03-09-2015, 02:52 PM
....somebody MUST have already asked for this, but here's some reinforcement....

For those with render farms, it would really help user interactivity if F9 renders could be rendered in slices over the render farm.

Ideally, this would be completely/as-much-as-possible transparent to the user, and all s/he would experience would be a very fast F9.

The feature would distribute the parts to the nodes and assemble them automagically. I guess it would be ok to see chunks come in and placed, as that might reveal enough to abort the render if the user could see it all going sideways.

Anything that increases interactivity speed is Good, of course.

dulo
03-10-2015, 03:58 AM
would be really nice indeed, like in modo or in houdini .. but I think it would be quite a big thing to develop .. quite time consuming for NT

Lewis
03-10-2015, 06:18 AM
Short answer would be : just like THIS - https://www.youtube.com/watch?v=v2TzxJB2abQ

:)

Hopefully they plan to release somethign similar since distributed rendering it's becoming standard in most apps nowdays and SNII is so convoluted , outdated, slow and lacks features :(

Sensei
03-10-2015, 09:01 AM
Short answer would be : just like THIS - https://www.youtube.com/watch?v=v2TzxJB2abQ

Very misleading video.
Especially they used scene with little or none texture...
They should show how it would work with 100+ MB textures with several MB objects, and whole beauty would be gone..

With such distribute rendering time spend on sending data through network will be very slowing down thing.

On LightWave VirtualRender http://www2.trueart.pl/?URIType=Directory&URI=Products/Plug-Ins/VirtualRender
can do render single frame on multiple machines.
On screen-shot of plugin it is working on 35-40 machines on single frame 16000x12000..

Lewis
03-10-2015, 09:41 AM
Sensei, Is there any post you can ake without sheamless selfpromotion, maybe just once ? please, thanks.

There is nothing misleading in that video. There is many videos/situations where TeamRender works great in C4D.
Here is more videos if you really need to see it differently form that short clear video. Ofcourse that renderign over network is nto 100% match like local but at least it's doable/usable and it's very easy solution/render conttoler that works user friendly.

Here is longer video explaining all the advantages/disadvantages/spped info.
https://www.youtube.com/watch?v=B997NoPJ-64

and more
https://www.youtube.com/watch?v=mhrtodaWNtY

Danner
03-10-2015, 09:42 AM
http://www.lightspeedrender.com/
It might be abandonware I haven't heard anything about this lately

Sensei
03-10-2015, 10:13 AM
Sensei, Is there any post you can ake without sheamless selfpromotion, maybe just once ? please, thanks.

No, thanks.



There is nothing misleading in that video.

Stop writing crap.
They're using simple scenes!



There is many videos/situations where TeamRender works great in C4D.

Situations not from real production. Where scene is SMALL, small object size, small texture quantity, low texture resolution or no textures at all. Procedural textures don't have to be send to rendernode, while photorealistic - yes, and sending/receiving takes time.



Here is more videos if you really need to see it differently form that short clear video. Ofcourse that renderign over network is nto 100% match like local but at least it's doable/usable and it's very easy solution/render conttoler that works user friendly.

Here is longer video explaining all the advantages/disadvantages/spped info.
https://www.youtube.com/watch?v=B997NoPJ-64

and more
https://www.youtube.com/watch?v=mhrtodaWNtY

Videos made for purpose of advertising feature to SELL copies of app..

Pure mathematics will disprove it.
If you have 100 Mbps network. At maximum idealized speed one computer can send/receive 12.5 MB/s data (more realistic is 9 MB/s).
Scene has 100 MB size for instance. But "64 bit loving" people I worked with don't have problem sending me scenes 300-400 MB in size.
So 5 computers will need 100*5/12.5 = 40 seconds JUST TO SEND/RECEIVE all data.
More realistic is 100*5/9=55.6 sec.
Then these data has to be LOADED to app. How long takes File > Load Scene alone executed by user inside of app? Same will have to be done by render node. Another delay..

What happened to your brain? In the last video link you gave guy is using PROCEDURAL objects, dynamically created.. So to send such "object" there is few bytes needed to send over network. It's generated on the fly by history/modifier stack.

Lewis
03-10-2015, 10:37 AM
Anddddddd again you are wrong (as usuall) and not listenind or watching vides properly.

Who in right mind is using 100MBit newtwork for network rendering nowdays ? Are you serious ? I'm using 1Gbit for last decade, if you are still on 100Mbit you are stuck in past and you don't need network rendering anyway.
So throughput is 125MB/s not 12.5 MB/s . I use NAS for content with Link Aggregation so i can get 250MB/s output easily and i don't have any noticalbe/lag slowdown even with using Amleto which copyes files linear to each machines Hdd so it's slower than peer2peer protocol.

So your math is obsolete in this case since it's not gonna be 40 seconds in your example but 4 secons or 2 in case of faster network (ofcourse that' sideal solution and rela number can be slightly lower but nothign drastic)

Textures in TeamRender are transfered by peer2peer protocol (similar to bittorrent) so every machine instantly can share parts/textures simultaneously so overhead is very small.

Watch the videos properly with understanding and stop crap...g out about things you obviosuly don't understand.

But evne if all that nonsense you worte would be any true (which is not) it would be still tons easier/faster to use 10 machines over Team Render to render single 5 hour render than 1 machine and do the slicing/saving/loading and setup manually as w eneed to do nowdays with LWSNII. Nobody asked aobu speed of fiel transfer but Ease of USE Network Rendering.

So if you didn't notice this topic/section is aobut about LW feature request and NOT about your 3rd party plugin shameless selfpromotion.

jeric_synergy
03-10-2015, 05:05 PM
My scenes ARE very very simple, but even so AFAICS it's pretty much always going to be faster to have MORE than one machine working on a frame.

It can't be SLOWER than one, right?

lightscape
03-11-2015, 12:15 AM
There's many split render plugins in lightwave but this one looks the coolest but its too bad I couldn't get it to work when previewing lightwave crashes.
http://www.maximcapra.com/?page_id=81

https://vimeo.com/92979418

I pm'd the author last week.

Sensei
03-11-2015, 02:22 AM
My scenes ARE very very simple, but even so AFAICS it's pretty much always going to be faster to have MORE than one machine working on a frame.

It can't be SLOWER than one, right?

Yes, it could be slower.
If transfer data is taking longer, and loading scene is taking longer time than single F9 render.

So simply benchmark with stopwatch things:
- how long your scene is loading in File > Scene > Load Scene. How many seconds?
- how long it takes to send it in RAW format to network neighborhood machines (if it's say 1 MB you will have to duplicate it 100-1000 times (don't use one large file, it'll not be adequate comparison), then divide time by same amount).
- how long typical F9 render takes? Less than 1 minute? Less than 5 minutes?

Make math equation:
x - time spend on F9 render
y - time spend on sending data to 1 machine
z - time spend on loading scene by app
a - quantity of machines used for distributed render
b - time spend on receiving back bucket region (can be pretty small insignificant)

if( x > ( y*a + z + x/a + b ) )
{
// if this equation is true, distributed render will be faster than F9 single render
}
else
{
// it's slower, for this case
}

x/a time spend on rendering by remote machine is true if machines are all similar speed.

jeric_synergy
03-11-2015, 09:45 AM
I assume my master machine can both be sending AND rendering at the same time, no?

So, that would be equivalent to one machine anyway.

papou
03-26-2015, 12:09 PM
i have done that kind of script few years ago: A renferfarm F9.
A script was slicing my scenes into several scenes
... sending them to my renderfarm manager (it was tequila).
... Another dynamics LW script was checking at rendered frames
... and compile slices into an full frame using command line application (it was imagemagick)
... the image was send to a command line image viewer on the user screen (it was Slowview)

Btw, it was only interesting onto long rendering images like MonteCarlo because, waking farms, loading scenes, assembling, etc.. could be longer that a classic F9 rendering time. Lightwave wasn't and is still not a distributed cluster rendering system.
You needs to have as many renderfarm free as you have slices. If not, it become extra rendering times. So the renderfarm must be free for full confort. It was never the case.
Then Fprime appeared. It was fast.

Don't ask for scripts, I lost all final scripts.

I would like Lightwave to be more efficient with renderfarm:
Distributed clustered, sharing memory, radiosity cache sharing, etc...

-