PDA

View Full Version : Does anyone use Amazon Virtual Servers with Lightwave?



robertoortiz
08-31-2016, 08:17 AM
On would think that this is a perfect marriage.
"The Amazon EC2Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developer:s.

Virtual Server Hosting allow for the virtual creation of hundreds of "rendering nodes" at a competitive price. Friends of mine who use Maya swear by this service.
https://aws.amazon.com/ec2/?sc_channel=PS&sc_campaign=acquisition_US&sc_publisher=bing&sc_medium=ec2_b&sc_content=web_server_bmm&sc_detail=+amazon%20+virtual%20+machine&sc_category=ec2&sc_segment=7011578133&sc_matchtype=p&sc_country=US&s_kwcid=AL!4422!10!7011578133!50080014975&ef_id=V3MF1gAAAo1xyEbo:20160831140959:s


SO does anyone use it?

erikals
08-31-2016, 08:31 AM
no, i looked at it way back, but i think i found Garagefarm was better/cheaper ?

jwiede
08-31-2016, 11:50 AM
Yeah, by the time you price it all out, the lower-cost commercial farms are still usually less expensive (or as expensive but without all the setup efforts).

sukardi
08-31-2016, 07:45 PM
I wrote this some time ago

http://rendergeek.com/setting-up-amazon-ec2-render-farm-for-lightwave/

I am still using it - but nowadays with gpu instances for Octane. Quite easy to use once you set it all up.

Some advantages

1. 100% self service - for good or worse. It is like having your own farm.
2. Can be very cheap if you use spot instances. Unfortunately, at the moment GPU instances is not readily available to use as spot instances. If you are using CPU, I tend to use the older compute instances which is about 50% or less cheaper per computing power.
3. You can set up your own plugins and versions
4. It really is unlimited and you can tap them at any time. (They will limit you in the beginning to like 10 instances but you just need to send an email to explain your needs)

Disadvantages
1. You need to pay to store your settings - software installations etc (called AMI). Not too expensive but there is recurring monthly cost.
2. It helps to have fast connection for Dropbox transfer - but I suppose it is the same with render farms
3. You really are on your own setting up and troubleshooting. Unless you pay for the very expensive enterprise support (or something like that), it may take a couple of days for them to answer your ticket.
4. Security issues. I know next to nothing about network security, so I just pray that hackers are not bothered enough to tamper with my running instances.

Those are some of my thoughts

S0nny
09-01-2016, 01:40 AM
IMHO AWS is affordable and much convenient than a renderfarm. For example, Garagefarm is fast in term of raw power, but in the end there is always cue and you'll wait hours even with high priority. As sukardi says, with AWS your instances are always ready at your request, it'a a matter of minutes, just the time to setup the ami and you are good to go, no cue, and custom setup with no problems.

Also, if you use some kind of plugins or a different release, or special setup you can't really use Garagefarm's renderbeamer (which is their fast and self service plugin service).
Just a basic example, if in your scene you use dplights with .ies file, you can't package scene in renderbeamer, and you'll end up uploading a scene without lights. So you need the old ftp way, contacting the guys to explain, make a spreadsheet with all the plugins etc, checking the paths for the lights and everything, hoping it works for every node etc etc.
Then you have to add a special fee because of the non-cloud account, which is something like 10$ or 20$ per job, I don't remember well but it's explained on their website (renderbeamer = cloud account, no fixed fee). With 20$ I can rent an EC2 4.8 large for like 10 hours...
To give you some numbers, a c4.8 xlarge is about 2800 cinebench R15 points.

@sukardi, I'm not into Octane (yet), but interest in the setup for the future. Interesting they have an ec2 with 4gpus (which model I don't know), how do you set it up?

sukardi
09-01-2016, 02:23 AM
@sukardi, I'm not into Octane (yet), but interest in the setup for the future. Interesting they have an ec2 with 4gpus (which model I don't know), how do you set it up?

Basically for Octane, I simply ran it in Lightwave trial version, one instance at a time. You still need a Octane + LW plugin license for each one though.

To run the GPU instances, I took one of Amazon GPU AMIs and modified it. There are some issues that the instance does not show the GPUs if you use to Windows Remote Desktop, so I need to switch to Google Chrome remote desktop to fire up Octane.

Roughly, a G2.8 (they use 4 Nvidia GRID GPUs - not particularly fast) is equivalent to double my GTX780 speed.

S0nny
09-01-2016, 02:50 AM
Basically for Octane, I simply ran it in Lightwave trial version, one instance at a time. You still need a Octane + LW plugin license for each one though.

To run the GPU instances, I took one of Amazon GPU AMIs and modified it. There are some issues that the instance does not show the GPUs if you use to Windows Remote Desktop, so I need to switch to Google Chrome remote desktop to fire up Octane.

Roughly, a G2.8 (they use 4 Nvidia GRID GPUs - not particularly fast) is equivalent to double my GTX780 speed.

Thanks, I'm wating for some news before investing in Octane (or another gpu render), but it's good to know it's possible. I was hoping for a much powerful instance though :P

samurai_x
09-01-2016, 03:03 AM
Garagefarm has a discount coupon.

Soth
09-03-2016, 06:24 PM
Hi guys!

Looks like you have very interesting discussion here.


IMHO AWS is affordable and much convenient than a renderfarm. For example, Garagefarm is fast in term of raw power, but in the end there is always cue and you'll wait hours even with high priority.
As sukardi says, with AWS your instances are always ready at your request, it'a a matter of minutes, just the time to setup the ami and you are good to go, no cue, and custom setup with no problems.
Well not always, but we are aware of the problem and hopefully we will fix it once for good later this year. I will definitely announce it on the forums here.


Also, if you use some kind of plugins or a different release, or special setup you can't really use Garagefarm's renderbeamer (which is their fast and self service plugin service).
Just a basic example, if in your scene you use dplights with .ies file, you can't package scene in renderbeamer, and you'll end up uploading a scene without lights. So you need the old ftp way, contacting the guys to explain, make a spreadsheet with all the plugins etc, checking the paths for the lights and everything, hoping it works for every node etc etc.
I wasn't aware of that, but as the LightWave is my favorite 3D program I would really like to keep our plugin feature wise as close to perfection as possible I will look into it this week. Doesn't look like lot of effort to add support for ies files in dpoint plugin.
Regarding different builds, renderBeamer support most (all recent) of them already.


@sukardi, I'm not into Octane (yet), but interest in the setup for the future. Interesting they have an ec2 with 4gpus (which model I don't know), how do you set it up?

Thanks, I'm wating for some news before investing in Octane (or another gpu render), but it's good to know it's possible. I was hoping for a much powerful instance though :P
We do have GPU server for renting, much more powerful than Amazon one (8 Titan cards).
http://garagefarm.net/en/8-gpu-superserver-benchmark

S0nny
09-06-2016, 01:37 AM
Well not always, but we are aware of the problem and hopefully we will fix it once for good later this year. I will definitely announce it on the forums here.

I'm reading my post now and I see I wrote 'cue' instead of 'queue' 8~. Anyway, I guess it's the downside of having low prices, the more the people using the service, the more the queue.



I wasn't aware of that, but as the LightWave is my favorite 3D program I would really like to keep our plugin feature wise as close to perfection as possible I will look into it this week. Doesn't look like lot of effort to add support for ies files in dpoint plugin.
Regarding different builds, renderBeamer support most (all recent) of them already.

It's good to hear, I talked about this with one of your guys back in march this year in the user chat, because I noticed renderbeamer works similar as LW package scene, so it isn't aware of dp web .ies lights because are managed by the plugin and not by LW itself. But I suppose the report gone missing for some reason.



We do have GPU server for renting, much more powerful than Amazon one (8 Titan cards).
http://garagefarm.net/en/8-gpu-superserver-benchmark

I didn't know that, if I ever switch to gpu render that'll be one very good solution for sure.

m.d.
09-08-2016, 12:42 PM
amazon uses the grid k340 GPU's for it's boxes....

the option with 4 gpu's is about $2.87 an hour....but likely would only be running 160 on octane bench

I think this is identical boxes as octane render cloud...as they have stated each instance would run 40 on octane bench, but they give you 20 for a bench score of 800....for about $10 an hour. So with ORC you are getting 5x amazon boxes for less than the price of 4.

Garagefarms system should also be around 750-800 in octane bench (care to run the test soth?) but goes for $7.50 an hour....but you have to install lightwave and plugins before rendering, but the first hour is free.

So as of right now garagefarm is the cheapest...but they only have the 1 machine, and you need a bit of time to setup.....still would cost $180 a day for what would take 2x970's in 4.5 days to render

ORC will be more flexible, and have the capability of much more power, for about 20% more than gargefarm....and amazon would be almost double garage farm.

A single box amazon deployment wouldn't be worth it for GPU rendering IMHO...if the garage farm machine is free I would jump on that

sukardi
09-08-2016, 05:56 PM
amazon uses the grid k340 GPU's for it's boxes....

the option with 4 gpu's is about $2.87 an hour....but likely would only be running 160 on octane bench

I think this is identical boxes as octane render cloud...as they have stated each instance would run 40 on octane bench, but they give you 20 for a bench score of 800....for about $10 an hour. So with ORC you are getting 5x amazon boxes for less than the price of 4.

Garagefarms system should also be around 750-800 in octane bench (care to run the test soth?) but goes for $7.50 an hour....but you have to install lightwave and plugins before rendering, but the first hour is free.

So as of right now garagefarm is the cheapest...but they only have the 1 machine, and you need a bit of time to setup.....still would cost $180 a day for what would take 2x970's in 4.5 days to render

ORC will be more flexible, and have the capability of much more power, for about 20% more than gargefarm....and amazon would be almost double garage farm.

A single box amazon deployment wouldn't be worth it for GPU rendering IMHO...if the garage farm machine is free I would jump on that

Thanks m.d. for the perspective.

However I don't think Octane Cloud is available at the moment. They probably would be my first choice.

With Amazon instances, I used to get the G2.8 (160 octane bench) for less than $1 using spot instances. Nowadays however, they are pretty busy. So there is no choice but to pay the full price at $2.87

GarageFarm, definitely will look them up when I have specific rendering needs.

However, for me the whole point using Octane is to have a smoother workflow. For instance, at the end of a long day, I know for sure that I need not more than 20 minutes to set up the render on Amazon Cloud.

That is just how I set up my workflow ...

m.d.
09-08-2016, 06:31 PM
never heard anything about spot instances....sounds like a good deal though.

ORC is in beta and people are using it....just not public yet. I looked into the price/performance of all of them and thought I would let people know.....
$1 for a spot instance is the best bang for the buck.....

sukardi
09-08-2016, 06:46 PM
Spot instances are the ones with dynamic pricing. You set a max price and let the system set the price you pay based on supply and demand.

Problem is, if the current market price rises above your max price, your instance will be kicked out without warning. Very good if you know for sure, that a particular instance type is in low demand...

S0nny
09-12-2016, 02:44 AM
A side node: garagefarm has updated the lw plugin for dp lights/ies lights I was talking about

Lightwave plugin - v27 09/09/2016

New version has been released!
- Added support for IES lights assest (Lightwave internal & DPKits)
- renderBeamer is staring automatically after installation now

That was fast, good job!

sukardi
10-04-2016, 01:24 AM
134693

Update: Amazon just added new P2 instances with single, 8 and 16(!)Tesla K80 GPUs.

The above uses the the 8 GPUs - 23 seconds for 960x540, 500 samples path tracing. I have no idea why it only shows 7 GPU instead of 8 as I have turned on all of them.

Exciting time for Octane ...

m.d.
11-30-2016, 03:48 PM
just found this update, as I have a 450hr+ render (on my machine) upcoming.....

the 8 K80 translates to about 584 octane bench, and they go for just over $7 US an hour...

Garage farm should still beat it price and performance wise....except the 16 K80 will be about 35% faster for about double the price