PDA

View Full Version : When to switch to Server Blades?



AbnRanger
12-26-2005, 09:01 PM
Anyone know from practical experience when it becomes feasible (cost effective) for a small studio to switch from a small number of machines as render nodes to server blades?
I recently met a small studio owner, who asked me about building some machines for a bigger network than the few workstations he currently had.
When he mentioned that he pondered using server blades, I told him that I didn't think it would be cost effective to even think about them at that level...and that it was more of a matter of available space, than performance. Blades are above my head, so I'd appreciate your input. Thanks

Speedmonk42
12-27-2005, 11:21 AM
Anyone know from practical experience when it becomes feasible (cost effective) for a small studio to switch from a small number of machines as render nodes to server blades?
I recently met a small studio owner, who asked me about building some machines for a bigger network than the few workstations he currently had.
When he mentioned that he pondered using server blades, I told him that I didn't think it would be cost effective to even think about them at that level...and that it was more of a matter of available space, than performance. Blades are above my head, so I'd appreciate your input. Thanks

Hmmmmm. Blades tend to have really expensive back plane cases you need to put them into. I would rather have a problem with a single 1U server than the single thing that feeds a whole bunch of them.

Are you so tight for space that 1U rackmounts will not be enough?

Lightwolf
12-27-2005, 11:52 AM
While we're still using 4U rackmounts, this might be an option:
http://www.atxblade.com/
They basically mount standard AXT boards vertically.

Cheers,
Mike

Draven
12-28-2005, 11:08 AM
Still too expensive.

at $1500+ a board, you can buy more capable 1U units for less

DonAVP
12-28-2005, 02:51 PM
Have you or he considered outside rendering? We (really small two man office) used ResPower in Huntsville, Georgia. They have 600+ CPU's and LW render time is a good price.

The interface is a little dated but they produce results in a short time minutes-hours verses days-weeks. They were very helpful and when I screwed up they gave me credit.

Don :thumbsup:

Draven
12-28-2005, 03:29 PM
I use Respower myself. they are a good short-term solution, but you can't count on always being able to access them, or a particular rendering service always being around. You'll know when you hit the point that you need to be doing all of your own rendering- the point will hit about the time you need to be downloading more than overnight.

Also, you can't render composites on many 3d rendering services, or if you can you spend ALOT of time uploading your footage- often, you'll upload three times as much footage as you download, at a minimum. Another reason to have a few render nodes.

And remember, we're at a cusp right now. Soon, even shots you are completing for local TV will need to be completed in at least 720p HD.

DonAVP
12-28-2005, 04:01 PM
Draven, noted you are a little ahead of us in that regard. Our work is not in the entertainemt field directly. We build and service 3D visualatoin simulators, also are doing equipment training CD/DVD's for defense contractors. So I got a foot in realtime gaming and canned video. Good Luck!

Don

Stooch
12-28-2005, 06:47 PM
http://www.boxxtech.com/applications/rendering_systems.asp

i find these to be very reasonably priced, thats what our company is getting.

Speedmonk42
12-29-2005, 12:59 AM
I was poking around and found these. This is the same form factor as the Shuttles, only it is not so standardized. There are some that very small pancakes a foot square, but only 2 inches high.

You could make a mountain of these and point an air conditioner at them.

Speedmonk42
12-29-2005, 01:00 AM
I was poking around and found these. This is the same form factor as the Shuttles, only it is not so standardized. There are some that very small pancakes a foot square, but only 2 inches high.

You could make a mountain of these and point an air conditioner at them.

Ooops

http://shop.store.yahoo.com/directron/miniitx.html

These prices seem a little to high though, I bet you can get them cheaper.

AbnRanger
12-29-2005, 05:21 AM
Thanks for all the tips. I looked at the pricing on 4U rack mounts, down to a single node with an AMD 64x2 Dual-Core 4400 (what I have on my system) with Micro-ATX MB's (so they will fit into very small cases). The latter was easily the most cost effective...by FAR (roughly $1100 w/ 4GB RAM, MSI MB onboard Nvidia Video, 160GB SATA II HD, Zalman CPU cooler, Aspire Q-Pack Case).

In order to double that capacity, it takes over $4K in a Dual Opteron 280 station in a low profile case. A comparable BOXX system is over $6300. Whoooa :eek:
We haven't even touched blades (talking about "Cutting Edge" technology) yet...and I don't think I need to. Not trying to be miserly, really....but giving up only a few feet of extra space, one can purchase 2 or 3 times the processing capability by going with compact individual nodes using dual-core CPU's.
Respower sounds like a good option when you're in a real pinch.

habaņero
12-29-2005, 10:06 AM
I'd go along with AbnRanger here, buying enthusiast favorites like the mATX Biostar Tforce 6100-939 with dual core 939 opterons, or the "dual sata" Asrock ATX board that will let you upgrade to M2 socket. These boards are so tested and tweaked at that you can feel quite confident about fixing any issues yourself off google, and they are like 75$. You can also use the dark force and overclock, which in my opinon makes perfect sense on parts that have sufficient quality for it. About +33%-50% is actually realistic with little penalty towards stability. Something like 700 Quid, some more Euros with Opteron 165 w/ the stock cooling, 2 gB ddr550, 40 gB Samsung sataII and cheapo but favorably reviewed 300W (m)ATX case and the 6100 onboard graphics.

The stock cooling you can get on those is entirely sufficient, and given a reasonable quality a 300W psu that is sold with mATX cases is usable if you don't intend to put high end graphics on it. There is a considerable penalty in time building and adjusting, I would recommend this type of solution mainly to people that can do most of their support themselves, small studio or freelance maths so to say. I'd build myself, but probably if you are not a nerd I'd have the shop put it together and test it. Power/$ even w/o overclock is incredible though, particularly with the 165 though the 170 might be easier to locate nowadays ...

Speedmonk42
12-29-2005, 09:52 PM
Why do you need a 160GB SATA drive? Why not buy a lot of 10GB hard drives on ebay for $50 ?

Is it not primarly the CPU speed with enough RAM that makes the biggest difference in rendering? SATA vs IDE probably makes no difference or fancy chipsets.

I wonder if a fancy motherboard makes any difference to rendering, unless it has to swap out becuase the scene is big. If you have scenes big enough that require lots of swapping then you definately should NOT be buying anything right now that is not 64bit with lots of room for RAM expansion because when 64bit stabilizes with plug-ins ect it will be your biggest speed boost to add more RAM than what any expensive HD and MOBO combination will give you now.

Not sure if that made any sense.

What about calling Dell and saying you want their lowest possible configured P4 with the smallest hard drive possible. On a lease and buying a bunch of them you might get it for something like 10-12 dollars a month. So for 500 dollars a month you could have like 50 P4's running.

How do unfancy P4's compare on a dollar per render time to fancy boxes we use to do our work?

Stooch
12-29-2005, 11:24 PM
well i priced out boxx units to about 3k a piece with a really nice xeon and 3 gigs of ram and alot of hdd storage space (we need some backup storage)

dont know where you are getting the 6k figure...

are you going with top of the line AMD boxes or intel? seems from the research i did, the xeons still appear to be the better cpu for LW (they are actually quite a bit cheaper then the comparable AMD)

AbnRanger
12-29-2005, 11:38 PM
Yeah, you're right about the HD size. I was thinking the same thing initially...but when I saw the HD's at Newegg, where a 160 GB model was priced about the same as many of the 80GB models (roughly $80), I thought it would make better sense to grab that extra storage space...seeing that it will come in very handy when working with a lot of video files. By giving each node a separate partition for storage only, you can knock out 2 birds with one stone (using your network for both rendering and storage capacity). Plus, with the HD being the slowest element in the system...doubling the transfer speed is no small matter. The HD transfer rate is doubled in SATA II's and the cost wasn't much different.

There's nothing really fancy about the components I mentioned...a new 64 bit motherboard w/ onboard video (in case you want to use a physical monitor for diagnostic/maintenance reasons) for $75. A very compact case w/ 420W power supply and slide out MB tray for about the same price...ditto for the HD. Each stick of 1GB of RAM is about $75 each. An AMD 64x2 4400 is mid-priced, but with some moderate overclocking, will perform like the top model (4800....both have 2MB L2 Cache).
Right now, P4 dual-cores can be had cheaper, but then there's the performance/power usage/heat issues. AMD, at present, is all around a better product

I guess it wouldn't hurt to call Dell first and see what they have to offer within that price range/unit. It's been my consistent experience that building your own units will generally save one mucho denaro.

AbnRanger
12-29-2005, 11:52 PM
well i priced out boxx units to about 3k a piece with a really nice xeon and 3 gigs of ram and alot of hdd storage space (we need some backup storage)

dont know where you are getting the 6k figure...

are you going with top of the line AMD boxes or intel? seems from the research i did, the xeons still appear to be the better cpu for LW (they are actually quite a bit cheaper then the comparable AMD)I ran the configuration for the specific components. But that included Dual-Dore/Dual Opterons 280 (top model) CPU with 8GB RAM.
I know that you can choose cheaper model CPU's, but the point was to estimate what it would cost to buy a pre-built model from a retailer like BOXX, compared to building it myself WITH THE SAME COMPONENTS (or better). About $2300 difference!

It's a common fact that if you want Premium Performance, you will generally pay a PREMIUM price. With the advent of Dual-Core desktop CPU's, there is a means to circumvent that issue to a sizable degree.

Nevertheless, that is quite a huge difference between what I came up with and what you found.

habaņero
12-30-2005, 06:42 AM
New sataII hard driver have fluid bearings and easier installation, as well as NCQ that does affect performance. (the 150 mb/s from sataI is mostly overkill already for single drive purposes). They run more quiet, cooler and are likely to last a bit longer than the old ones, though some of those seem intent on outlasting their inventors. Boot time for the farm is also a slight consideration. At 50 quid for the new, I would not consider old HW for this.

With a real hard drive you can use the boxes with effects/video packages, say while rendering with fprime on one. 6100 graphics, you can work fine on that one remotely with ultravnc, they have 1 GB networking and if you want a workstation you just put in a graphics card.

And finally memory speed I'd say is a significant factor as well as the amounts. Most server boards will only run ddr400, often only ddr333 or 266 with large amounts of it. While it is not nearly as important as the CPU speed, to run ddr550 is a significant difference. (It is though often not feasible with 4 gigs of memory today.) Also, infinimap means that you are not limited by memory to render large textures, which is the typical culprit of memory options.

I doubt either a Xeon system or the cheapest dells could compete with 700$/ 2 opteron cores at 2.2-2.4 ghz, maybe 2.6-2.8. The Xeons/P4s also swallow a lot more electricity and give off lots of heat for the same piece of math. Not religious about this, I just think that is the facts.

Draven
12-30-2005, 07:30 AM
hard drive speed or space isn't that critical for render nodes. thats what servers are for. All render nodes need is enough space for the OS and installed sofftware.

habaņero
12-30-2005, 07:39 AM
That is correct. The arguments for buying the new sataII drives is that fluid bearnings have better longevity/stability, and that you get a slight speed premium when installing, booting or if you are going to use the node for other stuff, to have that as an option. Buying used drives; no guarantees, and the risk to balance towards the few dollar gain is against hard drive crash. The new Ide drives are only marginally cheaper, and also can sometimes be a problem if you are going to trim the engine of the computer (sata can too though...).

In not such a long time, I guess memory sticks/drives will be the standard though.

nerdyguy227
12-30-2005, 10:34 AM
New sataII hard driver have fluid bearings and easier installation, as well as NCQ that does affect performance...

very true but 160 gb is a bit much though I dont know what you will have to store

AMD deffinatly is the better value at the time, mostly because of their on-die memory controler (faster speeds between proc. and RAM so lower latencies, and still very cool)

What about dual channel RAM? its supposed to be way faster and all and not cost too much more. Higher preformance ram is also a good investment since it allows for fast transfers of massive rendering information

something like this (http://www.newegg.com/Product/Product.asp?Item=N82E16820141236) if you get 2 then you'd have 4 gb, it is a bit pricy but very very fast and would absolutly be a good combo with the AMD X2

AbnRanger
12-30-2005, 01:01 PM
I initially considered a minimal-sized HD was the way to go, but when I saw how cheap one of that size was (plus, being SATA II to boot)...I thought it might make sense to also use the render nodes as extra storage space for video files when the need arises.

Draven
12-30-2005, 09:33 PM
But thern the files need to be shuffled back and forth to the server, creating extra network traffic.

nerdyguy227
12-31-2005, 04:46 PM
heres my suggestion:

get very low space but superfast 30-40 GB hds for the nodes then using that supposedly good deal on thoes 160 gb SATA drives, buy a few storage servers and set them up in a RAID configuration

and for the nodes consider dual channel and preformance ram- you would get an extra bost especially with the X2

Stooch
01-01-2006, 11:30 PM
Yeah, you're right about the HD size. I was thinking the same thing initially...but when I saw the HD's at Newegg, where a 160 GB model was priced about the same as many of the 80GB models (roughly $80), I thought it would make better sense to grab that extra storage space...seeing that it will come in very handy when working with a lot of video files. By giving each node a separate partition for storage only, you can knock out 2 birds with one stone (using your network for both rendering and storage capacity). Plus, with the HD being the slowest element in the system...doubling the transfer speed is no small matter. The HD transfer rate is doubled in SATA II's and the cost wasn't much different.

There's nothing really fancy about the components I mentioned...a new 64 bit motherboard w/ onboard video (in case you want to use a physical monitor for diagnostic/maintenance reasons) for $75. A very compact case w/ 420W power supply and slide out MB tray for about the same price...ditto for the HD. Each stick of 1GB of RAM is about $75 each. An AMD 64x2 4400 is mid-priced, but with some moderate overclocking, will perform like the top model (4800....both have 2MB L2 Cache).
Right now, P4 dual-cores can be had cheaper, but then there's the performance/power usage/heat issues. AMD, at present, is all around a better product

I guess it wouldn't hurt to call Dell first and see what they have to offer within that price range/unit. It's been my consistent experience that building your own units will generally save one mucho denaro.

well the boxxes i priced out had a 80gig primary and a 250 gig secondary! multiply that 8 times and you have quite a bit of spare storage. then back it up redundantly (2 copies of everything) and you got a reasonaly secure backup setup, not long term but certainly useful. Also, its not wise to get the best of the best, if you research your benchmarks you will find that the increase in price for "top shelf" speed is not justified by the performance boost! so price out something that gets nice and close to the best but not quite the best and use the savings to get more boxes, get single cores for now and later upgrade to dualies as they drop in price, etc.