PDA

View Full Version : Anyone got any good advice on storing large projects



Lude
01-29-2008, 03:57 AM
Hi guys I work for a small new media company (website, CD-Roms, Screen displays etc.) and recently we have been creating more animations at 720p res.

So our 3D projects are creating much larges files now. Where as a couple of years ago projects would consist of a few renders now they have thousands of frames.

So what I'm wondering is what method/file types do you use to keep your rendered frames. Do you for example keep your individual frames or create movie files of them before they go into a final edit, if so what files types/codecs do you think are best.

I think this gives the general idea of what i mean.

Lude.

Surrealist.
01-29-2008, 04:17 AM
I usually create tiff/bmp or targa still sequences and then import them into an NLE and convert them there. I Always keep the stills as backup. And I use avi in my NLE but from there I will then process the edited program as an avi master with all the sound and picture effects. I then use this master to create my other media such as DVD or web, quicktime etc.

Andyjaggy
01-29-2008, 12:18 PM
It depends what you are going to do with your stuff. tiff's are fine but if you want to do much comping they won't cut it. I prefer EXR as they are FP and contain lots of yummy buffers.

Space is always an issue but HD's are cheap these days, so stock up :)

Steamthrower
01-29-2008, 12:56 PM
A good cheap way to get hard drive space is to buy a lot of internal hard drives and buy the $25 enclosures that let you convert them to external hard drives.

Hopper
01-29-2008, 05:34 PM
A good cheap way to get hard drive space is to buy a lot of internal hard drives and buy the $25 enclosures that let you convert them to external hard drives.
You can also get Buffalo NAS storage cheap, fast, and reliable. I just bought a 2TB unit for $799.

Buffalo Storage (http://www.buffalotech.com/products/network-storage/)

They allow for standard RAID configurations. I use 0+1 and retrieval times are faster than my local 10,000 RPM drive for large files ( >1GB).

-Hopper

Mr Rid
01-29-2008, 10:18 PM
I keep source sequences on one Buffalo 1TB drive and back them up to another 1TB drive. Simple.

However, I cant recommend Buffalo since I had one of the drives crap out on me only 2 months after purchasing it (something with the kernel). Well within the one year warranty, Buffalo sent me a replacement drive but I was informed that if it were after the 1 year warranty, they will not offer support or attempt repair on their drives even for a fee. So, if the drive fails for any reason after one year, you may be screwed. At about 18 months after purchase, I had an 'ethernet error' with that replaced drive that I have no way of fixing.

Mr Rid
01-29-2008, 10:22 PM
P.S.
I would definitely keep whatever your original source files are and not make movies out of them. I have been burning 2 data copies of everything to DVDs. I have had a backup disc work fine after a burn, but then years later it mysteriously failed.

Hopper
01-31-2008, 06:56 PM
However, I cant recommend Buffalo since I had one of the drives crap out on me only 2 months after purchasing it
Good to know! It's strange to see different experiences with other people's implementations. I used 10 of the 1TB units without fail. As far as I know, they are still up and running fine, but a friend of mine had nothing but problems with his. They replaced it but wouldn't cross-ship. He was in a VERY unfriendly environment though .. very hot, lots of sand, and got lugged around everywhere (i.e. military). - I warned him.

kopperdrake
02-01-2008, 03:58 AM
I currently use a couple of the 500Gb external Freecom drives - no fans - very quiet and stack nicely. Plus they've been fine since I've had them - each is a mirror of the other. When they get full I'll just get a couple more - the beauty of having them in separate units rather than a small multi-disk box is that if you want to create a really secure system you can pull one away into another building/room incase of burglary or fire (or dropping them).

I always keep them online so I can access older projects to reuse textures/models - I thought about the permanent backup route like tapes or what have you but it's been invaluable to keep them accessible. I keep *all* the original files - never archive them or convert to another format...you never know what that client will come back for *tweaks* to a year-old project :D

Surrealist.
02-01-2008, 03:38 PM
Also take a look at an online storage solution to back up your back up. Many offer unlimited space for a monthly fee.

kopperdrake
02-01-2008, 05:05 PM
Mmm...but you need a decent connection for that if you're producing a lot of animation. I'd have no chance in the sticks here :D

Surrealist.
02-01-2008, 06:46 PM
True. I feel your pain.

But for people who have the speed, it is a very viable alternative.

digefxgrp
02-02-2008, 08:30 AM
When we investigated archiving a few years back, the answer was ď use different methods to be safe and backup many timesĒ.

For animation/VFX projects I archive ALL the elements used in creating the finished project at least twice (3 times if itís a long-term project) onto DVD. One set of discs is kept off-site. I also back off everything (often twice) onto separate hard-drives. One set of drives are also kept off-site. With drives prices being as low as they are, itís cheaper to be safe with redundancy than to assume a single drive wonít fail and have to go back and rebuild a project. I was told by everyone I talked to when using HDís make sure you power then up at least twice a year.

Thing is Iíve had both methods fail. Weíve lost all data on a single drive and Iíve experienced corrupt files coming off an archive CD or DVD. If it hadnít been for the duplicate backups we would have been screwed.

Also, when I archive to DVD I donít just copy one disc to the other. Both are made from the source files off the computers. That way if there is a bad file caused by the DVD write, that error is not transferred over to the duplicate disc.

zapper1998
02-02-2008, 08:47 AM
P.S.
I would definitely keep whatever your original source files are and not make movies out of them. I have been burning 2 data copies of everything to DVDs. I have had a backup disc work fine after a burn, but then years later it mysteriously failed.




Found out awhile ago, That DVD, CD, R/W discs after 10 to 15 years
Tend to deterioate.

[deterioate=did i spell that right ?] need to intstall IeSpell again..

Keeping them in a dark storage container with a moisture pack, will
make them last longer..

We had a new company move to Spokane, they manufacture the DVD's and
CD's, They wrote in there article the best storage of the Cd's...

Interesting article also..

What make of HD's are in the Buffalo's???

Michael

Edbittner
02-02-2008, 08:51 AM
A good cheap way to get hard drive space is to buy a lot of internal hard drives and buy the $25 enclosures that let you convert them to external hard drives.

What he said. I've got a 300gig outboard. Not gonna run out of space any time soon.
E.