PDA

View Full Version : high end production pipelines



adboy
12-18-2009, 09:12 AM
just trying to get a handle on how people work, can anyone who is working in tv or film in any application please give an insight to how they work so we can see how lightwave would fit into that kinda pipeline

from file formats if 2d and 3d to how the composits are put together

thanks

adboy

CC Rider
12-18-2009, 10:09 AM
You may want to be a little more specific with your question to get a response..."how people work" is a little broad...

adboy
12-18-2009, 10:44 AM
i left it open to get a few different types of reply

i start specifying and anyone that doesnt fit that bracket might not want to post

anyway i guess i'll ge tthe ball rolling

where i work its broken into some very specific tasks for big jobs, bespoke effects my have a single artis but for the most part its teams

modellers
using whatever they see fit to make a model as long as the uv's can be exported and are clean, (they are normally run throu uv layout after make them beautifull)

textures are then done in photoshop, normally floating point (but not always) so that when they are applied they dont blow out in the renders


matchmovers
doing the hard stuff and getting assest to match the plates

lighting
we have a lookdev team to make up loader scenes with hdr and some main lights set up so that al scene it the same area get the same treatment, its then down to the lighter to tweak, they then output passes, including beauty, various occlusions, individual light contribution passes (for relighting) all that sort of thing, the beauty is normall jsut used as a guide

then onto the effects guys to add some magic

who also render out various passes to give the compositors as much control as possible

this is all pulled together in comp to make a pretty picture

good times

JonW
12-18-2009, 01:11 PM
The best way is to allow the people who are actually doing the work to change things. They know what is going on & know where all the inefficiencies are.

Keep the managers out of the loop as they will stuff things up, (I have worked 16 years for one of our largest manufactures, I know about politics!), & if something is changed & improved they will take all the glory & insist on a larger pay cheque.

If it fails they will still insist on a larger pay cheque, & get a promotion!

djlithium
12-19-2009, 06:31 PM
Well you really missed out on the Webinar today at www.kurvstudios.com where I covered a lot of that in my lighting, rendering and break out then composite class using the BSG assets and techniques. The vids will be up on Kurv shortly for paid download and I hope to pack up the renders from today and the fusion flow so people can pull it apart. Those will hopefully be included in the content for the vids, so stop by there and check it out.

The first free class that I did should be on line there sometime today or tomorrow.

adboy
12-20-2009, 10:47 AM
thanks, i'll bare that in mind:)

adboy
12-21-2009, 03:04 AM
hmm

so no one's got a highend lightwave based pipeline they'd like to share?

adboy
12-23-2009, 05:12 AM
ok pretty much what i expected

ah well

Sensei
12-23-2009, 07:28 AM
That's forum for 3d graphics artists, but your question is for project managers, who don't even have to know 3d, just how to handle people..
Why would project manager waste time and read 3d forum and how to make some 3d effects, I don't know. That's job for lowest level workers.

adboy
12-23-2009, 08:14 AM
i work in 3d everyday and i'm not a project manager but i have a pipeline for getting work done

as do most td's that are worth anything or they at least know what the pipeline is even if they dont have to do every part themselves

i dont use lightwave at work , just wanted to see if anyone has found a way to use lightwave in a high end production and what the pipleline issues are

Cageman
12-23-2009, 09:07 AM
Highend LW-based pipeline:

Rendering: Janus, exrTrader and for LARGE texture maps; infiniMap Pro. If you need a good previewer, add FPrime (FPrime do final renders as well).

Animation: A mix between LW and Maya is a good start.

Data-management between LW and Maya: Add PointOven to your toolset, since it is a good, simple way of transfering vertex caches between Maya and LW. PointOven also serves as a very good MDD baker enmasse within LW itself, and there are no limitations to the number of polygons you have, since PO bakes MDDs directly to disk. Also make sure you take a look at FBX and figure out what it does/doesn't transfer well between Maya/LW. I personaly use a mix of FBX and PointOven, depending on the type of data that needs to be transfered.

Cageman
12-23-2009, 09:11 AM
That's forum for 3d graphics artists, but your question is for project managers, who don't even have to know 3d, just how to handle people..
Why would project manager waste time and read 3d forum and how to make some 3d effects, I don't know. That's job for lowest level workers.

A projectmanager that doesn't know what different tools does, is NOT the guy you want to design a production pipeline.

:)

Sensei
12-23-2009, 09:15 AM
as do most td's that are worth anything or they at least know what the pipeline is even if they dont have to do every part themselves

Can it be different than model, texture, lighting, rendering and compositing? ;)



i dont use lightwave at work , just wanted to see if anyone has found a way to use lightwave in a high end production and what the pipleline issues are

LightWave has fast, very good quality built-in renderer. And rendering nodes are free, so you just have to buy new machines and set up, without cost for rendering node license like in other packages/renderers. Often people make whole scene in other application, then convert to LW and render.

But it doesn't have instances, so you have to buy HD_Instance, if you need more copies of object without memory issues. Usual use- grass, tree and other vegetations in exteriors.

People that composite a lot also use exrTrader.

People that need extra large images use infiniMap.

People that model a lot use LWCAD (mostly archiviz people) and TrueArt's Modeling Pack (mostly making organic objects with EasySplit, EasyMesh and SwiftEdgeLoop tools, and airplanes, cars etc spline models with EasySpline).

For setting up lights and textures with instant feedback people use Fprime or mine real-time interactive renderer.

For easier setting up renderfarm people use ButterflyNetRender.

Captain Obvious
12-23-2009, 01:19 PM
At Cityscape, it sort of goes like this:

AutoCAD and Rhino for handling the CAD side of things.

modo and Modeler for cleaning up and prepping CAD exports and all additional modelling and UV mapping.

Photoshop for texturing work. Arroway are ace for textures, by the way. I highly recommend them.

Scenes usually set up in Lightwave. Rendering in either Maxwell, FPrime, Lightwave or Kray, depending on what's needed for the project. Kray is really useful, because it adds a fair bit of missing functionality to Lightwave (instancing, easy linear workflow, proper layered output and incredibly fast GI and soft reflections). It's hard to argue with Maxwell's results, but render times are insane.

ButterflyNetRender for network rendering. Works well enough, and supports LW, Kray, FPrime and modo. Not Maxwell, unfortunately.

Photoshop for comping stills, AE or Shake for animations.

walfridson
12-23-2009, 02:29 PM
Not Maxwell, unfortunately.
Not possible to set it up as custom renderer, like you do with softimage?

JonW
12-23-2009, 03:08 PM
I do architectural stuff.

So, Photoshop for images. Illustrator for drawings (I given up on CAD drawings mostly because the standard of drafting is so appalling these days & the disadvantages outweigh everything else)

Then back to Photoshop. Or Screamernet for frames. I try not to over complicate things.

What I am surprised about is the amount of rubbish a lot of the top end architectural renders are producing with tons of expensive software. It really boils down to the fact that a skilled professional will produce a good product regardless of the available tools.

It’s more about efficiently managing the assets you have.

Captain Obvious
12-23-2009, 04:22 PM
Not possible to set it up as custom renderer, like you do with softimage?
I don't think so, but I might look into it when I'm back in january.



What I am surprised about is the amount of rubbish a lot of the top end architectural renders are producing with tons of expensive software.
Oh, such as?

JonW
12-23-2009, 08:02 PM
I won’t name people. But there are a number of large architectural firms which I have done models for & they want to stick with their current 3d supplier. There have also been a number of fly throughs which have been on the news, & you just have shake your head. This stuff was done a decade ago.

Captain Obvious
12-23-2009, 09:09 PM
I figured you meant people like Hayes Davidson or, well, us at Cityscape. :P

adboy
12-24-2009, 03:24 AM
ah this is the kinda thing i wanted to hear:)

so kray lets u do real renderpasses? because for me this is where lightwave falls down horribly, getting passes out as secondaries rather that haveing to set up additional scenes is a must

all the power has to go to comp so the better detailed the passes the better

captain obvious i actually used to work for some of your competition many moons ago so i'm no stranger to the arhitectural game, will check out that renderer

:)

Cageman
12-24-2009, 03:29 AM
ah this is the kinda thing i wanted to hear:)

so kray lets u do real renderpasses? because for me this is where lightwave falls down horribly, getting passes out as secondaries rather that haveing to set up additional scenes is a must

all the power has to go to comp so the better detailed the passes the better

captain obvious i actually used to work for some of your competition many moons ago so i'm no stranger to the arhitectural game, will check out that renderer

:)

Don't know if Kray does passes, but Janus does! I recently used it in production where I had 41 passes within a single Lightwave scenefile. Each of these passes could also be rendered out to multichannel exr-files to hold the buffers LW can output. In this case though, we only needed the beauty, but still having everything separate from eachother. Janus also allowed me to quickly set things up so that each pass was masked out by all other elements, so it didn't matter in which order things were put together in comp.

:)

EDIT: Link to Janus.... (http://janus.faulknermano.com/)

adboy
12-24-2009, 03:35 AM
Sensei

i do use lightwave , have doens since 5.5, i own hd instance and fprime, used it in full production for tv shows, i'm just looking to see other peoples pipelines to see if they have found ways around lightwave shortcomings

cageman
cool i'll look that up too then, thanks :)

i just hope this is the kinda thing they are gonna build into this new core thing

does anyone have a way of getting light contribution passes out of lightwave without making seperate scenes and jsut turning things off?

Cageman
12-24-2009, 03:45 AM
does anyone have a way of getting light contribution passes out of lightwave without making seperate scenes and jsut turning things off?

Janus can create LightPasses for each renderpass. Lets say you have 3 lights in the scene, and you want to be able to relight in comp, you make sure you assign each light into a separate group (in this case, 3 groups). You then select your renderpass and assign those three lightgroups to the renderpass and enable a checkbox called LightPasses. Janus will then render light 1 into the Red channel, Light2 into the Green channel and Light3 into the blue channel. The final image will look weird though, and LightPasses are really only usefull if you have a compositing application where you can make use of this.

As for Global Illumination (as in having a GI-pass only), there is currently no easy solution to this if you are using Monte-Carlo or Final Gather. If you are using Background Radiosity, you can make overrides in Janus to create such a pass.

However, I think there is something going on on this front in LWHC regarding GI. ;)

Cageman
12-24-2009, 03:49 AM
I usually fake GI with either DP DomeLight or the standard DomeLight in LW. Sometimes I've made use of BG-GI... bottom line is that cheating that way allows me to use Janus to create the lightpasses I need. Pretty damn straight forward and looks nice enough.

:)

adboy
12-24-2009, 03:59 AM
i think u just sold janus to me cos that the exact way i set my passes now

does it do this as seperate passes so increased rendertime or does it just extract the info from lightwaves render buffers?

so basically is it a secondary or an actuall pass?

Cageman
12-24-2009, 04:08 AM
does anyone have a way of getting light contribution passes out of lightwave without making seperate scenes and jsut turning things off?

Btw... Janus works this way, but instead of you having to do it manually, Janus takes care of all the tedious work. So, you have a single scenefile where you work, and through Janus you create your passes. When you are ready to render those out, you select the Passes and break them out. Janus will then generate a scenefile for each pass that you put on a farm or batchrender.

If you need to change something in one of those passes, you simply do that in your main scene, and re-export the renderpass. Janus has a very good and tidy naming convetion, so you really don't have to worry about that either.

We did a project at work some time ago where we used LW for rendering, and at that time, I couldn't get Janus to work at my workstation. Since the demands for 100% control in comp was there, I had to manually setup two shots with LOTS of content. It took me two days to set everything up so that it rendered correctly with multichannel exrs etc.

When I got Janus to work at my workstation, I went back to test Janus with those two shots. It took less then two hours to set those two scenes up from scratch. The really cool thing with Janus is that you can import Groups and Renderpasses from previous setups. In this case, it was very beneficial, since the two shots had pretty much the same content, just a different camera angle and some additional stuff. I could load the settings I did in shot1, and then just assign the items into the already existing groups in shot2 and do some renaming on the renderpasses. Done!

:)

Cageman
12-24-2009, 04:12 AM
i think u just sold janus to me cos that the exact way i set my passes now

does it do this as seperate passes so increased rendertime or does it just extract the info from lightwaves render buffers?

so basically is it a secondary or an actuall pass?

Hmm... I may have to take a second look at what exactly is happening. I will record a quick and dirty tutorial about it so that you can see how it works...

:)

Stay tuned. :)

adboy
12-24-2009, 04:21 AM
thanks :)

Sensei
12-24-2009, 07:31 AM
i think u just sold janus to me cos that the exact way i set my passes now

does it do this as seperate passes so increased rendertime or does it just extract the info from lightwaves render buffers?

so basically is it a secondary or an actuall pass?

Pass plug-ins (there are three: Janus, LW Passporte and Surpass) are all using the same technique- generate separate LWS for each pass, that is later rendered usual way F9/F10/ScreamerNet. They are LScripts. They are not 3rd party renderers that could do everything in one go.

adboy
12-24-2009, 07:41 AM
yea i actually work with jeremy

i just hope in core all the buffers are customisable and availible as they are in renderman, that kinda functionality would give lightwave a chance in film work, its a real shame too cos the quality of the renderer is amazing and fast, jsut needs that last 10%

Cageman
12-24-2009, 08:16 AM
Hi adboy!

Go to this thread: http://www.newtek.com/forums/showthread.php?p=965352#post965352

Start from the bottom and go back in time to view my tutorials, since the introduction to Janus is quite slow-paced and not at all from a "speedy workflow" point of view. I just added my LightPasses videotutorial in that thread (last post). Let me know if you have any problems downloading from the FTP.

Cheers and Merry Christmas!

:)

Cageman
12-24-2009, 09:07 AM
I've attached a screenshot from a very recent project at work where we utilized Janus. I had to add black bars everywhere in this screenshot because what we are working on is very secret at the moment. :)

As you can see, there are 41 groups and 55 renderlayers within Janus. To say the least, this particular project is very compositing heavy and the fast workflow with Janus made it a breeze to setup. Some of those renderlayers have either surface overrides or object replacements, and one of them is actually using a different camera. As you may understand, this scene, if setup with the oldschool LW way of doing things, would have taken ages... and even worse would be if you need to change something... PAIN!

No pain with Janus.

:)

Sensei
12-24-2009, 10:03 AM
No pain, no gain.. ;)

Captain Obvious
12-24-2009, 02:16 PM
Kray can output things like reflections, refractions, direct and indirect lighting, diffuse texturing and stuff like that all from one render. It uses a bit more memory, but the render time is unaffected.

I haven't actually tried it, but I would presume you could set up Janus to work with Kray instead of normal LW, and have the benefits of both.

Cageman
12-24-2009, 06:08 PM
Can this be used for interiors? If so, how?

It's much harder to fake GI for interiors. The reason I keep away from GI (as in FG or Monte-Carlo) as much as possible is because it's not reliable enough for animation, especially the kind of stuff we do. I've never got GI-caches to work properly over Screamernet, which is what makes it kind of useless. The non-interpolated MC, wich is the only one that would work with the stuff we do, is still very slow to render. So... faking is the only way, really.

In such situations, one can bake the FG/MC based GI to textures on all those things that are going to be static within the scene, and fake GI for the characters, either through a lightrig or background GI.

I whish I had a simple answer to this, but I don't. :)

Greenlaw
12-25-2009, 02:18 AM
The pipeline for R+H Box is flexible and adapts to the job we take on. This is necessary because of the variety of work we get and the extremely tight deadlines that come with them. (Four to six weeks is typical, though a lot of jobs have been done in one to two weeks, and eight to ten is a rare luxury for us.)

The first part is getting the job. Typically, clients send us storyboards or at least a treatment, then we figure out how we would do the job, run up a schedule, check to see who's available to work in the Box, and make a bid. Then, when the job lands, we panic. :)

Sometimes the process is more creative: the client will ask us to come up with a concept, board it, and pitch it to them. These ideas come from collaboration between our director and the artists (staffers and freelancers,) and the boards are typically drawn by me and Keith Matz, though these days Keith and I are usually too busy to draw, so the Box has been contracting out to a freelancer. We like these 'show us what you got' situations the best because we'll usually come up with something that looks great but falls reasonably within the client's budget. These projects are the most fun to work on, and the client always gets a lot more for their money.

Once we're cleared to do the job, we have a number of tools available to us. The hard part is finding the talent who can use those tools. We typically like generalists for obvious reasons, but more and more artists these days are specialists. The problem with specialists, for us anyway, is keeping them busy for the whole duration of the job. A good generalist, on the other hand, rarely has downtime and can stay busy right up to delivery day. We appreciate all artists who works with us though, and everybody gets to express their ideas and participate in the creative process.

The actual production schedule can start off with a lot different activities. Some guys will start working on choreo (camera animation) and animatics (rough character animation and fx.) If there is live action involved, we may even previs that. Other artists may be working on final models or doing R&D for complicated fx. At this stage, most of the work is done in LightWave, but we also use other applications where appropriate.

The Box's primary modeling application is LightWave Modeler, but some guys will bring in their own Modo license if they prefer using that program. (I use both, Modeler for most of the work, and Modo for occasional tricky stuff.) The Box also has ZBrush and Mudbox, but a few of us prefer 3DC and will use our own license for this. (Personally, I'm hoping that we get Modo and 3DC officially in-house soon; they've both been too useful for me.)

The Box also has Maya, but we hardly ever use it for modeling. R+H Features, on the other hand, uses Maya for most modeling (as well as ZBrush and Mudbox.)

UV Maps are usually created in Lightwave, but some of the guys who have their licenses will use Modo or Headus UV Layout. (I use all three, and sometimes 3DC to do the job. For organic character stuff, I automatically prefer UV Layout; it's simply the fastest and most accurate.)

(Speaking of UV Layout, the developer has opened it up to run as a plug-in for other apps. Maya already has the connections to run UV Layout internally, but I'm still wishing somebody would do the same for LightWave. Sigh! Anyway, hint, hint.)

Traditionally, we did all our animation and fx in LightWave, but this has changed a lot in the past year. We still do all the camera animation and a lot of our fx work in LightWave, but now we do character animation in Maya (using Point Oven to get data to LightWave,) and with our current job, we started using Houdini for some fx. (Ken Wilder is heading that up for the Box; I'm just dabbling with it when I'm not too busy.) We also sourced out some FumeFX work, so indirectly we are also using Max here.

Occasionally, we use small specialty programs like Groboto, Particle Illusion, and even Poser. In short, we'll use whatever gives us the look we're looking for in the shortest time.

Oh, I almost forgot: we also use Vue. On our current project, I've been using it almost every day to create matte painting elements with great success. I wanted to use it for animated cloud passes, but version 7 doesn't export alpha channels for clouds, which made it mostly useless for this task; too bad because the clouds looked great. (Thank goodness for Hypervoxels, which still makes great animated volumetric clouds.) This Saturday, we're upgrading to version 8 though, which finally adds an exportable 'Cloud Mask,' among a lot of other useful features, which should make Vue more significant to our pipeline this upcoming year.

On some jobs, we use a lot of motion capture data, but we always contract that portion out to a local mocap facility. (Off the clock, I've been experimenting with iPi DMC and Jimmy|Rig, which may eventually help us for previs and animatics, but since both programs are at the 'work-in-progress' stage, I don't see this happening for many months yet.)

We also do green screen shoots. Usually, this is a big production done at real soundstage with professional actors and crew. Sometimes, however, we like to personally shoot animators and other staff members in our cafe area or parking lot, wearing costumes. (Hmm, that doesn't sound quite right.) Sometimes, we also shoot our own fx elements. For 'in-house' shoots we typically use consumer and 'prosumer' equipment, nothing too fancy.

Final rendering is almost always LightWave, but sometimes use use FPrime. We almost never render with motion blur, which has increased our efficiency by a staggering amount, especially since nearly all our work is for HD these days. We use motion vectors from Lightwave to add our motion blur in post using Fusion. Mike Popovich, our brilliant tech guy, has created a ton of useful tools that get us embedded everything in a single .exr file. Thanks to Mike, we can even apply subsurface scattering as a post effect in Fusion using a custom SSS data pass from LightWave.

Compositing is always done using Fusion, and we're starting to take Fusion's 3D environment more seriously. We also do our own editorial using Velocity HD, and occasionally we do our own sound and music, though that's always finished at a real sound facility.

That's about it. I think most smaller facilities work more or less this way. We play fast and a little loose in the Box because we have to, but the whole process is obviously more rigid and structured for big feature film productions like what R+H Features does. I kinda like our crazy system though; it rarely gets boring anyway. :)

Greenlaw

Greenlaw
12-25-2009, 02:54 AM
I forgot to mention tracking. We usually use SynthEyes, but we also use Boujou if we need an alternative track. (All tracking programs seem to give you different results.) Sometimes we rely on good-old-fashioned hand tracking in LightWave (a couple of shots we did for LOTR: The Return of the King were done this way,) or simple 2D tracking in Fusion. For those 'impossible' shots, we turn to Features and their proprietry software, who can nail any shot perfectly everytime. Sometimes they even give us useable terrain geometry generated by the tracking session...they're that amazing!

Greenlaw

Greenlaw
12-25-2009, 03:51 AM
For rendering passes out of a single Lightwave scene, we have MCP, a proprietary system we recently implemented for our current production. It's a work-in-progress, but it's already saving us the huge amount of time we used to spend breaking out layers manually. IMO (and everybody else's I'm sure,) Lightwave really should have had something like this built-in a long time ago.

Greenlaw

Greenlaw
12-25-2009, 03:59 AM
For textures and matte paintings, we're still in the dark ages though, using Photoshop 5.5. We do have Photoshop CS4 installed on one workstation somewhere around here, but I like to use Adobe Creative Suite 4 and Corel Painter 11 on my laptop for textures and matte painting work, and shuffle the files to my workstation.

Greenlaw

Greenlaw
12-25-2009, 04:20 AM
Some of our favorite third-party LW plug-ins (not including our in-house stuff): FPrime, Sasquatch, HD Instance 2.0, and Dynamite Pro. The usual stuff. Oh, and I mustn't forget all the great DP stuff; I especially love DP Infinite Light, which does soft raytraced shadows that work even with FPrime and HD Instance 2.0. It's helped make directional outdoor lighting a lot easier, especially since we can't do the 'spinning light' trick any more (see post way above for the reason.)

In Fusion, I rely a lot on Krokodov (which is now 'Legoware') and all of the Re:Vision FX stuff like Twixtor and ShadeShape.

Greenlaw

Greenlaw
12-25-2009, 04:30 AM
Oh, crap. I still have lots of presents to wrap. Happy holidays everybody!

Greenlaw

geo_n
12-25-2009, 08:33 AM
The hard part is finding the talent who can use those tools. We typically like generalists for obvious reasons, but more and more artists these days are specialists. The problem with specialists, for us anyway, is keeping them busy for the whole duration of the job. A good generalist, on the other hand, rarely has downtime and can stay busy right up to delivery day.

Can I ask how many people usually work in a project? I know it depends on a project but for example a simple commercial like this
http://www.youtube.com/watch?v=pJ0PLFx_Hno or this
http://www.youtube.com/watch?v=LUSM3WcCNSk&feature=related
It is not photoreal or film quality I know.
3 to 6 weeks finished if the client is not so demanding. I work on projects like these with one compositor. There's only 3 of us doing 3D and we're juggling 3 to 4 projects at one time. And the projects vary from lightwave to 3dmax because the client always gets the files after completion. It is very hectic and working 13hours or more is normal. I wonder how the western style work is done.

Captain Obvious
12-25-2009, 09:39 AM
It's much harder to fake GI for interiors. The reason I keep away from GI (as in FG or Monte-Carlo) as much as possible is because it's not reliable enough for animation, especially the kind of stuff we do. I've never got GI-caches to work properly over Screamernet, which is what makes it kind of useless. The non-interpolated MC, wich is the only one that would work with the stuff we do, is still very slow to render. So... faking is the only way, really.
I don't know exactly what you guys do at Massive, but here's my usual pipeline for getting animated GI that renders well:


1st off, use Kray.

If there are no major lighting changes in the scene (just people moving around, that sort of thing), I use a pre-calculated GI solution stored to a file and render a background pass and a people pass.

If there are major lighting changes or movements too big for a cached-to-file GI solution, I use Kray's frame interpolation feature. By mixing the GI of the neighbouring couple of frames, the amount of flickering is significantly reduced. This can cause problems with the light "lagging behind," though.


I can't say I've ever had any major issues with this, and it works really well over BNR/Screamernet.


Edit: it's certainly worth mentioning that architectural visualisation tends to feature quite static scenes. So it obviously doesn't work for everything.

Cageman
12-25-2009, 12:56 PM
1st off, use Kray.

And that made EOD for me. We simply can't use Kray based on numerous things... Janus being the most prominent one...

But... if you know how to make LWs native renderer to not generate TONS of flicker when using GI with a generated cache, feel free to elaborate. The flicker only happens if I render through screamernet. If I load the scene in LW on several machines, no flicker... but that alone is a workflow killer.

:)

EDIT: Oh and yes... if the GI-cache would work on the renderfarm, I would have used GI for the environment (all static stuff) and then render out an occlusionpass and shadowpass for the environment regarding all things that moves (characters etc).

Sensei
12-25-2009, 01:00 PM
Did you lock GI cache file?

Cageman
12-25-2009, 01:05 PM
Did you lock cached GI file?

Oh yes! :)

Of what I've gathered, screamernet can't open the GI-cache in other than locked mode, so even if I would not lock it, screamernt can't add to the cache in any way. Looking in the logs, I can see that screamernet reads and uses the cache, but it still generates flicker.

Sensei
12-25-2009, 01:22 PM
Maybe there is bug that lock file in exclusive access mode in 1st rendering node (the fastest one that loaded scene), then 2nd and others can't access locked file (locked in Operating System sense, not LW), therefore GI cache file is not used on these machines and GI pre-processing is done regular way, not animation friendly..

Try using just one rendering node in ScreamerNet/BNR..
If flickering will be gone, this might be a sign that nodes have file access problem..

Cageman
12-25-2009, 01:33 PM
Maybe there is bug that lock file in exclusive access mode in 1st rendering node (the fastest one that loaded scene), then 2nd and others can't access locked file (locked in Operating System sense, not LW), therefore GI cache file is not used on these machines and GI pre-processing is done regular way, not animation friendly..

Try using just one rendering node in ScreamerNet/BNR..
If flickering will be gone, this might be a sign that nodes have file access problem..

We use Muster, which works with packets rather than always communicating with the server. So, as an example:

Packet size 5 will tell each node to load and render 5 frames, then it will clear the memory and look for the next 5 frames that have yet to be submitted to any node. In this case, the flicker is visible every 6:th frame; a notable difference between the renders from the different nodes, but within the packet from a single node, the renders are fine. It doesn't matter if I use one node or all of them... the only workaround is to make sure that the GI-pass is submitted, in its full, to a single node. Ie. if the scene is 100 frames, I set the packet size to 100 frames, so that a single node will render the whole sequence.

I have not locked the GI-cache on disk... are you saying that THAT might be what I need to do in order to get flicker-free renders? :)

adboy
12-25-2009, 04:52 PM
hey man thanks thats extremely informative and also very close to our pipline except we only really use maya and xsi for modeling

but we use a lot of houdini, its got the the stage where i basically couldnt live without it, i only really started with it about a year ago, but it opens up an insane amount of possibilities, spend alot of my time wondering why all app's dont work that way

we have some of our own file formats for the inport and export of particle and geometry data, make things pretty seamless

we only use renderman and i love it, but man i'm jealous of your exr stuff

just for projects i do at home with lightwave that would be a good send, i take it those are just internal buffers put to different layers withing the exr

complete time saver

again thanks for the insight into your workflow

Cageman
12-25-2009, 06:44 PM
hey man thanks thats extremely informative and also very close to our pipline except we only really use maya and xsi for modeling

but we use a lot of houdini, its got the the stage where i basically couldnt live without it, i only really started with it about a year ago, but it opens up an insane amount of possibilities, spend alot of my time wondering why all app's dont work that way

we have some of our own file formats for the inport and export of particle and geometry data, make things pretty seamless

we only use renderman and i love it, but man i'm jealous of your exr stuff

just for projects i do at home with lightwave that would be a good send, i take it those are just internal buffers put to different layers withing the exr

complete time saver

again thanks for the insight into your workflow

You are welcome! :)

And yes... when I output a renderpass using exrTrader, those are just internal buffers that are stored within a single exr-file. I should give you a full list of those internal buffers. I will do that tomorrow though, since I can't remember them all from within my brain. :D

:)

EDIT: I just realized that maybe you reffered to Greenlaws post? Because I havn't exactly been detailed in my explanation. :)

Captain Obvious
12-25-2009, 07:14 PM
But... if you know how to make LWs native renderer to not generate TONS of flicker when using GI with a generated cache, feel free to elaborate. The flicker only happens if I render through screamernet. If I load the scene in LW on several machines, no flicker... but that alone is a workflow killer.

:)

EDIT: Oh and yes... if the GI-cache would work on the renderfarm, I would have used GI for the environment (all static stuff) and then render out an occlusionpass and shadowpass for the environment regarding all things that moves (characters etc).
To be honest, I'm not sure what sort of problems you're having with it. I mean, it's quite straight forward. :P Pre-process the GI on one machine (there's a button for it), which dumps the GI solution to a file. Then each machine loads it.

I've used this more times than I can count, over BNR/Screamernet. Worked fine every time.

This is pretty much the case for both Kray and Lightwave. You're in Malmö, right? I'm probably heading there soon. I'd happily swing by and have a look.

Cageman
12-25-2009, 08:16 PM
This is pretty much the case for both Kray and Lightwave. You're in Malmö, right? I'm probably heading there soon. I'd happily swing by and have a look.

Cool!

So... when do you think you will swing by? I can prepare
the environment (which would GREATLY benefit from a FG-GI regarding renderspeed) and then we can take a burger at BK and head over to the office to do some testing.

:)

EDIT: Oh...and yes... I'm in Malmö! :)

Greenlaw
12-25-2009, 09:22 PM
Can I ask how many people usually work in a project?

Your numbers sounds about right for commercial work. In the past, our crews ran between four and eight for smaller jobs, and about ten to fifteen for the big ones.

In recent years however, the scope of our work has gotten bigger, and our average seems to be between twelve and twenty artists. We've been prepared for some time (at least mentally,) to expand to about forty or so if needed.

To be clear, that's just the number for the Box at R+H. Depending on what's going on in production, R+H Studios as a whole can expand to over 1000 artists, which I think was our all-time peak earlier this year.

(BTW, if anybody reading this is interested in working at the Box, please send us a demo reel in DVD format, along with a resume and shot breakdown. The address is at the Rhythm & Hues Studios (http://www.rhythm.com/) website, and you can address it directly to Deborah Austin, the Box's producer.)

Greenlaw

Greenlaw
12-25-2009, 10:05 PM
...we use a lot of houdini, its got the the stage where i basically couldnt live without it, i only really started with it about a year ago, but it opens up an insane amount of possibilities, spend alot of my time wondering why all app's dont work that way

I'm barely scratching the surface with Houdini and still have a lot to learn, but Ken keeps raving every day about how intelligent it is compared to other apps for fx work. We're also getting a lot of help from our experts in R+H Features, which I wish I was more involved with but right now my head is too full from cramming Vue 8 xStream for this job. :)

Greenlaw

adboy
12-26-2009, 04:04 AM
heh
hey cageman thanks to you too :)
seeing some new possibilities from these idea's bouncing around :)

djlithium
12-26-2009, 04:16 AM
Over the past couple of weeks I have been sharing my BSG pipeline secrets on Kurv through a series of Videos. The one where I really get into it on "passes" is the lighting and rendering tutorial where I use a combination exrTrader and the DP Light Kit to get everything I want out of it for things we used to use multiple (excessive) render passes and GI or spinning light tricks to do on BSG. Tomorrow (today really at 1PM CST december 26th, 2009) I am doing the first video in a series of 3 videos on the digital pyro techniques we used on the show which also relates to the pipeline topics I have covered in previous videos. More stuff on Kurv soon (register here https://www2.gotomeeting.com/register/964860019 ) as well as on my site www.battlestarvfx.com

I think someone will also be winning another copy of StarPro Unlimited Tomorrow if I am not mistaken. Price is 14.95US for minimum of 2 hours.

Stop by, check it out and as always, any questions, please ask. :) =^..^=

djlithium
12-26-2009, 04:17 AM
Here is a sample image of part of what I will be covering in the video today.

geo_n
12-26-2009, 06:59 AM
Your numbers sounds about right for commercial work. In the past, our crews ran between four and eight for smaller jobs, and about ten to fifteen for the big ones.

In recent years however, the scope of our work has gotten bigger, and our average seems to be between twelve and twenty artists. We've been prepared for some time (at least mentally,) to expand to about forty or so if needed.

To be clear, that's just the number for the Box at R+H. Depending on what's going on in production, R+H Studios as a whole can expand to over 1000 artists, which I think was our all-time peak earlier this year.

(BTW, if anybody reading this is interested in working at the Box, please send us a demo reel in DVD format, along with a resume and shot breakdown. The address is at the Rhythm & Hues Studios (http://www.rhythm.com/) website, and you can address it directly to Deborah Austin, the Box's producer.)

Greenlaw

Four,eight for one project??? I think I didn't post correctly before. 1 person is doing 3 to 4 small projects like that in youtube link. That nissin cup noodle cm is done by one max guy and the other chocolate cm in lightwave. We each work with a compositor or sometimes just do it ourselves if its simple enough.
Right now we're 3 in house 3dguys doing 3-4 projects each. The quality is going down imho. I want to explain to the higher ups that we need more 3d guys but the crisis is really bad in japan and I guess this is cutting cost. If I say its like that in the west maybe they will believe me :D

JonW
12-26-2009, 01:31 PM
EDIT: Oh and yes... if the GI-cache would work on the renderfarm, I would have used GI for the environment (all static stuff) and then render out an occlusionpass and shadowpass for the environment regarding all things that moves (characters etc).

I had problems with SN trying to find the radiosity cache file.
http://newtek.com/forums/showthread.php?t=98254

Cageman
12-26-2009, 02:33 PM
I had problems with SN trying to find the radiosity cache file.
http://newtek.com/forums/showthread.php?t=98254

Thanks for the headsup! :)

I found the issue and what I do is I open up the scenefile and fill in the complete path to the cachefile.

Doing some tests now on my little rendergarden and the only thing I can say is that it takes about 10 times longer to render a frame now. :)

A frame that takes about 9 seconds to render, now takes over 3 minutes *sigh*

So... overall... where I thought I would gain alot of time, I now realise that I loose instead. Fake GI is the only way to do it in a timely manner.

:)

Greenlaw
12-26-2009, 04:30 PM
Four,eight for one project??? I think I didn't post correctly before. 1 person is doing 3 to 4 small projects like that in youtube link.

Sorry, miscommunication on my part. I was actually referring to the type of productions we've been doing lately in the Box (huge environments, multiple characters, and lots of fx at HD res,) in which case the crew size and schedule I quoted is definitely tight. I should also point out is that we really try to keep to a 40 hour week (R+H pays time-and-a-half to double-time for OT, which gets expensive very quickly, so it's actually cheaper for us to throw more artists at a project than to overwork a crew that may be too small for the job. Of course, here I am working over the holidays as I write this.) :)

Even so,

Greenlaw

adboy
12-26-2009, 07:04 PM
so a quick question about the way u use exr's
in renderman we have shaders that basically register what ever u want like, light contribution then can assign that info to a buffer

are u doing something similar and pumping the info to custom buffers in exr trader?

Greenlaw
12-26-2009, 08:14 PM
so a quick question about the way u use exr's
in renderman we have shaders that basically register what ever u want like, light contribution then can assign that info to a buffer

are u doing something similar and pumping the info to custom buffers in exr trader?

We have our own .exr saver. Normally it can optionally save 36 channels that LW makes available, and for our special passes, we use the standard channels to store the special data so, yeah, I think this is similar to what you're describing, but I'm not sure it can create additional channels dynamically. It might also be limited to conform to what Fusion can read, since this is what we composite with. At least, this is how I understand it; the system seems to grow and adapt from job to job, and I'm not completely up-to-date on it at the moment. :p

Greenlaw

geo_n
12-26-2009, 09:06 PM
Sorry, miscommunication on my part. I was actually referring to the type of productions we've been doing lately in the Box (huge environments, multiple characters, and lots of fx at HD res,) in which case the crew size and schedule I quoted is definitely tight. I should also point out is that we really try to keep to a 40 hour week (R+H pays time-and-a-half to double-time for OT, which gets expensive very quickly, so it's actually cheaper for us to throw more artists at a project than to overwork a crew that may be too small for the job. Of course, here I am working over the holidays as I write this.) :)

Even so,

Greenlaw

That's cool! 40 hours week with overtime pay. In japan studios don't pay overtime and 40 hours is unheard of. More like 65 hours a week. Besides that there's no specialist and everybody needs to know something about everything. I would like to propose a western style work process and pipeline to my company. I've seen many excellent works from western studios that I think they're doing something different.

Sensei
12-26-2009, 09:14 PM
That's cool! 40 hours week with overtime pay. In japan studios don't pay overtime and 40 hours is unheard of. More like 65 hours a week.

I don't work that much even in a month.. :p



Besides that there's no specialist and everybody needs to know something about everything. I would like to propose a western style work process and pipeline to my company. I've seen many excellent works from western studios that I think they're doing something different.

IMHO it's good that everybody knows everything. The same is in Poland. One man band is common here, not just in 3d. Definitely easier to survive in hard times. Additionally it's easier to burnout when doing all the time same thing.

geo_n
12-26-2009, 09:38 PM
I don't work that much even in a month.. :p



IMHO it's good that everybody knows everything. The same is in Poland. One man band is common here, not just in 3d. Definitely easier to survive in hard times. Additionally it's easier to burnout when doing all the time same thing.

What's your day job? :D
Maybe I should relocate to poland ;)

Sensei
12-26-2009, 09:47 PM
What's your day job? :D

Day-trader.. ;)



Maybe I should relocate to poland

Live in Poland and have on-line work in Japan and you would be king.. :)
With your own apartment (without credit), you can easily live for 600 usd per month with everyday drinking, everyday going to night clubs and eating in restaurants each day.

Greenlaw
12-26-2009, 11:57 PM
That's cool!

Yeah, R+H is a good company when it comes to its employees. The bad part is living and driving in Los Angeles...my commute is only 25 miles, but it still takes me over an hour to get there. Some of the guys I've worked with live outside of LA and their commute time can be double that. And LA's public transportation is not very practical for getting to this location.

Next year, the company is moving to a nicer and bigger facility, but unfortunately my commute will be even longer. I'm hoping that when we move, I can take the metro rail instead of driving and get some of my personal time back. :)

Greenlaw

Cageman
12-27-2009, 01:32 AM
so a quick question about the way u use exr's
in renderman we have shaders that basically register what ever u want like, light contribution then can assign that info to a buffer

are u doing something similar and pumping the info to custom buffers in exr trader?

exrTrader, as far as I know, can only store the buffers that LightWaves renderengine outputs (I've attached a screenshot showing the buffers that exrTrader can output).

When it comes to special types of passes, there have been some quite elaborate methods to output things like SSS (obviously, LW does not have an SSS-buffer) and many other things. While these can't be hooked into exrTrader, it may give you an idea on how to render those special passes, if need be.

A couple of links:

http://www.spinquad.com/forums/showthread.php?t=26052&highlight=Gerardo
http://www.spinquad.com/forums/showthread.php?t=19309&highlight=pass

:)

Lightwolf
12-27-2009, 02:53 AM
exrTrader, as far as I know, can only store the buffers that LightWaves renderengine outputs (I've attached a screenshot showing the buffers that exrTrader can output).
Yup, whatever LW generates and makes available, exrTrader exports it. More is planned for V2.0 - but that'll be a major revamp and probably implies some conceptual changes as well.

Cheers,
Mike

djlithium
12-27-2009, 03:12 AM
Can't wait Mike.
As always and since I first got my hands on it I am pushing exrTrader hard to anyone who asks. I use it all the time in the demos and tutorials and its indispensable just as an image saver, let alone for exr files. It pretty much replaces lightwave's saving system for me completely which is great :)

happy kat thanks mr. wolf!

Captain Obvious
12-27-2009, 09:22 AM
It might also be limited to conform to what Fusion can read, since this is what we composite with.
Surely Fusion can read and use any arbitrary EXR channels, and then you can just remap them to whatever you need? I haven't used Fusion that much, though, and never for any sort of normal compositing.



R+H pays time-and-a-half to double-time for OT, which gets expensive very quickly, so it's actually cheaper for us to throw more artists at a project than to overwork a crew that may be too small for the job
Uh, are you guys hiring? ;)

Lightwolf
12-27-2009, 10:59 AM
Surely Fusion can read and use any arbitrary EXR channels, and then you can just remap them to whatever you need?
Yes, it can get a bit messy though as the auxiliary channels associated with an image are pretty much based on RPF. You can insert anything into them and also extract them into anything later on, but there is a limit of how many aux channels you can use per loader (after that you need more loaders).
That's one thing I hope they'll tackle for the next major upgrade.

Cheers,
Mike

Cageman
12-27-2009, 11:41 AM
Surely Fusion can read and use any arbitrary EXR channels, and then you can just remap them to whatever you need? I haven't used Fusion that much, though, and never for any sort of normal compositing.

I have yet to stumble across something that doesn't work with Fusion regarding multichannel exr-files. However, Nuke, on the other hand, seem to less flexible in this department and it is somewhat more troublesome to remap a single channel value to be used in R, G and B. There is a remap tool in Nuke, but it is always confusing the hell out of me. :D Fusion, in this regard, is more direct and to the point, but lacks other facilities that Nuke has to streamline the use of multichannel exr-files.

Why can't there be a tool that, once and for all, does everything "correct" from the beginning?

:D

tuomas
12-27-2009, 11:58 AM
That's cool! 40 hours week with overtime pay. In japan studios don't pay overtime and 40 hours is unheard of. More like 65 hours a week. Besides that there's no specialist and everybody needs to know something about everything. I would like to propose a western style work process and pipeline to my company. I've seen many excellent works from western studios that I think they're doing something different.
Keep in mind that long hours with no overtime pay are very typical in small shops all around the world; it's not limited to Japan only. And in those small shops everyone is a generalist to a certain degree, there's no point in giving very specific roles to people when you constantly need them to contribute in various areas of the production.

Lightwolf
12-27-2009, 12:05 PM
I have yet to stumble across something that doesn't work with Fusion regarding multichannel exr-files. However, Nuke, on the other hand, seem to less flexible in this department and it is somewhat more troublesome to remap a single channel value to be used in R, G and B.
Nuke is a lot more picky about the naming of the channels - and doesn't follow the suggestions in the OpenEXR specs 100% either.

Single channels should be named A for example (alpha) to be used properly as opposed to the suggested Y (luminance or greyscale).

Cheers,
Mike

Captain Obvious
12-27-2009, 01:44 PM
Why can't there be a tool that, once and for all, does everything "correct" from the beginning?
Yeah, tell me about it. I like Fusion's tools and workflow (the 'Custom' node is ace), but I hate the viewport navigation tools compared to Shake or Nuke.

Lightwolf
12-27-2009, 03:27 PM
Will V2.0 be for 9.6 and/or CORE?
9.6
As for CORE, We'd rather wait for a final V1.0 before making any decisions on what to develop for it, for a number of reasons.

Cheers,
Mike

Greenlaw
12-27-2009, 07:51 PM
Can't wait Mike.
As always and since I first got my hands on it I am pushing exrTrader hard to anyone who asks.

I should have piped up; while we may not use it at the Box because we have our own proprietary saver, I use exrTrader for my personal projects, and I've found that it works great. I highly recommend it. shaderMeister is very useful too.

Greenlaw

Lightwolf
12-27-2009, 07:57 PM
I should have piped up; while we may not use it at the Box because we have our own proprietary saver, I use exrTrader for my personal projects, and I've found that it works great. I highly recommend it. shaderMeister is very useful too.

Now I'm curious... what could persuade you at the Box to use exrTrader instead of the in-house solution? ;) (I suppose a PM would be more appropriate).

Cheers,
Mike

Cageman
12-27-2009, 08:37 PM
Now I'm curious... what could persuade you at the Box to use exrTrader instead of the in-house solution? ;) (I suppose a PM would be more appropriate).

Cheers,
Mike

...but make sure that we all get to know the answer as well... ;)

Sensei
12-27-2009, 08:40 PM
Now I'm curious... what could persuade you at the Box to use exrTrader instead of the in-house solution? ;) (I suppose a PM would be more appropriate).


Do you want them to bankrupt?! :p

Greenlaw
12-27-2009, 08:42 PM
Surely Fusion can read and use any arbitrary EXR channels, and then you can just remap them to whatever you need? I haven't used Fusion that much, though, and never for any sort of normal compositing.

I think the exr loader had a fixed number of channels in 5, and it didn't read anything above that number, but this may be different in 6. I'll have to check when I'm back at work. And yes, you can remap any channel using the Boolean tool, which is how we used to bring up the z-depth channel, but now we use custom fog and dof tools that read the embedded z data directly, which is much easier.


Uh, are you guys hiring? ;)

Our current project has a full crew, but if another project were to come in tomorrow, we'd need to hire more hands. You just never know. The Box hires experienced freelancers throughout the year, and we're always reviewing demo reels.

Greenlaw

Greenlaw
12-27-2009, 09:59 PM
Now I'm curious... what could persuade you at the Box to use exrTrader instead of the in-house solution?

Hi Mike,

I don't make final purchasing decisions, but I do make recommendations, and at one time we almost did get exrTrader for the Box.

However, our tech guy Mike Popovich wrote an exr saver for the Box and it's very specific to the Box's pipeline. And because Mike can adapt the saver for new features as we need them, switching to exrTrader isn't likely to happen.

That doesn't stop me from recommending exrTrader to anybody else though. I like using it on my own projects, so please keep up the great work! :)

Greenlaw

adboy
12-29-2009, 05:31 PM
sorry man, what is the difference between the box and r and h, i think i missed something:)

Lightwolf
12-29-2009, 05:41 PM
However, our tech guy Mike Popovich wrote an exr saver for the Box and it's very specific to the Box's pipeline. And because Mike can adapt the saver for new features as we need them, switching to exrTrader isn't likely to happen.
Well, it was almost a rethorical question anyhow, I suspected as much (and would probably do the same in a similar situation).

Cheers,
Mike