PDA

View Full Version : Good light brightness for outdoors w/ linear workflow?



jrandom
03-17-2011, 08:21 PM
I'm diving into making physically-accurate materials as practice and I'm curious what a realistic light value would be for the sun. Since linear color (with sRGB re-applied to the display/output) is brighter, making blurry-reflection-for-specular objects gets difficult since getting a good bright highlight winds up making the rest of the surface too mirror-like.

Setting the output display to Linear fixes this (you use brighter lights and a lower reflection value so only really bright things like the sun show up as major specular highlights) but it's not very realistic from a real-world perspective.

Any advice? How bright should my sun be?

...

Since I'm on the subject, how bright is the sky compared to the sun?

Interesting side-note: When the display/output is set to sRGB, using a light value of 100%, and a fully-white material with its diffuse set to 0.18 (or 18%), this results in a material that is at the same brightness as a real-world 18% gray card used in photography -- eg. brighter than you'd naively expect. (I always thought 18% reflectance would be much darker, but in the really-real world it looks like 50% gray from a brightness perspective, and a non-linear workflow would have to use a 50% diffuse value to achieve the same result.)

jameswillmott
03-17-2011, 10:50 PM
Since I'm on the subject, how bright is the sky compared to the sun?


Download an HDR sky map and just read the values of sun and sky to compare.

clagman
03-18-2011, 06:58 AM
HDR is ok. I tend to feel that with multi-exposure photograph HDRs we still aren't getting enough gain in the over-bright areas that you do in nature (for the sun anyway). Example:

A decent HDR image and standard reflection/specular settings of say 50% or less. The resulting brightness of the reflection/specular channel doesn't have enough gain (for my taste) once you get it into post and you have to adjust (which is fine to do). At least compared to realistic camera exposure settings.

jrandom, I've found that tweaking in post is something that I almost always end up doing rather than attempting to get everything exact "in camera". I've even gotten into the habit of setting some things up specifically to work with in post when I know I'm going for a particular effect. So I'd say remember to use post as a tool for sweetening the shot. It sounds like what you are needing is to take the shot into Photoshop (or whateveryagot) and using curves/levels/exposure to increase the contrast. Even better, export each buffer separately and adjust a way.

Either that or I'm clueless. I can never know which it is ;^{)>


Oh as a side note I usually go over 100% on my lights for doing sun/outdoor shots. Vue sun_light is closer to 300%.

jrandom
03-18-2011, 10:27 AM
Post-process is always important. What I'm trying to figure out are good settings for the raw CG output to make sure that all my materials will look consistent together.

Without linear workflow, setting colors darker and lights brighter works very well, but when using linear and setting the display/output so sRGB it becomes much trickier. Hm... might have to start working with the HDR exposure filter...

jrandom
03-18-2011, 08:27 PM
Update: I've solved this problem by wiring up my Sun light to a master channel to set intensity (eg. 400% or 1000%), and then wire up nodes in my material to divide only the diffuse amount (eg: not specular) by the same light intensity master channel value.

This allows me to alter my sunlight intensity (and therefore the brightness of the specular highlights on a Delta material) while leaving the diffuse brightness unchanged in the final render.

jrandom
03-18-2011, 08:36 PM
Here's a screenshot of my base material node network (a work-in-progress). Once this is finished I will start constructing my surfaces from this starting point (so I don't have to recreate it from scratch each time):

http://www.newtek.com/forums/attachment.php?attachmentid=93775&stc=1&d=1300502167

The nodes on the left are my control nodes. I'm using Multiply nodes so I can wire in various nodes without having to remember where each one's output goes in the middle section. (This is where a simple pass-through scalar node would come in handy, but I guess Multiply will have to do for now.)

The highlighted "Diffuse Exposure Compensation" scalar node is hooked up to take it's value from the same Master Channel used to set the Sun light's brightness.

The "Total Light Reflectance" controls the total amount of light reflected from both diffuse and specular/reflection combined, making it easier to specify the physical attributes of the surface. It is currently set to 18% (0.18) to mimic the photography 18% gray card I'm using as a visual comparison metric.

Current downside: I'm using Sky Tracer 2, and it doesn't allow the sky's luminosity to be assigned to an envelope, so every time I alter my light's brightness I have to manually change the sky luminosity setting by hand. Argh.

Edit: I've attached the current node network pictured above in case anyone else wants to play around with it. I'm currently working on blurry reflection sample count optimisations, but it's not going as well as I'd hoped so I'm not including those bits here.

Tobian
03-18-2011, 08:58 PM
There's no correlation between the brightness of pixels in an HDR image and how bright a light should be in LW, because LW light units don't 'mean' anything at all.Point in case, if you take an area light, in a scene, and enlarge it by 10x.. what happens? the shadows get softer (and the specular gets a little larger). This is nothing like what would happen in real life, because increasing the area by 100 would add 100x more light to the scene. This is exactly what happens if you use luminous geometry.

With most HDR's you don't actually get an accurate capture of the sun's real brightness. You know this because they don't give a hot enough 'spec' hit on a material, which you need to suplement with fake specular. The converse issue with them though, is if they contribute illumination to the scene, then accurate capture would cause other issues, especially in LW as, in radiosity in LW becomes blotchier the higher the contrast ratio and the smaller the bright point, because nothing transmits rays (from bright polygons or images) so it has to rely on having high rays-per evaluation. The smaller the luminous polygon/brighter the contrast, the more likely you will get blotches, because of misses.

This is where you start to really push LW's rendering engine, and you have to accept that lights+spec+radiosity+reflection creates better solutions than just radiosity+reflection alone. We'll have to wait for real light emitting polygons, and real lighting units, for that to become closer to being possible.

jrandom
03-18-2011, 09:01 PM
Yep, that's why I'm currently working with actual Lightwave lights at the moment. I'll tackle the HDR problem once I've got the basics sorted out. :)

Tobian
03-18-2011, 09:13 PM
The other problem with this setup is the delta material...

1) materials are slightly slower than hand-mixed materials, except in features like reflection/refraction interpolation.

2) the specular in all of the materials is similar (but not the same as) the cook-torrence shader. This shader is broken, because it clamps at 1. You have to do a complex workaround, messing around with the normal to stop it happening, otherwise the spec won't properly work.

3) the default LW reflection/refraction blur is by far the fastest and most optimised, it even renders faster in the radiosity buffer, as no blur is applied there (I wish other reflection shaders did that!). Interpolation can be faster, but it is tricky to get working without blotchiness.

4) All LW reflections aren't really very accurate, in the way they smear specular, as the normal faces away from the observer. Again, there are a number of hacks for this, and of course DP's reflection node, which I tend to use these days, it's just quite slow, compared to native reflection :(

jrandom
03-18-2011, 09:40 PM
The other problem with this setup is the delta material...

1) materials are slightly slower than hand-mixed materials, except in features like reflection/refraction interpolation.

Oh man, I had forgotten about that. I started with a hand-mixed material but got so lost while trying to achieve photorealism and light conservation (what with all the node experimentation I'm playing with) that I'm threw it the delta to simplify the node network.

I got pretty close to a working hand-mixed material, but could never get the specular highlight to mix correctly with the blurry-reflection-generated specular (they mix if you're using something like Sky Tracer or other kind of background that has a bright hotspot where the light is).

You think it's worth it for me to roll my own materials instead of using the pre-made ones?

(It's weird that the material nodes would be slower. You'd think they'd be faster because all of the code for them would be in a single module instead of the program having to jump around to other nodes (which I'm betting does a real number on the CPU cache)).



2) the specular in all of the materials is similar (but not the same as) the cook-torrence shader. This shader is broken, because it clamps at 1. You have to do a complex workaround, messing around with the normal to stop it happening, otherwise the spec won't properly work.

I've heard this mentioned before. Do you have a link to a description of the workaround? I'm curious to see how it works.



3) the default LW reflection/refraction blur is by far the fastest and most optimised, it even renders faster in the radiosity buffer, as no blur is applied there (I wish other reflection shaders did that!). Interpolation can be faster, but it is tricky to get working without blotchiness.

I've never been able to get non-blotchy results out of interpolated reflection/refraction. The default LW reflection blur is definitely noticeably faster. I wish there was a way to use materials but keep the default LW reflection blur.



4) All LW reflections aren't really very accurate, in the way they smear specular, as the normal faces away from the observer. Again, there are a number of hacks for this, and of course DP's reflection node, which I tend to use these days, it's just quite slow, compared to native reflection :(

I'd love to read up on any hacks that you know of. I'm still hammering out my basic node network design and need to get it solid before I dive into the fun part (actually making surfaces).

I can't use any DP nodes -- I want to stick to standard nodes since not all plugins work on all platforms (as a Mac user this is a big issue), standard nodes means I can give my results to others and they'll be able to use them immediately, and will make my life a lot easier if I ever have to rent time on a render farm.

Matt
03-18-2011, 10:22 PM
2) the specular in all of the materials is similar (but not the same as) the cook-torrence shader. This shader is broken, because it clamps at 1. You have to do a complex workaround, messing around with the normal to stop it happening, otherwise the spec won't properly work.

I keep hearing about this Cook Torrance shader being apparently broken too. Can you elaborate on exactly what you mean, because the last time I spoke to Antti he checked it and told me it was working as it should.

jrandom
03-18-2011, 10:42 PM
Oh hey, it's a Matt!

*waves frantically*

Any technical explanation for things like the Delta material being slower than a hand-rolled one? What does it do that slows it down? Is it a quality/speed tradeoff?

jrandom
03-18-2011, 11:18 PM
Image test using the same input settings:

Color: 100% White
Total Reflection/Diffuse output: 18%
Reflectivity/Specular: 50%
Roughness: 50%
Diffuse Exposure Compensation: 10 (counterbalances the light luminosity for diffuse, see below)

Sun light is at 1000% with Sky Tracer 2 set at a luminosity of 10%.

(Granted, this isn't the best mockup due to the open top not accurately reflecting more enclosed scenes that would incur a higher amount of recursive reflections, but it's a start.)



Delta Material: 1m 14s

Pretty much exactly what I would expect from the inputs, exhibits close to physically-accurate qualities. This is the baseline image.
http://www.newtek.com/forums/attachment.php?attachmentid=93781&stc=1&d=1300511362


Standard Material: 1m 15s

Very plasticky/CG-looking, specular highlight is way off.
http://www.newtek.com/forums/attachment.php?attachmentid=93782&stc=1&d=1300511573


Make Material: 1m 12s
Diffuse: OrenNayar
Specular: CookTorrence
Reflection: Reflections

Specular highlight is the wrong size but might be fixable if I apply some kind of gamma curve (or similar non-linear function) to the conversion from the Roughness setting to the shader's Glossyness input. The blurry reflections are... I don't exactly know what happened there. They look interpolated, but they're not. The same number of samples was used for all three images. Appears to be less rough than the Delta blurry reflections, as seen in the tight sunlight hotspot reflected from the Sky Tracer 2 background.
http://www.newtek.com/forums/attachment.php?attachmentid=93783&stc=1&d=1300511718


Conclusion: Delta wins on a quality level by a wide margin. Make Material looks the most fixable as far as matching the look of the Delta material, but will take some doing to come up with a set of node calculations that achieve this from the same starting inputs. Not sure how to fix the blurry reflections, though. Also, had to dim the reflection amount by taking the scalar node representing the total reflection and wiring it into the reflection node's color input, and turning on "tint reflections". Otherwise reflections were at 100% no matter what else I did. Seems that Make Material could use a Reflection Amount input similar to the Standard Surface input that normally controls this (when the reflection shader is plugged directly into it instead of Make Material).


Make Material seems to have two massive drawbacks:

1. The above-mentioned reflectivity amount problem
2. No bump map/normal map input

#2 is especially serious because that means making multi-material surfaces (eg. mixing two previously-constructed materials via a mask) where each material has its own bump map becomes horribly complicated. All the other material nodes have bump map inputs, so why doesn't Make Material? The only current solution is to convert the individual bump maps into into colors, mix using a mask, re-convert back to vectors, and plug into the Standard Surface bump map input. If more than two materials get involved, the total node count really climbs.

Heck, if Standard Material had its own shader inputs like Standard Surface does, that would also fix the problem. Why oh why isn't the world perfectly like I want it??

funk
03-18-2011, 11:42 PM
I keep hearing about this Cook Torrance shader being apparently broken too. Can you elaborate on exactly what you mean, because the last time I spoke to Antti he checked it and told me it was working as it should.

Here's the original thread about the cook torrance clamping in lw http://www.newtek.com/forums/showthread.php?t=96074

http://imagic.ddgenvivo.tv/forums/ESC/CTfix.png

Tobian
03-18-2011, 11:47 PM
http://www.newtek.com/forums/showthread.php?t=96074 it has it's own thread. Sadly Gerrardo's image links have broken. It's a long know about bug and I am surprised Antti doesn't know about it. The workaround hack is basically multiplying a spot normal with a vector value. it's possible to get a subtler effect if you use a incidence angle, but it can be tricky to set up. I have no idea if it's been fogbugged, but I get a bit cynical about such things, since we just get told there's nothing wrong :p

Back to the subject at hand. There is a fully up to date DP node for the mac, what's the problem? and the workarounds aren't ideal, and not quite right either. there's a whole slew of them in the http://www.newtek.com/forums/showthread.php?t=104365 thread, but it's quite long, and I warn you, for the most part they don't work :D

jrandom
03-18-2011, 11:59 PM
I stopped using the DP nodes after that fiasco where he pulled them from distribution because one other person had distributed them to other people.

I'd like to emphasize this: DPont pulled his nodes for awhile because someone else gave someone else his free plugins.

I don't get the logic behind that, and I can't bring myself to rely on something that could vanish at any moment because of something so trivial. He's a talented developer and his nodes are very useful but I'm not going to put them into my workflow. What if I wanted to rent time on a render farm? I'd have to distribute the plugins to them for use. Does that violate the DP Nodes terms and conditions? I have no idea because I don't understand the actions of the developer. What if someone else upsets him and they get pulled again? What if the Mac nodes fall behind the Windows versions again?

I also plan on distributing my surfaces for free once I get some decent ones made and I'd like to not to have to require any users of them to go track down and download third-party plugins.

jrandom
03-19-2011, 12:03 AM
... ?

(browser malfunction, ignore this entry)

jrandom
03-19-2011, 01:04 AM
One final render for the night: I managed to get a look very close to the Delta material via Make Material simply by multiplying the Roughness by 4 before routing to Reflections' "Blur" input and then putting a Gamma function of 0.4 into the CookTorrance specular node.

Downsides:

No separate bump/normal channel, making mixed-material surfaces prohibitively difficult.
Render time was 6 seconds slower than the Delta material.


http://www.newtek.com/forums/attachment.php?attachmentid=93786&stc=1&d=1300518231

dpont
03-19-2011, 01:22 AM
I stopped using the DP nodes after that fiasco where he pulled them from distribution because one other person had distributed them to other people...

You are wrong, you don't know the reasons of this incident.



..I'd like to emphasize this: DPont pulled his nodes for awhile because someone else gave someone else his [i]free plugins...

A negative synchroneous event, but no direct relation.

It seems that many people react more about rumours
than real facts here, but this forum has moderation policy,
and I'm not authorized to expose publicly anything
concerning what caused this economic failure,
so you have more chance to read fancifull posts
that pieces of truth.


...
I don't get the logic behind that, and I can't bring myself to rely on something that could vanish at any moment because of something so trivial. He's a talented developer and his nodes are very useful but I'm not going to put them into my workflow. What if I wanted to rent time on a render farm? I'd have to distribute the plugins to them for use. Does that violate the DP Nodes terms and conditions? I have no idea because I don't understand the actions of the developer. What if someone else upsets him and they get pulled again? What if the Mac nodes fall behind the Windows versions again?..

Conditions are clearly written in each readme file.
Until you downloaded them, you are free to make
safe copies, and exchange them with co-workers in
a private way, (don't know how I could control this..)
but I'm opposed to public distributions on
cd, dvd, or internet web, especially in the NT Forum,
without my permission.

You are of course free to not use them.
Denis.

jrandom
03-19-2011, 01:50 AM
You are wrong, you don't know the reasons of this incident.

I stand corrected then. Step #1 in facing the world is always being willing to admit that I am wrong about something. It would be absurd of me to claim I know everything because I sure as heck don't.



A negative synchroneous event, but no direct relation.

It seems that many people react more about rumours
than real facts here, but this forum has moderation policy,
and I'm not authorized to expose publicly anything
concerning what caused this economic failure,
so you have more chance to read fancifull posts
that pieces of truth.

Argh. Darnit... see, now I am unable to construct a more accurate model of the world I live in. Curses. I hate it when I have to rely on unreliable information. Still, I have to respect your decision to not reveal your reasons.



Conditions are clearly written in each readme file.
Until you downloaded them, you are free to make
safe copies, and exchange them with co-workers in
a private way, (don't know how I could control this..)
but I'm opposed to public distributions on
cd, dvd, or internet web, especially in the NT Forum,
without my permission.

Yes, of course. You are free to apply any restrictions you desire to the software you have painstakingly created. I have no illusions about how much work goes into writing software as I, too, am a developer (but not yet at the point where I can write my own plugins).

What scares me is my inability to comprehend the particular restrictions you have selected. This isn't meant as a slight on your character, I just kinda freak out when I encounter things that don't make any logical sense to me.

There are many such things in this world. Might be why I'm such a shut-in.



You are of course free to not use them.

I know, it's just that they get recommended as the answer to so many Lightwave problems so frequently, which makes it difficult sometimes to find other answers.

And quite frankly, the sudden disappearance of them happened just as I was starting to rely on them, and this is what has made me very skittish about using them in the future.

I guess it's time for me to buckle down and finally learn how to write my own nodes.

BTW, your reply was downright poetic. I don't see that very often these days. Thank you.

Tobian
03-19-2011, 06:29 AM
jrandom. I can respect your POV, but 'solving' the reflection blur issue is not something as simple as just plugging a bunch of nodes in to 'fix it'. If you follow the thread the way the reflection shader takes samples is wrong, and it's very hard to compensate for. Several people tried and failed. The only real workaround is micronoise/turbulence, but this can add HUGE time to the render time, if you want it to be smooth, so given that, I use the DP shader. Sometimes you can fix issues with clever workarounds.. and sometimes you can't because of a lack of functionality, or speed issues.

Oh and DP, any speedups on the way for your reflection shader? :)

jrandom
03-19-2011, 11:43 PM
Well, here it is. My final staring-point rig for materials. This design is intended to make it easy to plug various nodes into the settings nodes on the left-hand side.

This isn't intended to be a fully-generalized solution for everything -- rather it's to be used as a starting point for materials with some of the pain-in-the-rear legwork already done in the middle section. The middle section shouldn't need any modification in most cases, but after a material has been designed, any unnecessary nodes can be deleted.

Use Instructions:
Note: For the following settings, 0 is 0% and 1 is 100%: Total Light Reflectance, Specular Amount, and Roughness

Primary Settings:

Diffuse Color: Primary color information. (See Total Light Reflectance)
Bump: Passthrough for bump maps.
Normal: Passthrough for normal maps
Total Light Reflectance: The total amount of light reflected from the object (diffuse + reflection). The Diffuse Color and Specular Amount settings are modulated by this setting. When using linear light, a Total Light Reflectance of 18% (0.18) mimics the appearance of an 18% gray card used in photography (although probably a tad darker than the real-world equivelant)
Specular Amount: Controls the degree of reflectivity and also the Standard LW Light specular highlight of the object.
Roughness: Controls the roughness of the diffuse and reflection properties of the material.
Diffuse Exposure Compensation: Allows for light sources brighter than 100% while keeping the diffuse colors properly exposed. This allows for bright specular highlights on minimally-reflective objects, simulating the look of many materials in direct sunlight. If the LW light intensity is 100%, this setting should be 1. If the light intensity is 300%, this setting would be 3, and so on and so forth. It's best to assign an envelope to this setting so that multiple materials can be used while still allowing for easy light intensity changes (without having to go back and changing this individually for each material).


Optimizers

Max Reflection Samples: The maximum allowed samples for blurry reflections. After 2 bounces, the number of samples is set to 1 instead. (I forgot to make a settings node for this so the bounce limit is hardcoded in the Limit Reflection Samples node in the middle section).
Switch to Simple Material after Bounce #: After the number of specified bounces has been passed, the output material switches to the Simple Material node instead of the Full Material (see below).


Full vs. Simple Material
For regular rendering camera rays and the first initial bounces, the Full Material is used. After the specified number of bounces has been passed, and for Radiosity and Preprocess rays, the Simple Material is used instead.

The Simple Material uses no blurry reflections, only the diffuse coloring and regular specular highlights. No bump or normal maps are used. This generates a very fast radiosity pass.

A previous optimization technique was to turn off reflectivity in the Full Material after the specified number of bounces, but this prevents reflections of specular highlights, and if used as a Radiosity and Preprocess limiter, prevents specular highlights from generating bounced lighting from LW lights. This is why the Simple Material is used instead of standard reflection suppression.

Caveat 1: The Simple Material's Glossyness and Specular settings are a close match for the Full Material, especially at a Roughness setting of 50% (0.5), but higher and lower settings are not an exact match. Close, but not exact.

Caveat 2: There's no easy-and-fast-to-render fix for accurate blurry reflections, so that issue is not currently addressed here.

Future Additions: I'm experimenting with using the Camera FOV and Ray Distance to lower the octaves used in fractal noise nodes (to help cut down on the amount of AA necessary for smooth renders when using them as bump maps), but that still has a way to go and so is not included here.

Whew.

Okay, so if any of you are crazy enough to play around with this madness, I'd be curious to hear your opinions. I'm cautiously suspicious that I'm just wasting my time better spent on hand-crafting simple materials that render fast, but this is just so much fun! (Good node-editing practice, too.)

Edit: Went and added the Limit Reflection Samples setting, had the wrong value hardcoded in the logic node anyhow. The .zip file has been updated.

ingo
03-20-2011, 04:15 AM
Wow, what a simple material setup, glad you had so much fun :hey: :D Where are the fresnel settings ?

clagman
03-20-2011, 08:08 AM
Hahaha! I couldnt bring myself to say it...

jrandom
03-20-2011, 09:17 AM
Wow, what a simple material setup, glad you had so much fun :hey: :D Where are the fresnel settings ?

... aw crud. I knew I was forgetting something.



Hahaha! I couldnt bring myself to say it...

But... but... you see... it is simple! From a functionality standpoint it's really quite straightforward... see the... those things there... and how the... with this...

I am such a dork. :rolleyes:

And of course by "Final" I meant "will still be tweaking the darned thing for hours". I put in the ability to have simple non-blurry reflections for the Simple Material to allow for more accurate GI without killing render times. You can specify how many bounces before it shuts of reflections (use -1 to not have reflections at all for Simple Material).

Now I have to go look at fresnel...

ingo
03-20-2011, 10:12 AM
final 1, final 2 ... who cares :D

What worries me is the mix of specular and reflection, but thats a problem with the composed materials like carpaint shader too. I try to avoid using specular since this is just a fake to speed things up, using reflection only (and blur it) is the way to go .... but slow.

jrandom
03-20-2011, 10:30 AM
final 1, final 2 ... who cares :D

What worries me is the mix of specular and reflection, but thats a problem with the composed materials like carpaint shader too. I try to avoid using specular since this is just a fake to speed things up, using reflection only (and blur it) is the way to go .... but slow.

Yes, reflection blur is freakin' awsome and specular is just a fast workaround but... BUT... regular LW lights aren't visible. There's nothing to reflect, so specular highlights are still necessary. Also, using specular highlights with no real reflection gets me faster GI passes. This lets me crank my GI settings higher to help combat flicker while the HappyFun blurry reflection version of the material still gets used for the actual render.

What I'm going to try to do is come up with two versions of each material I create -- a slower physical model, and then a faster "faked" version so that I can choose whatever material will work best for any given scene.

Sometimes you need CPU-killing fresnel reflecting refracting blurring and volumetric radiosity, and other times you can get away with Blinn and a bump map. :)

COBRASoft
03-20-2011, 10:42 AM
Looks very easy... NOT! :)

If somebody could 'combine' this whole setup into just 1 node, now that would be very kind, wouldn't it. I really admire your persistance jrandom and the result is certainly looking great.

jrandom
03-20-2011, 11:13 AM
Looks very easy... NOT! :)

If somebody could 'combine' this whole setup into just 1 node, now that would be very kind, wouldn't it. I really admire your persistance jrandom and the result is certainly looking great.

It's a definite possibility once I get into Node coding.

Actually, "easy" is the whole point of this monstrosity. The internals kinda sorta look complicated, but controlling them via the settings nodes on the left is very simple. The only weird bit is the "Diffuse Exposure Compensation", and that's really just the same as your light intensity (where 100% = 1)... or uh... you can set it differently if you need the diffused parts of your material to be overexposed when in direct lighting (say, sunlight coming in through a window or whatnot) -- So if your light is at 300% and the DEC is set to 1 then your diffuse can be up to three times brighter than 100%.

So simple my Mom could... well, okay, maybe not.

Captain Obvious
03-21-2011, 12:36 PM
I keep hearing about this Cook Torrance shader being apparently broken too. Can you elaborate on exactly what you mean, because the last time I spoke to Antti he checked it and told me it was working as it should.
Cook Torrence is broken, as are all the other specular shaders in Lightwave.

They internally clamp their values at 1.0, which means that they do not work properly with anything other than very high roughness values (low glossiness).

I think I bugged this a couple of years ago, in fact.

Matt
03-21-2011, 10:37 PM
Cook Torrence is broken, as are all the other specular shaders in Lightwave.

They internally clamp their values at 1.0, which means that they do not work properly with anything other than very high roughness values (low glossiness).

I think I bugged this a couple of years ago, in fact.

I asked Anttii about this, he assured me they are not. The 'clamping' is energy preservation.

I'm sure some will argue the merits of that, but that's what they are doing.

ingo
03-22-2011, 02:42 AM
Hey guys, how about posting some examples how it looks and how it should look with the correct settings. Would make things easier to understand.

Captain Obvious
03-22-2011, 04:55 AM
I asked Anttii about this, he assured me they are not. The 'clamping' is energy preservation.
That's stupid. It's not energy conservation. It's just incorrect, no matter wha Anttii says.

Tobian
03-22-2011, 07:49 AM
examples are posted in the thread posted a few posts up in the 'what's up with the cook torrence shaders' thread.

I am with Captain Obvious here, that's nonsense, and not what energy conservation means at all! The point about energy conservation is that it doesn't reflect more light back than it recieves. However, if it recieves more light than 100% then it should likewise be able to reflect more light than 100% - not just arbitrarily clamp it at 100%.

The issue here is partially because lights in LW are based on imaginary values, and the size and energy of lights are abstract, however, if you took a standard light, set it to inverse squared falloff and put it next to a 100% diffuse white surface, you'd get a hotspot several thousand percent high (which is what it's supposed to do) and if you then used that as a reflector card, it'd be correct. converserly, the specular would get clamped at 100%, which is both unrealistic and contrary to linear high-dynamic workflows. You've effectively put a 'limit dynamic range' switch on the specular shader, which should be totally optional, as it is in the renderer.

Matt
03-24-2011, 09:21 AM
That's stupid. It's not energy conservation. It's just incorrect, no matter wha Anttii says.

Ummm, thanks for that.

Also, I could be using the wrong words here! (After speaking to Antti again).

Matt
03-24-2011, 09:24 AM
You've effectively put a 'limit dynamic range' switch on the specular shader, which should be totally optional, as it is in the renderer.

That's a better way to phrase it, and I agree, but I'm just passing on what I've been told.

Captain Obvious
03-24-2011, 10:31 AM
Ummm, thanks for that.
Sorry, I didn't mean you're stupid. I mean the "energy conservation" excuse, specifically. I certainly have a way with words, don't I? My apologies, I didn't mean to offend anyone in particular.

However, saying this is energy conservation is just plain WRONG. There are no two ways about it. Intentionally clamping the Cook Torrence function means whoever wrote it simply hasn't understood the maths behind it, and that's that.

Intuition
03-24-2011, 10:46 AM
In vray the clamping does not really clamp the value of the brightness component of a reflected color but rather clamps its value for anti-aliasing purposes.

You may want to clamp values for this reason. Yet its clamping at the render level on a pixel comparison basis and not on the shader itself.

Captain Obvious
03-24-2011, 10:49 AM
In vray the clamping does not really clamp the value of the brightness component of a reflected color but rather clamps its value for anti-aliasing purposes.
Yes, exactly like Lightwave's option to limit the dynamic range. It happens after shading is complete, but on a subpixel level. You can do subpixel tonemapping in Lightwave with Pixel Filters.

Tobian
03-24-2011, 11:18 AM
Ok here's a quick test I did, just to showcase some of the issues here.

On the left are bounce cards next to area lights set to be inverse squared falloff but not affecting the ball. On the right are regular area lights. On each side I have moved them further away from the object. the lights in all cases are 1000% bright, and the material is *sic* energy consaerving (the spec isn't setup with energy conserving, or else it would be even less visible!)

If you use inverse squared falloff the lights get dimmer hotspots, the further away you get. this is not how it happens in nature, or indeed with the reflection examples to the left (The larger one is slightly brighter, but that's because of Fresnel falloff, I should have done them at the same fresnel angle now I think of it!)

If you set the lights to be non falloff lights, the spec is much closer to being correct (nuclear brightness!, it is a 10x bright light!)

If you use instead spherical lights with a very small radius (I could have used point lights) the problem becomes more apparent. the further away lights become dimmer, whereas they should just become smaller, as the examples to the left do. This is what happens in nature (specular becomes smaller, not dimmer, depending on distance) and in the reflection shaders also.

As Captain Obvious says, if you had much more diffuse blurry speculars, then the dim-ness would be roughly correct, but for tight high glossiness, it's completely wrong.

Pictures attached in EXR brightness, so you can analise the hdr brightness.

Tobian
03-24-2011, 11:29 AM
Here's the images in JPEG for those who can't look at EXR's and the scene, to play round with.

Note in 2 of those renders, I had enlarged the test card slightly, so there's an odd border round the highlight. I reset it back to 1m for the last one.

jrandom
03-24-2011, 11:33 AM
Oh yikes! I see exactly what's going on. I've been using direct lights recently which would explain why I never ran into this before.

NewTek! Please fix! Our sanity depends on it!

Captain Obvious
03-24-2011, 12:07 PM
Here's some more images. The first one shows the correct results, rendered out of modo. The second is what Lightwave renders. It won't match exactly because modo uses a different specular model that permits anisotropy (Cook Torrence doesn't). But they're fairly similar, apart from the massive error.

See how Lightwave produces a plateau. This is with a standard Cook Torrence node set to 10 % specularity.

Tobian
03-24-2011, 12:35 PM
In that image, is the light a really bright one, but some distance away, with an inverse square falloff, otherwise the highlight shouldn't be SOO bright at only 10% spec? :)

Captain Obvious
03-24-2011, 12:39 PM
The light is a default distant light, so (obviously) no falloff. Glossiness is at 90 % (roughness at 10 % in modo) which is why the highlight is so strong.

The whole thing is correct.

Captain Obvious
03-24-2011, 12:48 PM
(This is all modo, but 3dsmax/VRay gives the same results.)

Here's a correctness test. The first image is a scene with an area light, 10 % specularity and 100 % fresnel, a bit of diffuse, etc.


The second image is exactly the same, except the light has been replaced with a luminous polygon set to the *same* intensity. The images aren't exactly the same, but all in all they're pretty similar. The similarity between the specular highlight and the reflection of the emissive polygon is just astounding.

Tobian
03-24-2011, 01:06 PM
Well that's somewhat sickening :p

Here's a test I did, with a higher blur value and a (modified) inverse spec value so the disks about matched.... erghhhh!!!!!!

I will also point out this has an AA of 96, and I still can't get the noise out of the blurry reflections. This is one of the weaknesses of LW's blurry reflections, especially with high-contrasts and small sizes. (and one of the reasons why I would like a good Spec to replace where I'd need to use that!

Tobian
03-24-2011, 01:37 PM
Tried out the DP reflections, and I am getting much nicer and smoother results. Possibly because I used the default value of 27 samples, but I don't know what LW's and DP's sampling relativeness is like? I know I'd never use 27 samples in a normal render as the recursion causes it to take a billion years if there's much of it! :)

I don't seem to be able to establish any kind of relationship between how blurriness is altering the reflections vs how cook torrence is doing it, no matter how much I faff on trying to match the ratios.

jrandom
03-24-2011, 01:51 PM
I know I'd never use 27 samples in a normal render as the recursion causes it to take a billion years if there's much of it! :)

Hmm.. well, there is this insane node network posted by a madman on this forum that looks exactly like me that avoids the exponential recursion issue... :rolleyes:

Edit: Realized that the latest version of the insane node network was posted on another thread and not this one, so adding it here for completeness.



I don't seem to be able to establish any kind of relationship between how blurriness is altering the reflections vs how cook torrence is doing it, no matter how much I faff on trying to match the ratios.

I wound up running the Roughness value through a gamma of 0.4545, inverting it, and using this new value for the Glossiness input (and also to attenuate the specular amount). This got me much closer to matching the look of the blurry-reflection version of the material.

Tobian
03-24-2011, 02:06 PM
the best way to do it would be have the same turbitidy code which sent out sampling rays for the reflection match the specular, that way both of them would do the same thing, and be on the same page. Ideally it would be good if they could write a reflection shader, which also generates specular, and ideally also have anisotropy in there too.

Point in case, here's the test sphere with micro-noise applied to it. The Spec and reflection almost match. Micro-noise however is a huge render hog under normal circumstances!

I am in no way implying that's in any way 'easy' but for any chance at physically realistic surfaces, it's quite necessary. It would be a great way to solve both the spec issues and the reflection blur issues in one fell swoop!

Tobian
03-24-2011, 02:38 PM
NB: Jrandom. The sampling patterns of the default LW reflection blur, the reflection blur shader, the anisotropic reflections shader, and the DP reflections shader all produce massively different appearances at different values, so it's not as simple as that! :) In the first example where I used DP's reflection I had to use more than double the value to get the same disk diffusion size.

jrandom
03-24-2011, 03:24 PM
I've only mucked about with the reflection blur in the Material nodes, but I found it interesting that I had to feed the roughness value through a Gamma node to even get close to matching the behavior of blurry reflections (eg, a linear value didn't get me anywhere near the right amount).

Maybe different gamma values would work for matching the different reflection nodes?

("NB"? Darnit, I'm slipping behind on my forum slang. What does that mean?)

Tobian
03-24-2011, 07:47 PM
That's not forum slang, it's a little older than that :D

http://en.wikipedia.org/wiki/Nota_bene

Tobian
04-27-2011, 09:48 AM
For those interested, I stumbled upon my original source files for the reflection blur tutorial...

http://comby.star-fleet.org/ref_tut_arc.zip

hopefully now it can be restored back up at Light Wiki! :)

I also could do with updating it and giving it a more problematic scene to show the real differences in render times.

jrandom
04-27-2011, 10:18 AM
Tobian, I'm almost done with the initial version of my reflection blur optimization node plug in, but it follows my dual-material approach. Since I've got the basics figured out, would you like me to create another node that implements your reflection blur optimization strategy? If so, post the node network here that you most commonly use and I'll get it coded up so it'll be included in the first release.

Tobian
04-27-2011, 10:38 AM
When you say 'plug in' Do you mean you have coded something, or do you mean a cluster of nodes? I think your strategies include several of mine anyway, so I am not sure what 'else' you mean? :D

jrandom
04-27-2011, 11:52 AM
When you say 'plug in' Do you mean you have coded something, or do you mean a cluster of nodes?

Yep, it's a real actual plug-in. It's written in pure C (using the C99 standard) although I'll be switching to C++ once I figure out a good design strategy. I just finished the automatic input caching code last night (since some of the inputs to my optimizer are used more than once and I didn't want to trigger unnecessary node evaluations).

I'm going to be releasing the full source code of these nodes back to the community as well, so no one will have to worry about them being not supported in the future.

So, let's have it! What math/logic node networks do you find yourself constantly re-creating? Those are good targets for being nodified.



I think your strategies include several of mine anyway, so I am not sure what 'else' you mean?

My node requires the dual-material approach (a "full" material and a "simple" material), and that might not suit everyone's requirements. If you have a preferred approach, I can add that as a separate node so that both strategies will be available.

jrandom
04-27-2011, 02:00 PM
This might help: my post in another thread (http://www.newtek.com/forums/showthread.php?t=118850) with a screenshot of the node in action (http://www.newtek.com/forums/showpost.php?p=1134661&postcount=6).

I'm pretty sure that differs from how you structure your own optimization setup. :)

Tobian
04-27-2011, 04:03 PM
Ok moving the discussion over to that thread! :)

This could be very interesting! :)