View Full Version : Rendering of self shadow appears faceted

Dan Ritchie
04-21-2015, 02:42 PM

This thread is aimed at Lightwave group, and hopefully will help solve the issue, but others may benefit. Below, a discussion on the problem:



In the early node example i showed, rendering self shadow should have had the light direction vector reversed, causing the jagged end to be on the wrong side of the shadow. This was fixed later on in the video.

The example where I blended the surface normal and light vector to get a bent normal, the result should have been normalized, which I neglected, but at 2% blend, I don't think it colored the results much.

04-21-2015, 08:55 PM
Attach scene to play.

Are you using sub-patches?
This looks kinda like renderer used different geometry to do shadow calculation (before freezing - thus low poly count), than normally is used (frozen).
If you have sub-patches try increasing Sub-Patch Level (both Display one & render!), because VPR uses Display one!

I suggest freezing sub-patches permanently to triangles in Modeler, to see whether there will be difference in shadow look.

If you're not using sub-patches, then use them..

Without sub-patches:

With sub-patches:

Huge difference.

Dan Ritchie
04-22-2015, 11:21 AM
It's probably confusing that at one point in my video, I say "Good bye," but that's just the first part of the video. There's more to it, and subdivsion, ie, actually solving the problem of the surface itself, is the first solution I mentioned. I'll have to re-edit the video, I think.

So yeah, if you don't want to watch it, I presented 5 possible solutions

1) Solve the surface problem, ei, pixel level subdivision. (nicely available right now, but not always practical)
2) surface offset (lightwaves solution, but I think based on a wrong conclusion)
3) Bent normal solution. bend the shadow casting ray slightly toward the surface normal (by 2%-4%) (works and simple and fast to implement, but doesn't completely solve problem and may cause slight inaccuracy. Still, pretty good!)
4) bend the shadow casting ray by the difference between the faceted and smoothed normal. (produces superior shadows, but implementation is theoretical)
5) Determine if shadow is true shadow. (somehow) testing if intersecting with same polygon that emitted it, etc.

Dan Ritchie
04-22-2015, 12:54 PM
After some googling, I've found that the formal name for this error is the "terminator" effect or problem.

Dan Ritchie
04-22-2015, 01:46 PM
This page discusses the terminator problem in some detail, with possible solution


If I read it right, they made the shadow offset (Like Lightwave's shadow offset) a parameter of the surface instead of a global setting. The parameter is then interpolated in a way similar to the way normals are interpolated over a surface. Seems to make sense.

04-22-2015, 03:26 PM
Don't know if it'll help in your situation, but when I've run into shadow terminator problems in the past, I've switched to a Spotlight with Shadow Map shadows, and bumped the Shadow Map Size way up (to 4096, 8192 or even 16384). It also helps to move the Spotlight quite a distance from the object being lit and decrease the Spotlight Cone Angle to only contain the object and set the Spotlight Soft Edge Angle to the same small angle (e.g. 0.5 degrees).

Here's a sample scene containing both a Distant light (active in Scene Editor) and a Spotlight (inactive in Scene Editor), and screen shots of the difference in shadow smoothness.


04-23-2015, 02:44 AM
I guess this work around (softening the shadows) would work if you changed your distant lights to dome lights with an angle of 1.

Dan Ritchie
04-23-2015, 03:31 PM
I guess this work around (softening the shadows) would work if you changed your distant lights to dome lights with an angle of 1.

In my experience, soft shadows do not solve the terminator problem.

Dan Ritchie
04-23-2015, 05:53 PM
How they tackled it in Blender


04-23-2015, 11:41 PM
Great work Dan and yes I've encountered that problem in past many times so it would be nice to get "automatic" solution by software one day :).