PDA

View Full Version : Blocky, polygon-edged shadows?



Scott Gammans
10-25-2005, 08:44 PM
I have tried everything I can think of to get rid of a nagging problem I'm having with blocky, polygon-edged shadows on my Enterprise model.

Here is a closeup which shows the problem:

http://www.scottgammans.net/ussenterprise/ussenterprise_051025b.jpg
http://www.scottgammans.net/ussenterprise/ussenterprise_051025a.jpg

I have gone through a whole litany of suggestions for getting rid of this problem, and nothing seems to work:

At first I thought it might be some weird texturing problem, so I removed all of the textures, leaving only basic surface settings for diffuse, specular and smoothing. The problem was still there. Then I thought the luxidome globe of 80 shadow-mapped spotlights (for fake radiosity) that I was using in my scene might be causing the problem, so I removed them--and the problem was still there. Someone suggested turning off "Extra Ray Trace Optimization". Didn't work. The same person suggested upping the ray recursion limit to 24 (the highest setting). Problem still there. Single thread? Check. Problem still there. Shading noise reduction? On or off, problem still there. Polygons sharing vertices? No. Non-planar polygons? None. One or two-point polys? No. Little green men? Evil spirits? Bad juju? No, maybe, and most definitely. :(

The only thing that works is subpatching the model, but that's not a realistic solution. For one thing, subpatching would cause memory use to skyrocket, even if I set patch divisions to 1. Considering that LightWave already GPFs when I try to render at anything above 6400x4800 or so, subpatching would only make my memory problems even worse. And besides, the model has lots of ngons which can't be subpatched... I could spend weeks--months!--modifying the model so that subpatching would work.

So what's the solution here? Or am I up you-know-what creek without a paddle?

Here is a simple version of the model and scene file (http://www.scottgammans.net/ussenterprise/polyshadows.zip) that illustrate the problem. I've stripped out everything--textures, extra lights, everything extraneous--and the problem can still be reproduced with even this simple version of the model.

I would gratefully welcome any suggestions that don't involve subpatching the model. Thanks.

Scott Gammans
10-26-2005, 12:04 AM
Update (thanks to DaveK @ lwg3d.org for this!):

Apparently this has something to do with the way that LightWave handles specularity. If you turn off "Affect Specular" on the light source that is casting the block-edged raytraced shadow, the problem disappears (and so does the model because it becomes so dark!). Same thing with the surface definition itself: if you dial specularity down to 0%, the blocky-edged shadows blur into nothingness.

Of course, this isn't a solution--I don't want to change the appearance of my model to work around what is looking more and more like a bug in LightWave's raytraced shadows. But it is an additional clue to what is going on.

SplineGod
10-26-2005, 02:22 AM
The only time I can recall seeing this is when Ive used a shadow map on one of my spot lights. I fixed it by reducing the cone angle or increasing the shadow map resolution.

Darth Mole
10-26-2005, 03:12 AM
Massively annoying, I admit - but surely a five-minute Photoshop job?

Scott Gammans
10-26-2005, 05:25 AM
The only time I can recall seeing this is when Ive used a shadow map on one of my spot lights. I fixed it by reducing the cone angle or increasing the shadow map resolution.
Although the shadows look like the artifacts produced by low-detail shadow-mapped spotlights, if you open and run the attached scene you will see that these are raytraced spotlights.


Massively annoying, I admit - but surely a five-minute Photoshop job?
Perhaps on a simple, undetailed model like the one in the attached zipfile. The polygon-edge shadow artifacts on the full-detail model are much more complex, though, and are not confined to just the circled area in the closeup above. And besides, why should I have to retouch the shadows that LightWave produces? That's LightWave's job! If this is a bug in LightWave, I want it fixed, not swept under the rug.

Lightwolf
10-26-2005, 05:35 AM
What scale is your mesh and how far from the origin (i.e. 0,0,0 in XYZ) is it?

Cheers,
Mike

Scott Gammans
10-26-2005, 07:09 AM
What scale is your mesh and how far from the origin (i.e. 0,0,0 in XYZ) is it?

Cheers,
Mike
The Enterprise is centered on the origin and is built at 1:1 scale, so it's 288.646 meters long.

Lightwolf
10-26-2005, 07:10 AM
The Enterprise is centered on the origin and is built at 1:1 scale, so it's 288.646 meters long.
Well, o.k., scrap that idea I had then... (I've had issues with objects positioned very far away from the origin).

Cheers,
Mike

Scott Gammans
10-26-2005, 08:45 AM
Well, o.k., scrap that idea I had then... (I've had issues with objects positioned very far away from the origin).

Cheers,
Mike
Yeah, I had a feeling that was what you were getting at (someone else asked a similar question in a thread I posted on this subject at another BBS).

So, does anyone else have any ideas? Does anyone think it would be worthwhile to contact NewTek support? Is this a bug, or the expected behavior? If it's the latter, I still think NewTek should look into this--forcing the use of memory-hogging subpatches just to have smooth shadow lines seems a bit much.

gerry_g
10-26-2005, 09:07 AM
Why not consider changing your light type to solve the problem, the hard edges cast by the spotlights do'nt look to cool anyway, this model was lit with ray traced area lights, an expensive option I know, but far superior

CRUISING.jpg

Scott Gammans
10-26-2005, 09:15 AM
Basically because area lights suffer from the same problem--all that you're doing is masking it with a blurry shadow edge. And besides, area lights have problems of their own--the additional problems of greater memory and rendering time requirements. Even splitting the scene into pieces using "limited region" doesn't solve the memory problem above a certain point, and I'm trying to tackle a large resolution (~ 6400x4800).

I'm beginning to think that LightWave just isn't suited for large-scale rendering work.

gerry_g
10-26-2005, 09:38 AM
[QUOTE= area lights have problems of their own--the additional problems of greater memory and rendering time
rendering work.[/QUOTE]

Sorry I'm on a Mac, memory isn't such a big issue in OSX, and true although I'm animating it's only 1760 x 660 and my worst frame is just over 60 mins with my best being 2:54 mins. But for all this I donít think your out of options, you could turn off the spec channel for the light, then boost the image map in the Diffuse & Luminosity channels to compensate for instance.

Elmar Moelzer
10-26-2005, 11:25 AM
Hello!
Hmmm, havent had that problem yet.
Have you tried trippling the mesh?
You mentioned that SubDivisionSurfaces dont have the problem, well LW internally LW tripples the polygons after subdivision an prior to rendering.
So trippling might be a solution.
Forget it, does not help...
This is really odd...
CU
Elmar

MonroePoteet
10-26-2005, 11:52 AM
As a wild guess, I'd say you've exceeded a 32-bit floating point accuracy when calculating raytracing. The points on your model are very dense, meaning the distance between them is regularly in the 3rd or 4th decimal place, while the mantissa is large enough to preclude alternate floating-point representation with more digits of accuracy (if even available).

I'd guess that putting these values through the raytracing algorithm would require 5 or 6 (or more) digits of accuracy to provide a smooth shadow at that angle of incidence with that granularity of mesh. The 32-bit hardware will truncate the floating-point value, causing "noise" in the curve, which when magnified by the angle of incidence appears as a noisy shadow edge.

Buy a 64-bit machine, maybe? Or at least try rendering on one. Maybe somebody who's got a 64-bit machine could try it?

Just a guess, though...

mTp

Scott Gammans
10-26-2005, 12:49 PM
But for all this I donít think your out of options, you could turn off the spec channel for the light, then boost the image map in the Diffuse & Luminosity channels to compensate for instance.
Thanks... I tried that already and wasn't very pleased with the way things looked. Also, we're talking about tweaking dozens of surfaces on the actual model, vs. the four in the test model.


Hello!
Howdy! :D


Hmmm, havent had that problem yet.
Have you tried trippling the mesh?
Yep, no dice.


You mentioned that SubDivisionSurfaces dont have the problem, well LW internally LW tripples the polygons after subdivision an prior to rendering.
So trippling might be a solution.
Forget it, does not help...
This is really odd...
CU
Elmar
Yes, it is really odd, and very frustrating.


As a wild guess, I'd say you've exceeded a 32-bit floating point accuracy when calculating raytracing. The points on your model are very dense, meaning the distance between them is regularly in the 3rd or 4th decimal place, while the mantissa is large enough to preclude alternate floating-point representation with more digits of accuracy (if even available).

I'd guess that putting these values through the raytracing algorithm would require 5 or 6 (or more) digits of accuracy to provide a smooth shadow at that angle of incidence with that granularity of mesh. The 32-bit hardware will truncate the floating-point value, causing "noise" in the curve, which when magnified by the angle of incidence appears as a noisy shadow edge.

Buy a 64-bit machine, maybe? Or at least try rendering on one. Maybe somebody who's got a 64-bit machine could try it?

Just a guess, though...

mTp
Hmm, interesting theory! Unfortunately I haven't moved up to LW64 and 64-bit computing yet.

If any LW64 users are reading this, do a fella a favor--render this test scene (http://www.scottgammans.net/ussenterprise/polyshadows.zip) and see if the poly shadow artifacts still exist at 64 bits. Thanks!

Elmar Moelzer
10-26-2005, 12:55 PM
Hey!
Well, the problem gets nicely camouflaged if you add a little bit of motion to the object (e.g. in the y- axis) 0.5 to 1mm is enough. Then render with Motionblur on (which is what I always do, adaptive sampling is a big Nono anyway)
The shadow gets slightly blurry then and the problem nicely hidden.
As an alternative you could slightly move the light of course (or use the famous spinning light trick).
Hope that helps.
CU
Elmar

Scott Gammans
10-26-2005, 01:29 PM
Yep, you definitely can mask the problem with motionblur (or fuzzy shadow-mapped shadows), but I was kind of hoping for a more razor-sharp image.

Elmar Moelzer
10-26-2005, 01:34 PM
Thinking about it, I am starting to think that I have heard of that problem before.
I just never ran into it that drastically...
I think it is because I am mostly using soft shadows myself.
To hard shadows never look realistic, not even for space scenes.
The reason is that no surface is totally smooth ever.
However the bumpmaps in LW dont distort shadows, which is an old and everlasting feature- request of mine.
CU
Elmar

Scott Gammans
10-26-2005, 02:16 PM
Those aren't bump maps on the actual model... the radial gridlines on the saucer section are actual geometry.

And MonroePoteet, I told some of the experts over at lwg3d.org about your theory... trying to find someone who has LW64 to test it....

Elmar Moelzer
10-26-2005, 02:26 PM
Yupp, I know they are actual geometry, I was talking more in general, sorry...

Scott Gammans
10-27-2005, 06:54 AM
Well, at least now I know it's really a bug... the second major one that I've stumbled across during this project (the first being the dreaded "holes in shadows" bug with raytraced lights). Wes Chilton (aka "Nova Class"), a well-known LightWave guru, posted the following explanation for the phenomenon over at lwg3d.org:


This is a long standing bug in LW's surface shading code. It has to do with LW calculating the diffuse terminator at 0-degrees vs a variable solution like other renderers have. That means when you have raytraced shadows or even reflections (it happens with those sometimes too) that curve around a surface errors creep in when the diffuse terminator approaches 0-degrees to the camera.
So there you have it. At least Wes and some other folks here and at lwg3d.org were able to offer some workarounds.

SplineGod
10-27-2005, 03:48 PM
Another thing I found strange is that you have to set the surface to double sided to get the shadows to render close to proper. Turn off double sided and it looks worse. :)

Scott Gammans
10-27-2005, 04:02 PM
The surfaces are double-sided because of a different bug in LightWave--the "holes in shadows" bug with raytraced lights. I'm sure Larry is well aware of this bug, but for those of you who aren't familiar with it:

Under certain conditions, raytraced spotlights which cause an object to self-shadow cause "holes" in the shadows to appear which look like slits or pinpricks of light. Unlike the surface shading bug (which btw I have reported as an official bug in LW 8.5) this error has an acceptable workaround: make the surfaces double-sided. (You can also ameliorate the issue with shadow-mapped spotlights, which do not have the holes-in-shadows problem.)

Michaelb
10-28-2005, 07:25 AM
Well. for what it's worth, I'm thinking it's a surfacing issue. On the test scene I replaced the surfaces that look blocky and odd with some of the regular LW metal presets, and they worked fine if you remove the fill light, add more ambient light, and give the surface an actual object to reflect. I just used a luminous ball parented to the key light (excluded from the light too!), and the blocky looking part vanished. It looks like you might be pushing the specularity function beyond its ability to compensate. It cleared a lot of things up for me when I found out that specularity IS an approximation of reflection, just a faster way of doing it to save render time. Substitute some real reflection for specularity in there and I think you'll be fine.
(p.s. If you add too much ambient light, it might start to look "flat." If so, you might want to add some SG_Ambient_Occlusion to the surface to bring some shading detail back.)

Michaelb
10-28-2005, 12:55 PM
Oops. I forgot to say I noticed that blocky looking area as soon as I opened the test scene file. In the Shaded Solid OpenGL view it sure looked like a cast shadow from a light, but since my old video card doesn't support OpenGL shadows at all (!!!), I knew it would probably have to be a polygon shading problem. I'd be curious to know what version of LW you are running currently, and if anyone else with maybe LW 6 or 7 are seeing this too. I'm running LW 8.5 at the moment.

Michaelb
10-29-2005, 05:32 PM
I thought about it some more, and there also might be another way of looking at this that's easier to grasp. Let's figure the hull of the starship is a metal plate, painted a shiny or semi-gloss white. If you removed a single plate from the hull and brought it into your living room, it would look pretty normal under those lighting conditions. You would see a white paint (Color white, with high Diffuse), with fairly good reflections (high Specularity), and a smooth surface (high Gloss). The way your lower hull surface appears to be set up would duplicate a dull grey paint (Color white, but low Diffuse), totally reflective (100% Specularity) with a surface like rough sandpaper (low Gloss). I think if you adjust your surface settings and then change your lighting conditions to compensate (add in ambient light with SG_Ambient_Occlusion or Eki's Overcaster), you would get closer to the effect you are looking for.
That's just what I would do though. :)

toby
10-29-2005, 10:52 PM
The Enterprise is centered on the origin and is built at 1:1 scale, so it's 288.646 meters long.
Star Trek geek! Star Trek geek! :D

TOR
:king:
TNG
:twak:

If you've ever seen 'Trekkies', you'll know what I've been through the last 2 months, sitting next to the guy who starred in it. He's a Lightwave artist, and worked on 'tng' for a couple years. Really nice guy, but wow, bad jokes!

Actually this is not really a bug, not in the sense that a part of the code is broken. What's happening is that the detail of the surface is becoming exaggerated with the extreme glancing angle of the light source, the same thing that happens in real life.
Wes Chiltons' explaination merely states that LW's renderer doesn't have a 'variable solution' like other renders do, which is clearly added to work around this problem, a problem exists because we're trying to mimic billions of molecules with a few polygons. This is the same reason we need the other older cheat, Smoothing, which doesn't work with facades or extreme glancing angles.

The only things I can suggest is to increase the geoemtry's resolution, or soften the shadow slightly, or a little of both, or moving the light to a less glancing angle.

btw I had the same problem here:
http://vbulletin.newtek.com/showthread.php?t=40302
http://www.tobygaines.com/starfield.mov