PDA

View Full Version : smoothing not working



Mr Rid
07-20-2008, 04:28 PM
Can not figure out why smoothing is not working in certain areas with ray trace shadows. This is one object, default surface, one area light. Points are merged, polys aligned, non-planars tripled, nothing particularly weird about the mesh in the problem areas. Jaggy polys show up with single or double siding, or any smoothing threshold, or with other light type or quality setting, or any camera type. Shadow maps or Noise Reduction introduce their own brand of funk. Same in LW 9, 9.2, 9.3. Just will not render smooth.

Any thoughts how to cure?
9.3 Model+scene- 60868

60862

60863

60864

60865

prospector
07-20-2008, 05:33 PM
Hmmm, loading scene and says object is there but nothing showing.....
used perspective all way in and all way out, moved camera.....nothing

prospector
07-20-2008, 05:36 PM
nevermind,came up requestor and seen word surface and clicked no 8~
found it wanted object :hey:

prospector
07-20-2008, 05:46 PM
OK, something with light. Switched to distant and rendered...stuff gone
Swithched back to spot...rendered...stuff gone, looks fine now. still checking

Mr Rid
07-20-2008, 05:51 PM
The only way I can cure most of the problem is to tab it which puts the poly count over half mil8~ then smoothing of 2 with 5 iterations in Modeler. But I still get some little jaggies. The final scene is actually for a 4k render so it all shows up.

I just dont get why smoothing wont work where it really seems it should. Am suddenly made aware of the lack of control over this basic feature that is so integral to geometry rendering.

Mr Rid
07-20-2008, 08:08 PM
OK, something with light. Switched to distant and rendered...stuff gone
Swithched back to spot...rendered...stuff gone, looks fine now. still checking

Well a distant light is a different animal and is not going to render right (or most things photoreal). But I had the exact same problem with spot, point and linear lights. Ultimately it has to be rendered with an area.

bobakabob
07-20-2008, 08:31 PM
There look to be some deceptively sharp polys there... Turning off double sided and using an area light instead of spot improves things.

There are still some very small artefacts but these disappear if you increase the size of the area light (render attached).

Tabbing the object in Modeler also removes the tiny triangular edges if you're using the spotlight. Trouble is, because the object is tris, the subpatching doesn't make for a completely smooth surface on the model's rear.

I've never liked the combination of spotlight and ray trace in Lightwave and once had similar issues trying to render a simple subpatched globe! Shadow maps with fuzziness turned up to 10 work OK on this scene but area lights are infinitely better.

jameswillmott
07-20-2008, 08:56 PM
If you have 9.5, it fixes that problem easily. There is a Shadow Offset feature on the Objects tab to dial away those exact issues.

voriax
07-20-2008, 09:00 PM
Crazy, I've never encountered this problem before and not been able to figure it out. Even lighting without lights doesn't seem to fix the problem. The geometry is simply too jagged in some places to shade correctly.
I thought it only happened with hard shadow lights, but nope. Area based lights, geometry based lighting, image based lighting.. all crap!

voriax
07-20-2008, 09:03 PM
If you have 9.5, it fixes that problem easily. There is a Shadow Offset feature on the Objects tab to dial away those exact issues.

That was the first thing I tried, but you still end up with jagged edges in places. If you offset it far enough it does fix it, but wont your shadows become incorrect?

jameswillmott
07-20-2008, 09:09 PM
That was the first thing I tried, but you still end up with jagged edges in places. If you offset it far enough it does fix it, but wont your shadows become incorrect?

Yes they will, but for the most part you won't notice that they are wrong. ( Little details like the string that goes around her waist stop casting shadows if the value is too high for example )

You shouldn't need much of an offset to solve most shadow problems though.

voriax
07-20-2008, 09:14 PM
Using the chest angle, it does require quite an offset - between the breasts is the worst part by the looks of it.. You start to lose shadows before you get rid of all the jagged edges. I still think it's just a case of the geometry being too jaggy, and lightwave's inability to handle it.

prospector
07-20-2008, 10:15 PM
Well a distant light is a different animal and is not going to render right (or most things photoreal).
no, I mean just changing light (no settings) and changing light back to spot got rid of it here...It's like the light setting isn't taking hold untill they are switched then switched back.
Tried again and switched to distant and right back to spot without rendering first and it fixed it.

jameswillmott
07-20-2008, 10:18 PM
Even values as low as 0.5mm offset worked OK for the scene that was supplied for me... are we both doing something differently?

Mr Rid
07-20-2008, 11:21 PM
no, I mean just changing light (no settings) and changing light back to spot got rid of it here...It's like the light setting isn't taking hold untill they are switched then switched back.
Tried again and switched to distant and right back to spot without rendering first and it fixed it.

Uh, dont know if I follow. You are changing the 'Light Type'? Has no effect here.

Mr Rid
07-21-2008, 01:30 AM
There look to be some deceptively sharp polys there... Turning off double sided and using an area light instead of spot improves things.

But they aren't sharp if you look at the model closely. They should not cause such apparent jaggies with smoothing on.



...if you increase the size of the area light (render attached).

Afraid that misses the point. This scene represents only the key 'sun' light in a more complex scene of 5 area lights, but the key needs to stay small. Area, linear, or spinney lights have to be used for realistic shadows. Sloppy shadow maps are right out for closeup geom.

toby
07-21-2008, 01:54 AM
I don't think that smoothing of the normals has any effect on raytracing, which just looks at geometry, so those polys get 0 light. I'd blame this on the subdivision, ray (shadow) offset, and lighting too. The polygons would probably be far less noticeable if the lighting didn't plunge into darkness.
But also, for the amount of curves on this model and for how smooth it should be, I don't see how it could be done well at this low res.

Mr Rid
07-21-2008, 03:14 AM
If you have 9.5, it fixes that problem easily. There is a Shadow Offset feature on the Objects tab to dial away those exact issues.

I just installed 9.5 but I cant use it for this since the final model uses BRDF which I see is broken again. Didnt mess with it but dont know that I trust whatever the new shadow fudger is. I just see that this should work as is. After doubling the polys and smoothing in Modeler this mesh is smooth as glass but I still get some sharp poly edges. Kinda hard to see in this 1k (the 4k will take overnite). Sumthin's screwy in the smoothing.

60887
60888

Mr Rid
07-21-2008, 03:36 AM
I don't think that smoothing of the normals has any effect on raytracing, which just looks at geometry, so those polys get 0 light. I'd blame this on the subdivision, ray (shadow) offset, and lighting too. The polygons would probably be far less noticeable if the lighting didn't plunge into darkness.
But also, for the amount of curves on this model and for how smooth it should be, I don't see how it could be done well at this low res.

Actually the problem is still quite apparent with fill light.
60889

At 4k it was shameless.

PhantomPhish
07-21-2008, 05:58 AM
I just installed 9.5 but I cant use it for this since the final model uses BRDF which I see is broken again. Didnt mess with it but dont know that I trust whatever the new shadow fudger is. I just see that this should work as is. After doubling the polys and smoothing in Modeler this mesh is smooth as glass but I still get some sharp poly edges. Kinda hard to see in this 1k (the 4k will take overnite). Sumthin's screwy in the smoothing.

60887
60888

The problem is, although it looks like the mesh is smooth, for the purposes of raytracing, it is still composed of hard angles, which will show up as rips in the shadows if the angle of the mesh to the light is just right. ( or just wrong, depending on your point of view ) Pity BRDF doesn't work with 9.5 because 9.5's shadow fudging works pretty well (http://www.newtek.com/forums/showthread.php?t=82712).

toby
07-21-2008, 09:22 PM
The problem is, although it looks like the mesh is smooth, for the purposes of raytracing, it is still composed of hard angles,

yea that's what I thought -


9.5's shadow fudging works pretty well (http://www.newtek.com/forums/showthread.php?t=82712).
This sounds to me just like renderman's (and others') raytrace offset, which must've had a default value until now. If it's set to 0, polygons will shadow themselves all over the place. The problem with turning it up is that you lose detail shadows, the first to go would probably be the straps on the bikini.

It occurs to me now that subdividing would only serve to make the artifact smaller, requiring an insane level of subD to get rid of it completely. So I think that a combo of subd, ray offset and a small area light could be the answer. This model is definitely too lo-res for 4k renders. You can see polygons in the silhouettes -

Mr Rid
07-23-2008, 06:33 PM
What doesnt make sense to me is why a cube with polys at 90 degree angles has better smoothing than the figure where the angle difference is far less.

same surface-
61006

61007

jameswillmott
07-23-2008, 06:39 PM
It's not smooth shading that's causing the rips, it's shadow casting. Or are you referring to something else?

Giacomo99
07-23-2008, 07:44 PM
The problem is, although it looks like the mesh is smooth, for the purposes of raytracing, it is still composed of hard angles, which will show up as rips in the shadows if the angle of the mesh to the light is just right. [/URL].

Phantom Phish hit the nail on the head. I don't know how the original model was built, but the places that are artifacting look pretty...funky to me. (Sorry, I can't think of a more descriptive word.) If the base mesh was a mostly-quad SubPatched form with detail applied as a displacement map, those artifacts would be easy to fix (if they occured at all.) But that model just doesn't have very clean poly flow at all, so any rendering of it is going to have artifacts--in fact, I think it's a tribute to Lightwave's capabilities that you're not getting more artifacts.

The only solution I can see is to rebuild the form using SubPatches so it will yield a cleaner render.

Edit: Also, turning off "Double Sided" in the surface characteristics might help. I don't have time right now to do a test render.

bobakabob
07-24-2008, 06:36 AM
Would getting rid of the tris then applying subpatching to various degrees help?

Matt
07-24-2008, 07:10 AM
I think what's not helping is the incredibly bad poly flow, cleaning up the areas as bobakabob said would help a lot.

archijam
07-24-2008, 08:17 AM
It's not a matter of 'tabbing' your whole model.

On a recent project I used meta-nurbs only on problem areas/joins, then weight mapped to the joins to the normal geometry to make it look seamless ... worked a treat.

4dartist
07-24-2008, 08:30 AM
I have been having the same issue on my terrains. Self shadowing in lightwave 9.3.1, in my opinion, is horrible. This was a thread I started a while back about the same issue, it was never resolved there but I posted some images showing what i was getting.

http://www.newtek.com/forums/showthread.php?t=85007

The only solution I found was to just increase the poly count to a number I was very uncomfortable with. Even then the problem was there, just a lot let noticeable.

We should make bug reports, and hopefully it will grab some attention.

Giacomo99
07-24-2008, 09:09 AM
Self shadowing in lightwave 9.3.1, in my opinion, is horrible.

Just out of curiosity, do other apps handle this sort of thing better? If so, how do they do it? Please advise.

4dartist
07-24-2008, 09:18 AM
Hum.. good point.. I say lightwave selfshadow is bad, but it may be the same as every other app, i really couldn't say.

All i know is the shadows are poor enough to cause the renders to be unsatisfactory. So the selfshadows aren't bad because other programs do it better, they are bad because they are unrealistic and undesirable.

would be interesting to see a model that really exposes the problem tested in a few apps and the results. Might help expose the issue.

Matt
07-24-2008, 09:19 AM
Anyone tried Mr. Rid's model in another package?

bobakabob
07-24-2008, 10:57 AM
Matt: Anyone tried Mr. Rid's model in another package?

It would also be interesting to see how the file rendered in FPrime using the spot + raytraced shadows.

prospector
07-24-2008, 01:01 PM
no it don't work there either

prospector
07-24-2008, 01:08 PM
Think I got it
Not getting ANY artifacts
lemme check all places marked above to be sure

prospector
07-24-2008, 01:17 PM
OK, went thru posted images and rendered same spots, clear as a bell.
There was a problem in modeler.
Did a merge points..nothing changes
redid merge points to default FIXED which is .0394 and removed 27,667 points
did unify polys and eliminated 22,634.
Renders fine

prospector
07-24-2008, 01:19 PM
Poly count goes from 81,326 to 49,627

should get 22,634 polys eliminated

Giacomo99
07-24-2008, 01:43 PM
Did a merge points..nothing changes
redid merge points to default FIXED which is .0394 and removed 27,667 points
did unify polys and eliminated 22,634.

Be careful. I tried to reproduce what you described above and it totally chewed up her bikini--created all sorts of 2-point polys and other undesirable junk. The clothing looks to be a separable surface, so I'd cut and paste it onto a separate layer before getting too enthusiastic with the Merge tool.

prospector
07-24-2008, 01:49 PM
interesting...lemme try that again then either its, body doing it OR bikini is doing it and casting strange shadows...BRB

prospector
07-24-2008, 01:58 PM
yer right...forget that above :thumbsdow
tho if I put just body on layer 2
dissolve layer 1 (suit)
and render just body I still get no pointed shadows...strange

back to checking

4dartist
07-24-2008, 02:00 PM
Think this is the same problem? If a problem at all i guess.. looks like it is to me.
http://www.4dartist.com/cgtalk/ugly.jpg

Here is the scene and object

scene (http://4dartist.com/cgtalk/shadows.lws)

object (http://4dartist.com/cgtalk/test.lwo)

All I did was make a box and jitter it in Y and then subdivide it (modeling tool, not sub-d's)

prospector
07-24-2008, 02:06 PM
OK, started from beginning and did what I might do if it was mine.

First thing
Merge TrigonX
just too many polys for me
Hit 'TAB' at level 1 (added no new polys, just smoothed out mesh)
left body on layer 2 and also did same for suit on layer 1
sent to Layout

dissolved suit (layer 1)
rendered body...perfect
dissolved body
undissolved suit
rendered out...perfect

undissolved all
rendered out....perfect :)

prospector
07-24-2008, 02:09 PM
here's what I got with yours using same way as above

4dartist
07-24-2008, 02:10 PM
i haven't tried the body model, but i opened modo and rendered the mesh i made and it renders much better.
Prospector, put the light at a low angle that would cast shadows
and turn on raytrace shadows if they aren't on.

http://4dartist.com/cgtalk/modo.jpg

prospector
07-24-2008, 02:10 PM
don't know what's going on with shadows but subdividing to level 1 adds no more polys but fixes them

maby problem should be bugged??

4dartist
07-24-2008, 02:13 PM
ya.. it basically masks the problem because it's on a much smaller scale. I'm totally reporting this as a bug because of what it looks like in modo. (which is exactly what I would expect the shadows to look like).

prospector
07-24-2008, 02:15 PM
OK did raytrace thing
got raytraced shadows but were smooth without pointy ends

4dartist
07-24-2008, 02:16 PM
which lw are you using? I used 9.3.1 UB

prospector
07-24-2008, 02:18 PM
got these

prospector
07-24-2008, 02:20 PM
I used 9.3.1 UB

not that
or 9.3.1

prospector
07-24-2008, 02:22 PM
(which is exactly what I would expect the shadows to look like).
They look like that in Modo?
I would expect them to look as I rendered...smooth...no points

prospector
07-24-2008, 02:23 PM
Still, there is something wrong as it shouldn't leave pointed shadows with such high polycount meshes even unsub'd ......I would think

4dartist
07-24-2008, 02:27 PM
I didn't turn the mesh into subdivision surfaces. If you just open the scene i uploaded and hit render, it gets those gaps in the shadows, like holes that light leaks into, but if you load the same mesh into modo, you get the shadows with no holes or oddness. I said it renders shadows like i expected because unsubdivided it has more boxy outlines that would show up in the shadows.

prospector
07-24-2008, 02:30 PM
ahh.
don't remember that ever happening in LW with such high poly count objects and surface smoothing on.
Could be me tho...getting old and hard to remember my name :D

prospector
07-24-2008, 02:58 PM
Nope it's a no-go there too
just reloaded scene and now shadows are REALLY grainy
:cursin::cursin:
still looking tho

prospector
07-24-2008, 03:02 PM
:screwy: WTF
did as I did before;
changed to distant light, NO RENDER NO SETTINGS CHANGE
just went BACK to spotlight and rendered' getting this;

prospector
07-24-2008, 03:05 PM
got this untill I just ticked light off and on
:compbeati:cursin:

prospector
07-24-2008, 03:26 PM
went to original model, still getting pointed shadows
hit 'tab' (with still trippled model (original)) at subdivision 1 (no extra geometry)
rendered fine
saved model
reloaded scene, replaced original model with new Tabbed one
rendered...GRAINY
ticked light off and on
rendered fine
got down to 2.5 on render level and ok..2.4 and below pointy shadows return

ARRRRRRRRrrrrrrrrrggggggggggggg

Mr Rid
07-24-2008, 07:58 PM
It's not smooth shading that's causing the rips, it's shadow casting. Or are you referring to something else?

Well its both. The underside of the cube (post 18) would normally be in shadow as in the 1st image, but if you exaggerate the smoothing to 100 then LW tries to smooth the underside with the left side facing the light and you have a smooth gradient with no sharp edge between two polys at 90 degrees! Thats an extreme example yet LW smoothing/shadows deal with it. Yet they totally give up on the figure where there is a MUCH shallower angle between the problem polys. Why isnt LW handling it? I dont see that modifying the geometry should be necessary.


Phantom Phish hit the nail on the head. I don't know how the original model was built, but the places that are artifacting look pretty...funky to me. (Sorry, I can't think of a more descriptive word.) If the base mesh was a mostly-quad SubPatched form with detail applied as a displacement map, those artifacts would be easy to fix (if they occured at all.) But that model just doesn't have very clean poly flow at all, so any rendering of it is going to have artifacts--in fact, I think it's a tribute to Lightwave's capabilities that you're not getting more artifacts.

The only solution I can see is to rebuild the form using SubPatches so it will yield a cleaner render.

Edit: Also, turning off "Double Sided" in the surface characteristics might help. I don't have time right now to do a test render.

Single-sided wont trace properly and doesnt fix the problem. Remaking the object should not be an option. We are often given models by a client that we dont have any choice about. But I see that LW shading should be handling such a shallow poly angle.


Would getting rid of the tris then applying subpatching to various degrees help?

The object originated as quads but were mostly non-planars that caused many errors, so they were tripled.

I keep looking at the cube example and dont except how error-fied the figure is comparison, especially after I tabbed the figure into a half million polys. Smoothing really ought to be handling it.

prospector
07-24-2008, 08:20 PM
tried everything and trick I could possably think of
Only way to get smooth shadows is to Sub'D the model to 1
then run render display to 2.5. (scene display can stay at 1 or even 0 for faster screen updates and faster workflow).
Don't know what else to check...need a programmer to look at it deeper.

prospector
07-24-2008, 08:25 PM
Why isnt LW handling it?
Like I said above, I don't remember LW doing this on such a poly heavy model


I dont see that modifying the geometry should be necessary.

It shouldn't need it. But only way I could get it to work was to Sub'D it to level 1 (no extra geometry added) well till rendertime at level 2.5 and above.

Very strange

toby
07-24-2008, 11:20 PM
I see that LW does not smooth shadows over polygons at all. If you do a render with smoothing on and one off, toggle between the two you'll see that the outline of the shadow stays exactly the same.

Maybe other apps do smooth shadows (mental ray doesn't), but judging by how much of these errors go away by turning off double-sided, self-shadowing polys has to be the biggest problem. There's also long, thin tri's in the cleavage area, those frequently cause problems, and some sharp corners caused by tripling of the bad poly flow.

Mr Rid
07-24-2008, 11:44 PM
... There's also long, thin tri's in the cleavage area, those frequently cause problems, and some sharp corners caused by tripling of the bad poly flow.

But if you look at the polys right between the breasts (little diamond pattern there) or in the armpit where the errors are in the first render, the tris are not long or thin at all.

I see this as a confused math bug at this point. And as mentioned, increasing the hell out of the poly count mostly shrinks the problem to a smaller level. And it still doesnt make sense when the cube works at such an extreme angle.

toby
07-25-2008, 12:31 AM
Most of those errors go away when you turn off double-sided.

And you can see here that smoothing makes no difference to shadows, only to shading.

dwburman
07-25-2008, 12:52 AM
Is this an animated project or a still? If it is a still, you could render a separate shadow pass, massage it in Photoshop and apply it to the model... heck, you could still do that with an animation, it'd just be a lot more work.

I agree that it doesn't look like it should be having those shadow problems.

bobakabob
07-25-2008, 11:50 AM
Got to be a bug because surely non planars should be cured by tripling polys as Mr Rid made the point right at the outset. Render both models here and it seems LW doesn't seem to like tris!! And Merging trigons on Mr Rid's model helps but there are still problems.

Switching off double sided, Merging Trigons and tabbing the object cures the problem.

4dartist's original ocean has loads of non planar quads so it's not surprising it didn't work. However, triple the polys on 4dartist's ocean and the same ugly shadows appear. Once the original is tabbed it renders fine.

Can anyone from Newtek advise?

Giacomo99
07-25-2008, 01:29 PM
Yes--I'd always assumed this was one of those all-app problems that had to be endured, but after hearing Modo's renderer doesn't have this problem, I think the guys at NewTek really need to get on it.