PDA

View Full Version : Reflection Blur Optimization Part II + New Plug-in



jrandom
05-29-2011, 08:10 PM
It's done! Finally! The first version of my Optimizer plug-in is ready for both Mac and Windows!

Woo!

http://rainybrain.org/Assets/Lightwave/Plugins/Standard-Optimizer.png

What it's for: This node replaces the crazy-mad node networks some of us have been using to facilitate faster rendering when using blurry reflections. There have been some rather long discussions on the topic, and I took the best of those ideas along with some of my own and rolled them into a single plug-in.

What I'm looking for: If this plug-in doesn't fit your workflow, or if you have any logic/math node networks you find yourself constantly creating, let me know what you use and I can add them as new node types in this plug-in.

Kind of important disclaimer: The Windows version has not been thoroughly tested, and the 32-bit versions for both platforms haven't been tested at all. Also, I've only tried these in Lightwave 10.0 and have no idea if they'll run in 9.6 (since I don't currently have that version installed).

Let me know if y'all run into any problems, and I'll do what I can to resolve them. I'm very curious about Lightwave 9.6 support (since a lot of people still use that) and if it doesn't run then I'll see what I can do about getting that version supported.

The attached .zip file includes MacUB, Win32, and Win64 versions of the plug-in, along with documentation and source code.

May the forks be with you.

Edit: Public download page (http://rainybrain.org/?page_id=152) is now up. New versions will always be available at that page.

Edit #2: Rebuilt the Win32 version using the proper linker settings for 32-bit plugins. Also renamed the Windows plug-ins to end in '.p' for autoscan compatibility.

funk
05-29-2011, 10:47 PM
Im just trying to install it on 10.1 win32 and get "Cannot load external plugin file"

jrandom
05-29-2011, 10:49 PM
Im just trying to install it on 10.1 win32 and get "Cannot load external plugin file"

Drat. Any way to try the 64-bit version?

Edit: What version of Windows are you running? Can you run Task Manager after starting Layout and verify it's running in 32-bit?

jrandom
05-29-2011, 11:13 PM
I just tried loading the 32-bit .dll in 9.6 (found it installed in my copy of VMWare; I had forgotten) and that loads just fine, but you can't add an instance because the LWNODECLASS_VERSION in the 10.0 SDK must have been updated.

I don't do anything fancy, so I still might be able to get it to run in 9.6 if I add in a check for the previous value of LWNODECLASS_VERSION. Anyone happen to know what that is off the top of their head? :P

funk
05-29-2011, 11:16 PM
OK it installs in lw10.0 x64, but you need to change the file extensions to ".p" if you want it auto-scanned

First thing I noticed is, it's way too big :) You really need to abbreviate the text

Some ideas:
Full Material > Hi Mat
Simple Material > Lo Mat

eg.
Full Material Reflection Sample = Hi Mat Ref Samples


Now I have to understand what it all does

funk
05-29-2011, 11:18 PM
Drat. Any way to try the 64-bit version?

Edit: What version of Windows are you running? Can you run Task Manager after starting Layout and verify it's running in 32-bit?

Running win7 64 as listed in my sig. I use 32 bit lightwave though because too many plugins arent 64 bit yet. I only have lw 10.0 x64 installed for testing

funk
05-29-2011, 11:23 PM
In lw10.0 32bit, the error is "Plugins were not found or could not be added"

jrandom
05-29-2011, 11:24 PM
OK it installs in lw10.0 x64, but you need to change the file extensions to ".p" if you want it auto-scanned

Oh! Right! I'll get that taken care of in the next release. Thanks for pointing that out.

Good to know it at least run in 64-bit 10.0. I also successfully got it to load in 32-bit 9.6 (although I can't add a new instance of the node in the node editor due to an internal LW API versioning mismatch -- might be work-aroundable; don't know yet). Weird that it won't load in 32-bit 10.1.



First thing I noticed is, it's way too big :) You really need to abbreviate the text

I run at 2560x1440 so that might be why it doesn't look large to me. :) I've been trying to shorten the names from the very start, but I have issues with short, confusing names thanks to years of working on bad programs with obfuscated variables. Because of that, I tend to err on the side of "Wordier but Obvious". I'll keep hacking away at it.



Now I have to understand what it all does

Let me know if any parts of the documentation are unclear. Good documentation is crucial and I'm a bit of a newbie at writing them. I've already noticed that I never specified in the docs where the plug-in appears in the node list.

funk
05-29-2011, 11:28 PM
I'll just wait till you get the 32 bit version working because I just dont have all my plugins set up in 64 bit

jrandom
05-29-2011, 11:52 PM
I'll just wait till you get the 32 bit version working because I just dont have all my plugins set up in 64 bit

I may have had some problems with my linker settings for the 32-bit Windows build. Can you give the attached plug-in file a try and let me know if it works?

funk
05-30-2011, 12:05 AM
Yes this one works :) Now to play around

jrandom
05-30-2011, 12:06 AM
Yes this one works :) Now to play around

YAAAAYY!! I'll get this new version repackaged and released. Thank you!

funk
05-30-2011, 12:39 AM
I've only done some quick tests.

At first I was getting longer render times, until I reduced the Max samples to 8 (matching the delta node)

I'm also getting too much of a difference in output. Its almost like shadow cast from the surface become much weaker.

It's probably user error though.

Edit: uploaded a quick example... look at the floor shadow

jrandom
05-30-2011, 12:57 AM
The trick is to find the right balance of when to have the Optimizer switch over to the Simple material. I've been testing using a Worst Case Scenario for blurry reflections (indoor scene that causes a lot of recursion in the reflections), and that's where this node really shines (compared to using regular blurry reflections).

It's not going to beat out shortcut materials that don't use blurry reflections at all, but it's not intended for those.

Getting the Max Reflection Samples set correctly does take some experimenting, and it's going to be different for each material. It defaults to 64 because that's what it takes to get a smooth render using an AA setting of 8 when the material has high Specularity and Roughness settings and is used in an indoor scene. As always, use the lowest number of samples you can get away with.

Shadow issue... not entirely sure what you're referring to here. It might have something to do with the Simple Materials reflection stand-in (a white diffuse color). That can be tricky to work with if you have highly-reflective surfaces. The reason its there is that when I don't use white diffuse as a reflection stand-in (and just shut off reflections), the resulting GI is too dark. Hm... Maybe bump up the GI SM Switch from '-1' to '0'?

Also, if you have any node networks you use for material optimization that fit your workflow better and would be handy to have in a single node, post the node network here along with a description of what it's for, and I'll code it up and add it to the plug-in.

Sad, sad limitation: There is no way for me to take an existing Lightwave node and stuff it into another node. That would be handy, but is (as far as I know) impossible. This restricts me to math/logic nodes and any custom equations you can come up with.

(I originally wanted the Delta, Conductor, and Standard Material nodes to be contained within the Optimizer so you'd only have a single material output to deal with. Ah well.)

jrandom
05-30-2011, 01:04 AM
Edit: uploaded a quick example

Huh. Interesting. I'll have to make some better test scenes and experiment further.

If there's not a lot of reflection recursion in the scene then you probably won't notice much of a speed-up, and it would probably be better to skip the Optimizer. If you got a lot of reflection recursion, and/or it's killing your GI times, then the Optimizer could come in handy.

... yeah, I'm betting the lighter shadows are from my reflection stand-in attempt. I'll keep playing around with that code and will remove it if it's deemed too much of a problem.

Edit: Are you using the node network setup as shown in the example .nodes file included in the Nodes Examples folder?

funk
05-30-2011, 01:17 AM
Yes even importing your node samples I see the same problem with shadows.

Look in tight areas between cushions in this example. The tight shadows vanish

Edit: In the 3 example I have all surfaces of the couch using your optimizer and all shadows vanish

jrandom
05-30-2011, 01:21 AM
Yikes! That is pretty bad shadowing, although I'm not sure exactly which part of my plug-in is causing the problem. Are you able to post a demo scene here that illustrates the worst problems you're having?

I can start debugging the logic and techniques used in the optimizer against that scene to help fix these things. No point in having a plug-in if it's not useful. :)

funk
05-30-2011, 01:22 AM
I just added a 3rd example above... It seems no shadows are cast at all

Maybe its because that scene uses no lights. Its all lit by a HDRI image

jrandom
05-30-2011, 01:23 AM
I just added a 3rd example above... It seems no shadows are cast at all

Whoa, that is really strange. I've not observed this issue on my own system, so if you could upload an example scene for me to debug against, that would be super helpful.

funk
05-30-2011, 01:34 AM
heres a sample scene... all lit by white background using gi

First image is with no optimizer

Second image is with optimizer (go into surface obj1 and connect it)

jrandom
05-30-2011, 01:51 AM
I've narrowed the problem down to the Material Switch, believe it or not! If I use the Full Material for both inputs on the Switch, there's no shadow, but if I plug the Full Material into the Surface directly, the shadow reappears. There should be no difference between the two!

I think we just found a bug in LW10's Material Switch node.

If you replace the Material Switch with a Material Mixer, plug the Full Material into A and the Simple Material into B, and Use Simple Material into Alpha, you get your shadow back.

So... where do we report the Material Switch bug?

Edit: I'll get my sample .nodes files and documentation updated to specify the Material Mixer instead of Material Switch in the meantime.

funk
05-30-2011, 02:01 AM
Report it to [email protected]

jrandom
05-30-2011, 02:10 AM
Drat, I'm too late to edit the primary post with the new version.

Attached to this post is the updated .zip file with the .nodes files altered to use Material Mixer instead of Material Switch. The documentation has also been updated.

jrandom
05-30-2011, 02:17 AM
Report it to [email protected]

The bug has been reported to that e-mail address. Thanks for all your help in giving this plug-in its initial shake-down.

funk
05-30-2011, 02:30 AM
I reported it too. Well, everything is working much nicer with the material mixer now. Nice job jrandom. On my simple tests I'm seeing 2-3 second speed up. I have to really test it out on more complex stuff though.

Now... make it smaller! :)

funk
05-30-2011, 02:35 AM
I just tried it in lw9.6 too and I see the error about wrong version. I had that same problem with one of the dpkit plugins and reported it to Denis. He fixed it very quickly, so maybe he can give you some hints?

Tobian
05-30-2011, 05:52 AM
I think the material switcher bug has been around for a long time actually!

If I can think of a test scene for this I will have a play with it. Now I'm hoping you can try out the one I suggested :D

funk
05-30-2011, 07:37 AM
I got a reply from newtek. The switch node will work if you enable "use transparency" in the GI tab

Yes... its odd... because we arent using transparency anywhere, but it works

I still consider it a bug though. Using transparency on more complex scenes will just increase render time for no good reason

Tobian
05-30-2011, 07:43 AM
Yes, unless you do what I do and separate all objects with transparency and tell them to be ignored by radiosity :) But yes, it's still a 'bug' as it's not transparency, it's material mixing!

jrandom
05-30-2011, 12:29 PM
I've just added Lightwave 9.6 support (seems to render okay). Any 9.6 users want to give it a spin?

jrandom
05-30-2011, 12:45 PM
Now I'm hoping you can try out the one I suggested :D

That one's next in line! Although... I was hoping to get some suggestions from other people so I could see if there was any commonality between them and yours (why build multiple nodes if they can easily exist as one node?). If I don't get any other node network suggestions then I'll go ahead and dive into the one you sent me as it's own stand-alone node.

What would I call it? "Tobian Optimizer"? I'm not very good at coming up with names.

torturebori007
05-30-2011, 01:05 PM
That one's next in line! Although... I was hoping to get some suggestions from other people so I could see if there was any commonality between them and yours (why build multiple nodes if they can easily exist as one node?). If I don't get any other node network suggestions then I'll go ahead and dive into the one you sent me as it's own stand-alone node.

What would I call it? "Tobian Optimizer"? I'm not very good at coming up with names.

Quick question, or wait first thanks for this and your time.

Any chance this will work on Mac lightwave x64?

jrandom
05-30-2011, 01:12 PM
Any chance this will work on Mac lightwave x64?

Absolutely! That's what I develop and test with. :)

torturebori007
05-30-2011, 01:25 PM
WOW. I have to say, that this is the very first plug in I have EVER gotten. Because I cannot find them for mac x64. Thank you, thank you.

I will load this once I get home and will give you feed back. Not that you need it since you tested it on a mac. WOW!!!

Thanks again. I have done all my work plug in free, for the longest time.

jrandom
05-30-2011, 01:39 PM
I have done all my work plug in free, for the longest time.

I became a bit gun-shy about plug-ins as well. Some never made the jump to 64-bit, some don't work as well on Mac, some vanish for no apparent reason.

Kinda scary if one's work relies on those plug-ins. That's why I include the source code with each release -- if I fall off the face of the earth I don't want to negatively impact anyone else's work. :)

torturebori007
05-30-2011, 01:47 PM
I became a bit gun-shy about plug-ins as well. Some never made the jump to 64-bit, some don't work as well on Mac, some vanish for no apparent reason.

Kinda scary if one's work relies on those plug-ins. That's why I include the source code with each release -- if I fall off the face of the earth I don't want to negatively impact anyone else's work. :)



Thats great thinking....I appreciate it big time.

COBRASoft
08-15-2011, 08:29 AM
Bump :)!

Any progress on this plugin?

jrandom
08-15-2011, 10:51 PM
Sadly, no. Been super busy with work, and with some other projects (including my first attempt at a live-action feature-length movie).

If you know any other C coders, you can always throw the source code at them and see if they can bend it to your will. I did my best to keep the code sane and easy to understand.

lardbros
08-18-2011, 06:24 AM
I became a bit gun-shy about plug-ins as well. Some never made the jump to 64-bit, some don't work as well on Mac, some vanish for no apparent reason.

Kinda scary if one's work relies on those plug-ins. That's why I include the source code with each release -- if I fall off the face of the earth I don't want to negatively impact anyone else's work. :)

Brilliant policy, and very refreshing! :D

Thanks very much for the hard work... I wish it was easier to create these node networks within LW Node Editor itself... it's a shame we can't create them ourselves (with no programming ability). :D

erikals
10-25-2011, 06:27 PM
I've just added Lightwave 9.6 support (seems to render okay). Any 9.6 users want to give it a spin?

Yep!

so far so good :]

(halved my rendertimes on some scenes, still have to test more though...)

jrandom
10-25-2011, 11:23 PM
Yay!

I've been hellishly busy with both my day job and trying to get everything lined up for my first feature film so I haven't touched Lightwave in months. I've got the 10.1 install files just sitting there, waiting.

Not enough hours in the day!

erikals
10-26-2011, 05:32 AM
... I've got the 10.1 install files just sitting there, waiting...

hehe \:] me too,...

3dworks
06-14-2012, 04:48 PM
is this still working with LW11 or is there any problem when using the new unified sampling?

cheers

markus

jrandom
06-14-2012, 05:06 PM
I have no idea, as I haven't even had time to try it out yet on LW11. In theory it should just work -- it's just a utility node used to control the settings of other nodes; nothing fancy.

jrandom
06-14-2012, 05:06 PM
Also, I kinda broke my cross-platform dev setup and haven't reconstructed it yet. (That's what I get for trying to get CLANG w/ C++11 support working on Snow Leopard...)

zapper1998
06-14-2012, 06:21 PM
Have not Put the 10.1 LW Files back on my main drive..

Will try this weekend...

And will try it out this weekend on LW 11.0.1..

Looks cool and sounds reallly cool....

Thanks
Mike