DP Kit - Shadows node & VPR issues

[dpont]

Ok so we have a proof,
I have to test other scenes or different shading
until it crashes for me...

Denis.

[dpont]

Still no clue, just my thoughts,
seems that the only difference between your config and mine
is multithreading, I wonder if you can test this scene
with VPR on a single thread, but don't know if it uses the
same Render settings than F9 render.

Denis.

[D-Lab]

Hi,

It is a multi-core problem but the multithreading setting in the render panel has no effect on this, I had to activate only 1 proc in the msconfig util and reboot.

Now, with only 1 core, no crash...

Tchao

[adk]

Hi,

It is a multi-core problem but the multithreading setting in the render panel has no effect on this, I had to activate only 1 proc in the msconfig util and reboot.

Now, with only 1 core, no crash...

Tchao

Yup that's worked for me too set processor affinity to one core and it now works perfectly with VPR. Cheers a bunch for looking into this Denis & D-Lab. Much appreciated.

[dpont]

Yup that's worked for me too set processor affinity to one core and it now works perfectly with VPR...

The code of Shadows node is safe for multithreading
considering its evaluation for normal rendering,
which is the imperative statement for shaders,
so this should be reported to NT.

Denis.

[adk]

I'll FBugz this and hope that they can replicate it. If it makes VPR even more stable (it's been pretty solid for me) then I'm all for that. Cheers Denis.

[grangerfx]

The code of Shadows node is safe for multithreading
considering its evaluation for normal rendering,
which is the imperative statement for shaders,
so this should be reported to NT.

Denis.

This crash happens when the node editor sends a NewTime call to your shader while VPR is active. VPR is calling your shader from other threads at the time. You probably change some internal values during NewTime which are being used by the VPR threads. This is really a bug in Layout since we should not be trying to do a preview shade at the same time as we are rendering and that will be a pretty big feature to implement in a future release.

For now you may be able to work around this crash by handling calls to your shader's NewTime function differently such that critical data needed by the rendering threads is not modified in such a way that it causes a crash.

-Mark Granger

[Sensei]

Denis - are you null checking the all function calls inside of LWNodalAccess such as nodalaccess->illuminate() etc. etc.. ?

Sometimes they are NULL..

[Lightwolf]

This crash happens when the node editor sends a NewTime call to your shader while VPR is active. VPR is calling your shader from other threads at the time. You probably change some internal values during NewTime which are being used by the VPR threads. This is really a bug in Layout since we should not be trying to do a preview shade at the same time as we are rendering and that will be a pretty big feature to implement in a future release.

For now you may be able to work around this crash by handling calls to your shader's NewTime function differently such that critical data needed by the rendering threads is not modified in such a way that it causes a crash.

-Mark Granger

It would also help a lot if VPR evaluations would be flagged (shaders, nodes, procedurals). Since a more careful evaluation of data pre-generated during NewTime might require more mutex locks, this does in turn impact performance (heavily in some cases).
However, that is (currently) only required if VPR is evaluating the node/shader/procedural - a flag would allow for more streamlined code for the production renderer.

Cheers,
Mike

[Sensei]

When LW v11.0.1 is opening Surface Editor it is executing functions:

node->Init( LWINIT_PREVIEW );
node->NewTime();
node->Evaluate(); nodalaccess->flags & LWRT_PREVIEW is true..
[optionally] node->Cleanup();

when there is also Node Editor with your node, it's called again in above order.

When LW is rendering F9/F10/VPR it is executing functions:
node->Init( LWINIT_RENDER );
node->NewTime();
node->Evaluate(); nodalaccess->flags & LWRT_PREVIEW is false..
node->Cleanup();

So, to not have problems, you just have to have 2 sets of initializations.
Different for previews, different for renders. And in Evaluate() use flag to check which one to use. And they won't interfere each other.

Like f.e.

int index = 0;
if( mode == LWINIT_PREVIEW ) index = 0;
else if( mode == LWINIT_RENDER ) index = 1;

data->Data[ index ] = [....]

then in Evaluate():

int index = 0;
if( nodalaccess->flags & LWRT_PREVIEW ) index = 0;
else index = 1;

using data->Data[ index ]

NewFrame() will initialize/free memory related to preview data->Data[ 0 ] and won't kill stuff used by full render..

The quickest possible thing to do is just to do nothing (return immediately from Init/NewTime/Cleanup/Evaluate) when mode == LWINIT_PREVIEW and ( flags & LWRT_PREVIEW ) != 0. It should immediately stop crashing.

[Sensei]

To Init() add

data->Mode = mode;
if( data->Mode != LWINIT_RENDER ) return;
[...]

To NewTime()/Cleanup() add

if( data->Mode != LWINIT_RENDER ) return;
[...]

To Evaluate() add

if( nodalaccess->flags & LWRT_PREVIEW ) return;
[...]

Should stop crashing if grangerfx correctly identified source.

[Lightwolf]

So, to not have problems, you just have to have 2 sets of initializations.
Different for previews, different for renders. And in Evaluate() use flag to check which one to use. And they won't interfere each other.

Try this:
Surface and node editor open, showing a preview, VPR rendering as well.
Scrub the timeline. Press F9 for extra fun (a little less though).

Having said that, most of these issue occur with FPrime as well.

Cheers,
Mike

[Sensei]

Should not you sleep?

All of this is because it was not designed to work in interactive environment. And there should be just renderer index, and renderer pointer (this was added in some LW v9.x way too late and it's only available in Evaluate() - not enough). Send to any rendering function. Init/NewTime/Cleanup don't get them.. Surface Editor would have different renderer index, Node Editor different, VPR different, full render yet another. Plugin would have array of data, with 0 ... renderer_count - 1 entries. Each entry modified and read by different renderer.

[Lightwolf]

Should not you sleep?

I tried to but that didn't work out...

All of this is because it was not designed to work in interactive environment. And there should be just renderer index, and renderer pointer (this was added in some LW v9.x way too late and it's only available in Evaluate() - not enough). Send to any rendering function. Init/NewTime/Cleanup don't get them.. Surface Editor would have different renderer index, Node Editor different, VPR different, full render yet another. Plugin would have array of data, with 0 ... renderer_count - 1 entries. Each entry modified and read by different renderer.

Indeed, a render index would solve the problem elegantly as well.
The downside is duplicate memory use of big memory structures need to be handled in the respective calls... as they'd likely need to be duplicated.

Cheers,
Mike

[Sensei]

I tried to but that didn't work out...

I am going to play Counter-Strike, you can join me killing police CT

The downside is duplicate memory use of big memory structures need to be handled in the respective calls... as they'd likely need to be duplicated.

There will be called Cleanup() (with renderer index), and memory will be freed.. So, I don't see that as issue.

If node really cares about memory, it can not use renderer index, and instead use mutex as last resort.

I prefer speed of rendering than memory usage.