"F9 render on NRC renderfarm" script?

slacer

Member
NRC seems to translate the paths correctly. But if I use multiple platforms, I need to use UNC paths instead of mapped drive letters.
Since a remote share will be mounted into a local folder (/Volumes on a Mac), everything is local for Lightwave.
The smb://server/share syntax is only required at mount time, not during the rendering.

I will try to create an nrc file for the test as well.
 

Attachments

  • Bildschirmfoto 2021-06-21 um 20.56.44.png
    Bildschirmfoto 2021-06-21 um 20.56.44.png
    159.9 KB · Views: 30

slacer

Member
On my Mac the nrc file looks like this:

/Volumes/Dokumente/LWS/Scenes/dynamic_scenes.lws
/Volumes/Dokumente/LWS/Renders/dynamic
false
Before the render task is started, you can see different paths for Windows and Mac.
 

Attachments

  • Bildschirmfoto 2021-06-21 um 21.13.54.png
    Bildschirmfoto 2021-06-21 um 21.13.54.png
    259.8 KB · Views: 21
  • Bildschirmfoto 2021-06-21 um 21.28.18.png
    Bildschirmfoto 2021-06-21 um 21.28.18.png
    330.3 KB · Views: 26
Last edited:

vncnt

Well-known member
Getting there.

segments.jpg

[sneak preview in this image]

To do:
  • Send Full Frame Sequence (seemed logical to me to add this option) [ready]
  • Send This Frame Segment [ready]
  • Send This Sequence Segment
  • F9 Render Segments [ready]
  • Send Segmented Frame [ready]
  • Send Segmented Sequence [ready]
  • Build Fusion (9 and 17) Comp
  • Reset Segmentation [ready]
  • Additional Camera flags
  • Unix/macOS path compatibility
  • Additional remote scripts for keyboard shortcuts
  • Storage of segmentation parameters in LWS scene file
Also testing output files, and revitalizing the Comp generator!
 

vncnt

Well-known member
Export as Fusion9 works again.
Imports fine in Fusion17. I could start assemble rendering at once. Nothing needed to be tweaked manually.
Scene parameters for frame dimensions, startframe, endframe, stepsize, input/output paths have been implemented.
Even and odd segmentation values, segmentation values with inefficient coverage (non-codec-friendly).
It all seems to work.

Some hard-coded parameters to be implemented:
  • LightWave output image file extention
  • Fusion output image file extention
The *.nrc Task file still has the unique hex number but I had to ditch it for the *.lws scene file and the segment output directories.
Segment corrections and/or segment assembly, without available task #, would have become overly complicated.

Another idea is a method to add temporal distribution schemes.
A more advanced method than the standard Framing option in NRC can do:
first (0 %) -> last (100 %) -> middle (50 %) -> 25 % -> 75% -> 12.25 % -> 37.5 % -> 62.25 % -> 87.25% -> etc

Fusion9_export.jpg
 

slacer

Member
It is looking great! :D
I wonder if you think of these renders as temporary only.

If you force all renders to end up in the same NRC_Output folder and all scenes create a bunch of Final_Render_xxx.png files, you would need to ensure not to overwrite the files from your previous session. Things could go wrong if you start a full render sequence and load up another scene to work on. As soon as you send images from your new scene to the queue, you might get into the situation of overwriting stuff.
 

vncnt

Well-known member
The scene name is still part of the directory names that each task is generating. So if you load another scene, it can't interfere.

And yes, any scheme requires coordination to prevent extra rendering. That's why it should be automated.

First I need a method to analyse the output directories to get unique file names for each channel, that has actually rendered.
The classic Legato plugin already has some of that code (to visualize rendered/missing/corrupt output files). Interesting how this idea starts growing.
 

vncnt

Well-known member
Latest version:
  • Added Channel name and File extension.
  • Implemented Channel name and File extension in Build Fusion 9 Comp but needs more testing with available Fusion formats to get the right descriptions.
  • Changed the UI to add a graph area. It stretches with the resizable dialog.
  • Added background raster based on Preview Range, incl. Render Start (green), Render End (blue), and current time (yellow).
The idea is to add a colored block to mark each file per frame per channel type.
This system makes it easy to spot problems in the output files - and even define a range of frames that need to be rendered again!

LegatoSendToNRC_23JUN2021.jpg
 

vncnt

Well-known member
I'm feeling like Frankenstein right now: isolated the File Analysis from the Legato plugin.
It's based on the older buffer structure so I'm rewriting on the fly because the NRC system is different.
Also prepared the blocks of code for mouse interaction - not very functional yet, but it doesn't crash.

Status at this moment: there is a rainbow!
It needs to be stretched to the larger set of possible buffer output file names.
As a bonus, it seems the render duration (derived from the file statistics) is alive as well.

In this image, the graph is vertically stretched to show all details.
Channel #22 seems missing.
Render duration is based on seconds. I don't think there is a parameter for a more refined resolution.

Many possibilities: output file analysis, task assignment, cpu management, render time prediction ...

rainbow.jpg
 

vncnt

Well-known member
Here is something to play with!
Features:
  • Segmentation System
    Every change is reproduced in the Camera viewport. Click the '-' and '+' buttons to select different segments of the viewport.
  • Statistics
    Rainbow colored. Click on a row to highlight channel + display channel name on top.
    Handles all 62 standard named buffer files that end with a sequence number.
    Rendertime per frame, total, and average.
    Missing frame files remain gray, in this example: buffer channel 33, frame 7 .. 12).
    Zero sized files are highlighted, in this example: buffer channel 2, frame 7).
  • Frame Range
    Select a Frame Range with LMB-drag, and click the Send Full Frame Selection button to send a render task to the NRC queue to (re-)render the selected frames.
    Or press key 'g' on the keyboard, to zoom into the selected frame range.
    If nothing is selected, press key 'a' on the keyboard, to zoom into the render range.
    Or press key '+' or '-' to zoom-in/out.
    If nothing is selected, you may RMB-click in the graph area to reset pan and zoom settings.
  • PAN timeline
    LMB-click in the graph area, to change the current frame - don't move - then MMB-drag in the graph area to PAN.
  • ZOOM timeline
    LMB-click in the graph area, to change the current frame - don't move - then RMB-drag in the graph area to ZOOM.
Known limitations:
  • PAN/ZOOM currently needs you to drag from the current time - I'll try to fix that later.
  • Statistics currently only handles full frame output. Segmented output is not implemented yet.
  • Statistics, segmentation, and Build Fusion 9 Comp currently only use PNG files. Other file types will be implemented later.
  • You need to click the Init Segmentation button to render segments.
  • If you selected a Codec Friendly value of '/64', but need to revert to '/4', will need to click Init Segmentation again. It doesn't reset by itself.
  • The Build Fusion 9 Comp button needs you to specify a Channel, for example: Final_Render, or Alpha, etc.
  • The Segment Overlap (px) is not implemented yet.
  • Display refresh optimizations are disabled for now.
Let me know how this version works for you.


SendToNRC_26-JUN-2021.jpg
 

Attachments

  • Legato_SendToNRC___build26-JUN-2021.zip
    159.8 KB · Views: 23

slacer

Member
Hi Vincent, thank you for sharing your tool.

I tried the following settings and created 8 segments.
I selected two files and startet the render tasks one by one.
Both render tasks used the same file name and only the image from the latest finishing job is in the output folder.

Did I miss something?
I checked the generated scene file and found the problem. The buffer name is Final_Render and the segment for the given frame is not part of the scene file.
IIRC, you mentioned to even create subfolders with the project name in order to prevent to overwrite file 1 from one scene with file 1 from another scene.
I don't know if you plan to implement this, or if it should be in the provided build alreads.
Could you add the build version as comment to the generated scene files? This would help to identify, if I used an outdated version.
 

Attachments

  • Legato_settings.PNG
    Legato_settings.PNG
    38.5 KB · Views: 18
  • config.PNG
    config.PNG
    45.4 KB · Views: 19
  • overwritten.PNG
    overwritten.PNG
    15.4 KB · Views: 17
  • RenderSettings.PNG
    RenderSettings.PNG
    27.6 KB · Views: 18
  • partial_scene.PNG
    partial_scene.PNG
    31.8 KB · Views: 18
Last edited:

vncnt

Well-known member
You unchecked the Use Preferences Output path. That is probably the cause of your problem.

Check it, and the output images are collected in separate directories, ready for assembly in Fusion.

Like this:
\test_box+Segment=1_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=2_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=3_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=4_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=5_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=6_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=7_8+Frame=7.lws\Alpha_00007.png
\test_box+Segment=8_8+Frame=7.lws\Alpha_00007.png

And yes, the names in those directories are equal.

This is the situation that the script behind the Build Fusion 9 Comp button expects.

Let me now if it works.

[EDIT] I've had some weird problems after re-enabling Use Preferences (due to some internal Layout bug maybe?) so you might need to copy/paste that section from a clean file. Or simply start all over with a new LWS scene file. See also https://forums.newtek.com/threads/n...nal-render-sequences-bug.164738/#post-1614825
 
Last edited:

slacer

Member
Hi,
I just followed the steps in the manual which states to
Don't leave it to the default Use Preferences Output Path.
Preparing Scenes for NRC rendering
And this affects me, because I use Windows and MACs for rendering.

I found the missing setting in the Network Render Controller. I had to enable Organize output subfolders by task name.
Since I rendered a single Frame with 8 segments, the generated Fusion 9 script could not find the required files.

I wanted to render Frame 152 but I got the following path in the Fusion comp file:
Filename = "C:\\NCR_output\\InteriorProject+Segment=6_8+Sequence=165_300_3.lws\\Final_Render_000.png",
but I needed
Filename = "C:\\NCR_output\\InteriorProject+Segment=6_8+Frame=152.lws\\Final_Render_152.png",
I have no clue where the numbers 165_300_3 came from.

After renaming the paths in the comp file, Fusion was able to render a single image from those 8 segments.
But the final filename was "Final_Render_00000.png", which is not correct for frame 152. This relates to the configuration in this snippet:

Code:
Saver1 = Saver {
            CtrlWZoom = false,
            Inputs = {
                ProcessWhenBlendIs00 = Input { Value = 0, },
                Clip = Input {
                    Value = Clip {
                        Filename = "C:\\\\NCR_output\\InteriorProject+merged\\Final_Render_00000.png",
                        FormatID = "PNGFormat",

Ok, the sequece came from a preselected scene range, but I don't know how to get rid of this in order to render only 1 single frame in Fusion.
 

Attachments

  • MixedVersions.PNG
    MixedVersions.PNG
    9 KB · Views: 19

vncnt

Well-known member
The manual needs to be updated.
Organize output subfolders by task name seems to override all situations.
Doesn't that lead to problems in your mixed environment?

I need to verify if the Fusion 9 script has problems dealing with a single frame task - looks like I've skipped that and focussed on sequence handling instead.

For now, I think the Fusion script always starts with an output frame 00000.
I should address that part in Fusion file later ( ... the final filename was "Final_Render_00000.png" ...).

The Filename = "C:\\NCR_output\\InteriorProject+Segment=6_8+Sequence=165_300_3.lws\\Final_Render_000.png" seems to come from a different moment.
Pressing the wrong button.
Sequence=165_300_3.lws means startframe=165, endframe=300, stepsize=3.
Can you verify that?

Nice issues to put on my todo list (with high priority).

In the mean time I´ve added a button "Send Segments Selection" (similar to Send Full Frame Selection), and started to revive the code blocks for Render Time Prediction.
A method that stores F9/F10/F11/F13 (yes) render statistics per frame, and is able to display it in the graph area.
It's using Render Presets (anything relevant that LScript is able to read in LW2020, at least some Camera settings) - need to adjust this to the LW2020 situation.
It should provide an indication of expected render time.
Would be interesting if I could get the name of the render node (from the NRC Controller) with that information for render node profiling.
To deal with multiple hardware (old/new).
 

slacer

Member
The manual needs to be updated.
Organize output subfolders by task name seems to override all situations.
Doesn't that lead to problems in your mixed environment?

I need to verify if the Fusion 9 script has problems dealing with a single frame task - looks like I've skipped that and focussed on sequence handling instead.

For now, I think the Fusion script always starts with an output frame 00000.
I should address that part in Fusion file later ( ... the final filename was "Final_Render_00000.png" ...).

The Filename = "C:\\NCR_output\\InteriorProject+Segment=6_8+Sequence=165_300_3.lws\\Final_Render_000.png" seems to come from a different moment.
Pressing the wrong button.
Sequence=165_300_3.lws means startframe=165, endframe=300, stepsize=3.
Can you verify that?
My scene file starts with these lines:

Code:
LWSC
8

RenderRangeType 0
FirstFrame 165
LastFrame 300
FrameStep 3

This means, these settings will be used if I open the scene.

I tried to change the settings, but new rendered sequences were configured to keep the sequence form 165-300
 

vncnt

Well-known member
I tried to change the settings, but new rendered sequences were configured to keep the sequence form 165-300
How and when did you change these settings?

The plugin stores the current settings, changes them according the request, saves the LWS scene file as a temp file, and finally restores them.
After this, you can save the scene to keep your new settings or quit without saving the new settings.

If needed, I can make a version that writes to a log file. For a better understanding.
 

slacer

Member
How and when did you change these settings?

The plugin stores the current settings, changes them according the request, saves the LWS scene file as a temp file, and finally restores them.
After this, you can save the scene to keep your new settings or quit without saving the new settings.

If needed, I can make a version that writes to a log file. For a better understanding.
I changed the start and endframe for the animation in the layout time slider.
Legato Send To NRC reflects these settings. But Send Segmented Sequence created 165-300 as range.

This is what I found in the scene file:

Code:
LWSC
8

RenderRangeType 0
FirstFrame 165
LastFrame 300
FrameStep 3
RenderRangeObject 0
RenderRangeArbitrary 1-120
PreviewFirstFrame 315
PreviewLastFrame 330
PreviewFrameStep 1
CurrentFrame 315
FramesPerSecond 24
AutoKeyFixedFrame 0

This means my changes in the UI only modified the PreviewFirst and Last frame, but not the Render First and Last frame.
Navigation with LMB and RMB in your plugin was strange and did not work as expected. Using the MMB resulted in Frames in the range from -3000 and 3000.
I found no way to reset the frame range in your plugin, so I closed Layout without storing my changes in order to get a fresh framerange from 165-300.

Ok - using the mouse to select a range of frames in your plugin works for Send Full Frame Selection but not for segmented sequences.
I had my focus on the segmentation part and this is not working for sequences in respect to the selected start and end frame.
 

Attachments

  • 315-330.PNG
    315-330.PNG
    388 KB · Views: 17
Last edited:

vncnt

Well-known member
I changed the start and endframe for the animation in the layout time slider.
Legato Send To NRC reflects these settings. But Send Segmented Sequence created 165-300 as range.

This is what I found in the scene file:

Code:
LWSC
8

RenderRangeType 0
FirstFrame 165
LastFrame 300
FrameStep 3
RenderRangeObject 0
RenderRangeArbitrary 1-120
PreviewFirstFrame 315
PreviewLastFrame 330
PreviewFrameStep 1
CurrentFrame 315
FramesPerSecond 24
AutoKeyFixedFrame 0

This means my changes in the UI only modified the PreviewFirst and Last frame, but not the Render First and Last frame.
Navigation with LMB and RMB in your plugin was strange and did not work as expected. Using the MMB resulted in Frames in the range from -3000 and 3000.
I found no way to reset the frame range in your plugin, so I closed Layout without storing my changes in order to get a fresh framerange from 165-300.

Ok - using the mouse to select a range of frames in your plugin works for Send Full Frame Selection but not for segmented sequences.
I had my focus on the segmentation part and this is not working for sequences in respect to the selected start and end frame.

Your scene file states the changes you made very clearly:
  • FirstFrame 165
  • LastFrame 300
  • RenderStep 3
It looks correct to me. And it should render between frame 165 and frame 300, step 3.
The Preview Range is not the same as Render Range.

There is an open issue with single frame assembly in Build Fusion 9 Comp.
Sequences should be fine (but currently always output, starting at frame 0).

There is also an issue with the number of digits that Build Fusion 9 Comp expects.
Expect these issues to be addressed in a next version.

I'll post 2 images to explain some start/end frame definitions.
LightWave doesn't know about a selected Frame Range concept. The plugin is using this to set the Render Range in the temp LWS scene files, that are sent to be rendered.
But only if you click the Selection-related buttons: Send Full Frame Selection or Send Segments Selection. (buttons can/will be renamed to keep the UI as compact as possible)

Even with a Frame Range selected in the interface, the Frame buttons will only process a full frame or a segmented frame, and
the Sequence buttons will only process a full frame sequence or a segmented frame sequence.
 
Last edited:

vncnt

Well-known member
The yellow arrows indicate the Preview Range.
The orange arrow indicates the Current Frame.
The green arrow indicates the Render Start Frame.
The cyan arrow indicates the Render End Frame + 1 frame (to include that frame, because it wouldn't look right in the Statistics mode)

The blue arrows show the relation between Selection related buttons and the Frame Range selector in the plugin interface.

To select a Frame Range: LMB drag in the plugin graph area. When selected, you can't pan or zoom anymore.

To PAN in the plugin graph area:
  1. unselect the Frame Range: LMB click anywhere in the plugin graph area
  2. place the mouse above the yellow line (Current Frame) --> this is a known limitiation, as stated earlier, to prevent a giant (and confusing) glitch
  3. now, MMB-drag from the yellow line to left or right
  4. the LW timeline and the plugin timeline (the Preview Range) should now pan left/right as expected
To ZOOM in the plugin graph area:
  1. unselect the Frame Range: LMB click anywhere in the plugin graph area
  2. place the mouse above the yellow line (Current Frame) --> this is a known limitiation, as stated earlier, to prevent a giant (and confusing) glitch
  3. now, RMB-drag from the yellow line to left or right
  4. the LW timeline and the plugin timeline (the Preview Range) should now zoom in/out as expected
To ZOOM into a selected frame range:
  1. select a Frame Range: LMB drag in the plugin graph area
  2. press keyboard key 'g' to goto the selected Frame Range ('+' and '-' keys are also active)
To RESET the plugin graph area:
  1. unselect the Frame Range: LMB click anywhere in the plugin graph area
  2. RMB-click in the graph area
  3. the LW timeline and the plugin timeline (the Preview Range) should now revert to the Render Frame Range as defined in Render Properties

frames.jpg
 
Last edited:
Top Bottom