A "smart node" script for OSX

stib

can't.. re.. member
I've written a little shell script for OSX machines being used as render nodes. The idea of this script is that the nodes will look through the rendered frames until they find one that isn't rendered, and render it. This way they all act independently, so if one goes down it won't affect more than the frame it was rendering at the time. I use it at work where my render machines are other user's desktops (mostly designers and photographers who usually have lots of cycles to spare most of the time), and they often get restarted or turned off when the user leaves work and ignores my sticky note telling them not to shut down (grrr). The script is designed to run at the lowest possible priority, so other users won't notice it (though if your scene requires a heap of memory it might be a problem).

Your remote machines will need to have access to the volumes on your workstation that hold the Content folder and any other shared assets (for example I have a separate volume that renders get sent to). I created a user called rendernode on each of the machines, and in that user's preference I turned on remote login so that I can connect via ssh. This means I can start the process without having to walk around the building, and I added a script to mount all the required drives to the user's bash_profile so that they mount them when I log in. The command to mount these volumes (which you run on the remote machine) is something like

Code:
 mkdir -p /Volumes/Content; mount_afp afp://rendernode:[email protected]/Content /Volumes/Content

This shell script actually needs zsh to run. If you're using the default osx shell (bash) you invoke it thus (in this case I'm starting it on designmac12 as user rendernode):

Code:
ssh [email protected] zsh smartnode.sh /Volumes/Content/scenes/myscene.lws
you can specify start frame, end frame, step and niceness (more nice means lower priority, default is lowest possible priority).

here's the code. Copy it into a new file called smartnode.sh and put that on all your remote machines.

Code:
echo "smartnode v0.2"
if [[ ! -z $SSH_CLIENT ]]; then #only run this bit on remote machines
        echo ">>>>>>>>>> starting smart node  <<<<<<<<<<<<"
        #My workstation has lwsn in the $PATH environmental variable, but the remote machines don't
        #customise the line below to reflect the location of the lwsn application
        lw=/Applications/NewTek/LightWave3D11.6.2/bin/lwsn
else
        lw=lwsn
fi # [[ ! -z $SSH_CLIENT ]]
if [[ ! -z "$1" ]] ; then #user supplied a file
	
	#these are how many zeroes are in the prefix formats LW uses when saving image sequences
	zeroes=(3 3 4 4 3 3 4 4 5 5 6 6 5 5 6 6 5 5 6 6)
	#these are the separators used before the numeric part
	separators=("" "" "" "" "_" "_" "_" "_" "" "" "" "" "_" "_" "_" "_" "." "." "." ".")
	#this is whether a suffix is saved
	suffixes=(0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1)

	if [[ -e "$1" ]]; then #only proceed if the scene file exists
		#if not specified in the command line, read start finish and step from file
		#f is a placeholder in case the user wants to use the value from the file
		if [[ ! $2 == "f" ]]; then startf=$2; fi
		echo "start frame:" ${startf:=$(grep PreviewFirstFrame "$1"|sed "s/PreviewFirstFrame[[:space:]]//")}
		if [[ ! $3 == "f" ]]; then finishf=$3; fi
		echo "finish frame: " ${finishf:=$(grep PreviewLastFrame "$1"|sed "s/PreviewLastFrame[[:space:]]//")}
		if [[ ! $4 == "f" ]]; then step=$4; fi
		echo "frame step: " ${step:=$(grep PreviewFrameStep "$1"|sed "s/PreviewFrameStep[[:space:]]//")}

		#niceness defaults to very nice
		niceness=$5; echo "nice-ness is ${niceness:=20}"

		#check the scene file for the output formatting
		prefix=$(grep SaveRGBImagesPrefix "$1"|sed "s/SaveRGBImagesPrefix[[:space:]]*//")
		nameFormat=$((1+$(grep OutputFilenameFormat "$1"|sed "s/OutputFilenameFormat[[:space:]]*//")))
		echo nameFormat $nameFormat
		zero=$zeroes[$nameFormat]; echo "zeroes: $zero"
		separator=$separators[$nameFormat]; echo "separator $separator"
		isSuffix=$suffixes[$nameFormat]; echo "isSuffix $isSuffix"
		if [[ $isSuffix == 1 ]]; then
			suffix=$(grep RGBImageSaver "$1"|sed "s/.*(\(.[A-z]*\))/\1/")
			echo "suffix $suffix"
		else
			suffix=""
		fi
		#this is how it all looks:
		echo "saving to ${prefix}${separator}$(printf %0${zero}d 1)${suffix} etc"

		sleep 5 #give user time to panic
		for i in {$startf..$((finishf/step))}; do
			m=$(printf %0${zero}d $((i*step)))
			theframe=${prefix}${separator}${m}${suffix}
			echo "skipping completed frame: $m"
			if [[ ! -e "$theframe" ]]; then #frame does not exist
				echo "_____________________________________________________\\nrendering $m to $theframe"
				touch "$theframe" #stop other nodes from rendering this file
				#actually render something
				nice -n $niceness $lw -3 -t10 $1 $m $m 1 #render the frame
			fi #[[ ! -e "${prefix}${separator}${m}${suffix}" ]]
		done
   else #file does not exist
		echo "!!!ERROR!!! scene file $1 does not exist"
   fi #[[ -e "$1" ]]
else #didn't specify a file
	cat <<HERE
	smartnode: render on a network using lwsn

	Usage: sh smartnode.sh SCENEFILE [start_frame] [end_frame] [step] [niceness]

	SCENEFILE is relative to the current cwd or absolute
	
	start_frame end_frame and step are read from the file if not supplied.
	+ The parameters must be given in order, but are all optional.
	+ use the letter "f" as a placeholder if specifying a later parameter
	  for example: smartnode.sh /path/scenefile.lws f f 10
	  would render from the start point to the end point specified in the file,
	  but using a step of 10, regardless of the value in the scene.
	
	Niceness determines the priority of the process:
	+ Higher numbers are lower priority (nicer), valid values are from 0-20.
	+ Use caution when using a low value. Defaults to 20 (very nice).
HERE
fi #[[ ! -z "$1" ]]
 
Last edited:
Top Bottom