Nuke and Shotgun Integration

molecule-style

In my years at the Molecule in NYC, first as a freelancer and then as a perma-lancer, I had lots of time to observe how a mid-sized studio manages the sometimes huge number of shots that need to be kept track of on a daily basis. When I started, the studio was using Ftrack to manage its shot pipeline, and while poking around online I saw that ftrack had a python API, and a lightbulb went off – Nuke + Python + Ftrack = ? Awesomeness, anyway.

So I went to work on building a nuke-based shot tracking system, first for artists but then expanding to include supervisors as well. I called it “the dashboard”, and though it started small it quickly became essential to the Molecule’s pipeline. When the studio switched tracking packages to Shotgun, a lot more functionality was exposed and the whole thing just got 50% better. You can catch a quick glimpse of it in this promotional video from Autodesk – at around 0:55 in this video, the awesome Rick Shick talks about how he uses it instead of the web-based interface almost exclusively in his role as comp supervisor:


Gone were the days of manually creating contact sheets for each project; now every artist and supervisor had access to dynamic contact sheets, through which they could see and change statuses, read and post notes and images, and quickly open any shot.

JPEG compression in nuke (in-line)

meet JPEGer

I’ve been annoyed with the basic jpeg nuke workflow for a while – the only way to get that magic compression look was to write it out as a jpeg, and then read it back in. So when I discovered blink scripts last week, building this was a great way to learn.

Introducing – JPEGer! Including quality slider, and options for chroma and alpha (jpeg-ing an alpha is untested right now). Interestingly enough, in our testing, the results were identical across multiple machines, so this should be safe for a renderfarm. I’m not sure if it’s coded in the most efficient way, but it seems to only take a few seconds/frame, even on 4k images, which seems workable.

Download:

JPEGer Gizmo
tested on Nuke 10v5, Linux

Nuke Ramped Defocus

A quick group for better defocus

If I have an image in nuke that I want to control the falloff of the depth of field, ordinarily I would pump a ramp into the mask input of defocus. However, this would only work if I want my defocus to start at 0 and move to some value. If, however, I want the smallest amount of defocus to be something other than 0, say 2, I can’t achieve this effect with a good result, even if I adjust the start value of my mask input (instead of ramping from 0-1, say .2-1). If I input this ramp into the mask input on a defocus, this is the result:

maskedDefocus

Instead of moving from a 2 defocus to a 10 defocus, instead I get the original un-blurred result, with the 10 defocus merged on top of it at 20%, creating a halo.

With a few merges and two defocuses instead, I can get a much better result:

rampedDefocus

So I grouped this little setup and posted it here. Simply add a gradient map into the ramp input, 0-1, and set your minimum blur and maximum blur values, and you’ll get a much smoother result.

Download:

rampedDefocus
tested on Nuke 9.0v4

Nuke VHS Noise Group

mograph in Nuke?!

vhs063

I’ve recently had to create (and re-use) a customizable VHS tracking-error style effect in Nuke, and have collected the node group in the attached file. It’s mostly controlled by some nobs on the “MASTER” node, though there are some other tweakable elements here. Unfortunately there’s no documentation so you’ll have to play around with it, but I’ve been getting good results from it.

Download:

vhsNoise
tested on Nuke 8.0v1

Nuke multiGrad

an advanced 4-point gradient gizmo for rotoscoping

***updated 3/9/2015

I’ll be the first to admit that I’m no expert on rotoscoping, but in my experience one of the first tools I look for is a multi-gradient – something that I can use to paint out large sections of the image, and then add detail to. So I was surprised that, besides this shake-like 4-point gradient there wasn’t anything in Nuke that was what I wanted. The shake gradient is nice, but I was really looking for one that would allow me to move the points around in space, and would interpolate colors around the points as well as in between. With some patience and a lot of math (point-slope anyone?) I created a gizmo that will do just that. Below you can see the results on a racecar:

Of course, it’s not perfect, and there would need to be fine tuning around the edges, but all the rotoing in that image was done just with this tool. The user pipes in the source image to “Source” and a mask to “matte”, and the gizmo will comp it onto the source automatically, preserving the source alpha (the user can turn this feature off under the “Matte” tab. It also includes python scripting that will automatically grab the color of the source at the points, which makes painting out areas really quick and easy.

Control Panel

Because of the math involved, points 1 & 2 must be at the top, and 3 & 4 at the bottom, or it will start to act screwy. If it needs to rotate, you’re better off translating the whole result.

Download:

4pointgradient.txt
tested on Nuke 9.0v4

Houdini CloudMaker OTL

a custom asset that takes rough input geometry and creates awesome clouds
cloud

I’ve been working on some cloud-related projects, and I needed a streamlined way to make many different clouds – with art-directed shapes – quickly and simply. I’ve come up with an OTL that takes rough input geometry and then builds a wispy, fractal-y volume for rendering.

The asset is implemented in Volume VOPs, using simple noise equations to extrapolate cloud-like edges. It creates an intermediate geometry out of metaballs, then creates a volume and adds noise. With this, you can quickly and easily create simple cloud shapes and convert them into high-quality clouds for rendering.

Download:

cloudMaker OTL
requires houdini 12.5 or higher

Terminating a child process from python

fixing a problem with calling 3rd party applications as subprocesses

I’ve been working on improving my renderfarm, and I’ve run into some trouble trying to close clients remotely. You can easily end a python process with sys.exit(), however if a rendering application (say Nuke) has been spawned by the script, it will not close. After some digging (and a bunch of great help from stackOverflow) I seem to have come up with a solution. When you call a 3rd party application with either call() or:

process = Popen(allRenderArg, env=os.environ)

Python will create a tiny, dummy process which only exists to call that application. When you exit your script, python will clean up those dummy processes, but won’t kill child processes, thereby leaving the renders going.

The solution that I’ve come up with is to get the pid of those dummy python processes, then use some unix commands to find the pids of their children, and kill them with os.kill(). If this was a linux platform I could do it in a more efficient way (possibly using pstree) but on OSX I have to use grep and some fancy code:

processId = process.pid
print "attempting to terminate "+str(processId)
command = " ps -o pid,ppid -ax | grep "+str(processId)+" | cut -f 1 -d \" \" | tail -1"
ps_command = Popen(command, shell=True, stdout=PIPE)
ps_output = ps_command.stdout.read()
retcode = ps_command.wait()
assert retcode == 0, "ps command returned %d" % retcode
print "child process pid: "+ str(ps_output)
os.kill(int(ps_output), signal.SIGTERM)
os.kill(int(processId), signal.SIGTERM)

There might be a nicer way, but I don’t know it.

OSX Paths, X11 and paths.d

part rant, part fix to an obnoxious bug: "1/bin"

If you’re working with a renderer from the command line, you’ll often need the location of that renderer in the path, so you can call it easily (without always needing the full path). On OSX, there is a folder paths.d under /etc, where you can place text files that will contain paths to be appended to $PATH in terminal sessions:

/etc/paths.d/Nuke6.3v8 contents:

/Applications/Nuke6.3v8/Nuke6.3v8.app/Contents/MacOS

This morning I was attempting to add some other versions to the path (7.0v1, 7.0v6) and I was running into some ridiculous errors. When I added another file:

/etc/paths.d/Nuke7.0v1 contents:

/Applications/Nuke7.0v1/Nuke7.0v1.app/Contents/MacOS
what I got in my path was instead
/Applications/Nuke7.0v1/Nuke7.0v1.app/Contents/MacOS1/bin

Why? I have no idea. I tried different file names, different text encoding types, etc and was always getting that extra text on the end. I noticed that I had one other file in my paths.d, from the installation of the osx developer tools: 50-X11. When I removed that file, everything worked! However, I want to keep that in my path (not sure what apps might be there), so I renamed it to x11 and now that extra text is gone.

I hope this helps someone else in the same situation. I don’t understand what macintosh is doing half the time, and this is that half.

Nuke command line, “argument not used”

how to appropriately specify frames to Nuke in the command line

Working with my python-based nuke render farm, I’ve been shoring up the code to deal with some unusual instances (single frames, fewer frames than clients, et al). When passing arguments to Nuke for rendering, I used this format:

Nuke7.0v6 -m 8 -x projectFile.nk -F 250-500

When I need to send multiple frames, or frame ranges, to the same instance, the documentation says you can use multiple instances of the -F argument, like so:

Nuke7.0v6 -m 8 -x projectFile.nk -F 250-500 -F 550-700

What you’ll get, though, is only the second specified frame range (in this case, 550-700) rendering, and the following output:

"-F": argument not used
"250-500": argument not used
"-F": argument not used

After discussing with Foundry support, it seems the -F switch will only work if placed before the -x switch, like so:

Nuke7.0v6 -m 8 -F 250-500 -F 550-700 -x projectFile.nk
Otherwise, Nuke assumes the last number is the legacy method of passing frame ranges to the command line renderer, and ignores your -F switch. I’ve requested that this be specifically mentioned in the documentation.

Houdini Animation Curves otl

being an operator that can extrapolate one set of values to another using curves
curves

I’ve recently been working on an idea that requires me to do separate luminance values in an image into x discreet “steps”, such as 0-12. However I don’t want to do the distribution linearly (read: equally distributed), I need to account for the fact that luminance is non-linear.

Continue Reading →