Loading render passes/aov’s inside maya renderview with python

load aovs maya python

One of the nice features of vray is the ability to load the different render passes inside the vray framebuffer in maya. This give us the option to troubleshoot for example the aliasing issues like noise and such. Lately I have been working mostly with arnold and wanted to add this feature. In this post I am going to share how I did it, using python.

UI Considerations

maya_user_interface

We should define first where and how we want to “render” this effect. For me makes sense to be an optionMenu like you see in the vray Framebuffer or inside nuke. Also we need some place to hold the UI, it can be a floating window that you can access with a shelf button, inside the renderview itself or even in a docked panel where you have your custom scripts like I do. In my case I use it inside my custom scripts docked panel, if you want to use it in the renderview you will have to copy the “renderWindowPanel.mel” file inside C:\Program Files\Autodesk\Maya20xx\scripts\others, paste it in a a scripts path that maya reads and edit it. In this particular post we will be testing our script inside a simple maya window.

Main idea

The idea behind this is very simple, maya’s renderview has the ability to load images (file > open image), and we will assign to each menu item inside the optionMenu a render pass or AOV as I will be refering from now.

Initial setup

maya initial setup

It is important to understand that if you are working in linear workflow and using 16/32 bit images like exr’s, you need to setup a few things so this works correct. You need to have the plugin “OpenEXRLoader” loaded in your plug-in manager, you probably have this already, and also if the images we load in the renderview are linear we need to set the “defaultViewColorManager” in display > color management to linear in the input. If you are working with arnold in the gamma settings change the “display driver gamma” to 1.0.

At this point, create a simple scene so you can test the script that we will be building. Don’t forget to go to the AOV’s tab in the render settings and make a few to test like “N, P, direct diffuse, direct specular, indirect diffuse…”.

Building the UI

import maya.cmds as cmds
import os

def aovUI():
	if cmds.window('aovWindow', ex=1):
		cmds.deleteUI('aovWindow')

	cmds.window('aovWindow', w=150,h=100,s=0)
	cmds.formLayout('wndLayout')

First of all we import the needed modules, the maya.cmds and the os python module to deal with files and folders.

Next we make a function called aovUI, and write the basic aproach of dealing with maya windows. Check if the window exists and delete it , so we dont have any duplicated windows, then we define the window itself(w and h is the size, s flag is to set the window not sizeable by the user) and give it a layout to hold the ui elements like buttons and such, in this case a formLayout.

descAov = cmds.text( label="Load AOV's",w=150,h=22,fn='boldLabelFont',bgc=(0.15,0.15,0.15))

aovsList = cmds.optionMenu( 'aovsListMenu',w=95,label='AOV') #add cmd later
activeAOVS = cmds.ls(et='aiAOV')
aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

cmds.menuItem(label='beauty')

for item in aovsNames:
	cmds.menuItem(label=item )

#Attach controls
cmds.formLayout('wndLayout',e=1,af=[(descAov,'top',0),(descAov,'left',0)])
cmds.formLayout('wndLayout',e=1,af=[(aovsList,'top',25),(aovsList,'left',5)])

cmds.showWindow('aovWindow')

Now setup some variables, the “descAov” is just a text label to inform the user what this tool does, the label flag is the text to render, “w and h” are the size of the text box since we will be using a background color in it, the “fn” flag is the font style.

Then we create an optionMenu “aovsList “, set the label and the width as you want. Later we will add a “change command” to load the specific AOV to the renderview.

The next step is to store in a variable “activeAOVS ” the aov’s present in the scene (normal, diffuse, specular, etc). You can use the command “ls” to list the exact type of node (et flag), and we pass it as a string, in the case of arnold renderer is “aiAOV”. Select the line “cmds.ls(et=’aiAOV’)” and see what the script editor will print, in my scene:

 Result: [u'aiAOV_N',
 u'aiAOV_P',
 u'aiAOV_direct_diffuse',
 u'aiAOV_direct_specular',
 u'aiAOV_indirect_diffuse',
 u'aiAOV_indirect_specular'] #

Ok great, we have a list of all the active AOV’s in the scene, one thing that we dont want is the prefix “aiAOV_”, since we will be filling our optionMenu with those names and also because arnold creates folders with the name without that prefix to save the AOV’s. So we need to clean that prefix and store it in a variable.

That variable is “aovsNames” , which looks complex but is just saying that for each element in activeAOVS, split the word in the “_” and keep the last part of it [1], [0] is the first part.

Now that we have the names of the aov’s we can fill the optionMenu. But first we need to consider that arnold will always output a “beauty” pass, even if you didn’t select it in the AOV’s tab, and that’s important since we want to see also the final result. So, since we dont have the “beauty” aov in our list, we will add it mannually.

In the next step we fill the optionMenu with all the others aov’s. For each item in our active list create a menu item and label itself.

Returning to our UI elements, we need to attach/position those controls in the formLayout we created earlier, otherwise they will be on top of each other. Just pass the layout name you want to edit, “wndLayout”, say you want to edit it with the “e” flag, and set the position you want.

Finally we show the window with the “showWindow” command. By now executing this function, calling the function name, aovUI(), you should have something like this:

maya aov window

Main command to load AOV’s

Having the UI done we will add now the functionality with another funtion.

def loadAOV(*args):

	activeAOVS = cmds.ls(et='aiAOV')
	aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

	if activeAOVS == []:
		cmds.warning("No aov's setup")

Start by defining the function with the argument “*args”, whenever you execute a command from an UI element you need to pass this argument.

To keep this simple and not dive into classes or arguments, define again the aov’s variables names in this function. Next we need to make a test, if the activeAOVS returns an empty list we will use the “warning” command to show in the command line that we don’t have any AOV’s present in the scene, so we can’t use this feature.

def loadAOV(*args):

	activeAOVS = cmds.ls(et='aiAOV')
	aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

	if activeAOVS == []:
		cmds.warning("No aov's setup")
	else:
		CurrentProject = cmds.workspace(q=1,fullName=1)
		rview = cmds.getPanel( sty = 'renderWindowPanel' )
		selectedAOV = cmds.optionMenu('aovsListMenu',q=1,v=1)

		ImgTempPath = CurrentProject + '/images/tmp/'

If we do have aov’s, we continue and set some needed variables. CurrentProject will use the workspace maya command to return the project path, for example:

Result: u'D:/Maya_projects/guitarra_portuguesa' #

The “rview” will return the renderview window that will be used to load the images. In the variable “selectedAOV “, we query/ask which is the selected element/AOV in the optionMenu, by using the query flag “q”, and the value “v”.

When we render an image inside maya, they are saved in a folder named tmp inside our project images , for example:

Result: u'D:/Maya_projects/guitarra_portuguesa/images/tmp/' #

This is the result of adding the “currentProject” with “‘/images/tmp/” in the variable ImgTempPath.

Get the image folder, name and extension

The goal of this main command is to get the correct path of the image to load in the renderview. By now we have the folder where the images are saved. But if you do a test render and open your tmp folder inside your project’s images folder you will notice that arnold will create different folders for each aov, and then an image inside of it. The name of the image is defined in your render common settings just like the extension. To get those you can use the following commands:

FileName = cmds.getAttr('defaultRenderGlobals.imageFilePrefix')
extension = cmds.getAttr('defaultArnoldDriver.aiTranslator')

The problem with these commands, particulary the “FileName”, is that sometimes you have no custom name and you are using the scene’s name or if your scene is not saved it will be named “untitled”. To get around those issues without doing tests I found that using some python OS commands we could get there faster.

def all_files_under(path):
	for cur_path, dirnames, filenames in os.walk(path):
		for filename in filenames:
			yield os.path.join(cur_path, filename)

This function will search inside our tmp folder and return the path and file name/extension. This will be used inside our main function to pass the “path” argument and get the last modified file.

Complete the main fucntion

Returning to our command, we will manipulate the result of the “all_files_under” helper function to complete it.

def loadAOV(*args):

	activeAOVS = cmds.ls(et='aiAOV')
	aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

	if activeAOVS == []:
		cmds.warning("No aov's setup")
	else:
		CurrentProject = cmds.workspace(q=1,fullName=1)
		rview = cmds.getPanel( sty = 'renderWindowPanel' )
		selectedAOV = cmds.optionMenu('aovsListMenu',q=1,v=1)

		ImgTempPath = CurrentProject + '/images/tmp/'
		latest_file = max(all_files_under(ImgTempPath), key=os.path.getmtime)
		splitPath = latest_file.replace('/','\\')

		splitPath = splitPath.split(os.sep)

		for n, i in enumerate(splitPath):
			if i in aovsNames or i == 'beauty':
				splitPath[n] = selectedAOV

		splitPath[0] = splitPath[0] + '\\'
		pathToImg = reduce(os.path.join,splitPath)

		#print pathToImg
		cmds.renderWindowEditor( rview, e=True, li=pathToImg )

def all_files_under(path):
	for cur_path, dirnames, filenames in os.walk(path):
		for filename in filenames:
			yield os.path.join(cur_path, filename)

We left the main command in the “ImgTempPath” variable. Now define one named “latest_file” with a python code that will return the path to our latest modified/rendered image. In my scene the result is:

'D:/Maya_projects/guitarra_portuguesa/images/tmp/indirect_specular\\guitarra_new.exr

This give us already the name and extension of the image, the problem is that the aov folder will need to be replaced with the selected aov, otherwise will always load the aov in the “indirect_specular” folder or any other that the helper function returns.

So, in the result above we see that we get double backward slashes (\\) just before the image name, to avoid any issues with the OS python commands we will make the path cleaner making all the slashes equal, that’s what the “splitPath” variables does, replace the all the “/” with “\\”. This might be different in your particular system language, I am not sure, but for me this was the way to do it, so I could split all the path replace the aov folder with the selected aov and join the path again to get the final result.

Next we edit the splitPath variable with an “os.sep split” to get the path into “pieces”.

 Result: [u'D:',
 u'Maya_projects',
 u'guitarra_portuguesa',
 u'images',
 u'tmp',
 u'indirect_specular',
 u'guitarra_new.exr'] #

Now you can see why we are doing this, we split the path so we can replace the aov folder with the selected aov like I said before.

In the next step we do a for loop to replace the the aov folder name with the one we select in the optionMenu. The “n” represents a counter, the ‘”i” is the iterator in the “splitPath” list. Then if the “i” is the “aov folder”, in this case “indirect_specular”, we replace it with the selected aov. We discussed before the “beauty” issue, so we need to test also if the helper functions returns the beaty folder in the path.

In my case I need to add some slashes to the item in the splitPath that holds the drive (‘D:’), otherwise my next step won’t work properly. That’s the “splitPath[0]“, where we add to the drive slashes. (Probably there is a better way of doing this).

Now we join the pieces of the path to get the full path again in the “pathToImg”, using the python’s “reduce” and “os.path.join”.

Finally we can load the selected image/aov in the renderview, using the maya command “renderWindowEditor”, passing the “rview” variable that we used before, edit flag “e”, and loadImage flag “li” with the final path, pathToImg.

Add the main command to the UI function

Almost done, we just need to add the “loadAOV” command to the optionMenu UI item.

"""Find the line:"""
aovsList = cmds.optionMenu( 'aovsListMenu',w=95,label='AOV') #add cmd later
""" and replace it with: """
aovsList = cmds.optionMenu( 'aovsListMenu',w=95,label='AOV',cc=loadAOV )

All together

Here is the final script, in the last line we call the UI function.

import maya.cmds as cmds
import os

def aovUI():

	if cmds.window('aovWindow', ex=1):
		cmds.deleteUI('aovWindow')

	cmds.window('aovWindow', w=150,h=100,s=0)
	cmds.formLayout('wndLayout')

	descAov = cmds.text( label="Load AOV's",w=150,h=22,fn='boldLabelFont',bgc=(0.15,0.15,0.15))
	aovsList = cmds.optionMenu( 'aovsListMenu',w=95,label='AOV',cc=loadAOV )
	activeAOVS = cmds.ls(et='aiAOV')
	aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

	cmds.menuItem(label='beauty')

	for item in aovsNames:
		cmds.menuItem(label=item )

	#Attach controls
	cmds.formLayout('wndLayout',e=1,af=[ (descAov,'top',0), (descAov,'left',0) ])
	cmds.formLayout('wndLayout',e=1,af=[ (aovsList,'top',25), (aovsList,'left',5) ])

	cmds.showWindow('aovWindow')

def loadAOV(*args):

	activeAOVS = cmds.ls(et='aiAOV')
	aovsNames = [i.split('_', 1)[1] for i in activeAOVS]

	if activeAOVS == []:
		cmds.warning("No aov's setup")
	else:
		CurrentProject = cmds.workspace(q=1,fullName=1)
		rview = cmds.getPanel( sty = 'renderWindowPanel' )
		selectedAOV = cmds.optionMenu('aovsListMenu',q=1,v=1)

		ImgTempPath = CurrentProject + '/images/tmp/'
		latest_file = max(all_files_under(ImgTempPath), key=os.path.getmtime)
		splitPath = latest_file.replace('/','\\')

		splitPath = splitPath.split(os.sep)

		for n, i in enumerate(splitPath):
			if i in aovsNames or i == 'beauty':
				splitPath[n] = selectedAOV

		splitPath[0] = splitPath[0] + '\\'
		pathToImg = reduce(os.path.join,splitPath)

		print pathToImg
		cmds.renderWindowEditor( rview, e=True, li=pathToImg )

def all_files_under(path):
	for cur_path, dirnames, filenames in os.walk(path):
		for filename in filenames:
			yield os.path.join(cur_path, filename)

aovUI()

Ok, now you should be able to load your render passes/aov’s inside maya without using any external app to check your passes before going to the compositing stage. Let me know if you have any issues with the code, and try to make your own tools with python and maya commands. Have fun!

  • Pingback: Maya | Loading Render Passes and aov’s inside the RenderView with Python

  • Brent Tyler

    This is pretty smart. Have you figured out how to do this from multichannel EXRs?

    • therenderblog

      Hey Brent, thanks for the input. I haven’t tested yet. It can be tricky and there is no default option to load render passes in the renderview. Maybe you can do it by silently opening imf_disp *behind* maya, I will test it and let you know. That would be a nice feature.

  • Edmond Hyun Ho Yi

    wow it’s so lovely Script !!!

  • Nguyên Minh Bùi

    Hello! I tried to do as you told but keep having this error*
    # Error: IndentationError: unindent does not match any outer indentation level #

    • therenderblog

      hey, that’s an Indentation Error, probably when you copy some code from here the formatting is incorrect. Save your code as a .py file and send me to therenderblog@gmail.com, I will correct it for you.

  • hus787

    A few suggestions and fixes, which if used would definitely make your code much cleaner and simpler to read.

    There is no need to walk the entire `images/tmp’ folder to hunt for the latest (by getmtime) file and then do a replace with selectedAOV. This method will easily fail in case the user adds a custom file prefix setting in the render settings e.g. `/_. What could have been used is something like:

    img = pc.renderSettings(fin = True, lut = True, fpt = True)[0]
    pathToImg = img.replace(”, selectedAOV)

    that’s it. (`pc` is `import pymel.core as pc`).

    Side note, since it would have a minimum affect on the performance of the script:

    Care should be take with the use of `reduce`. `os.path.join` supports multiple arguments. And a quick profiling I did to compare `reduce` used with `os.path.join` and `os.path.join` used with multiple arguments (`xrange(100000)`) showed that the former was ~57x slower than the latter.

    Another user convenience suggestion:

    When the user chooses a different AOV run this in `loadAOV`:

    pc.setAttr(“defaultArnoldRenderOptions.displayAOV”, selectedAOV)

    • therenderblog

      Thanks for that :)

    • lokesh

      Hi,

      I am lokesh, i program some tools in maya using python. As i am new to
      arnold, i am not able to understand how to add default (built-in – ex:-
      deep_scatter) aov in to scene using python in maya.

      can you please help me figure out how to add built-in aov.

      Thanks in advance.

      • therenderblog

        Does this works?

        aovListSize = cmds.getAttr(‘defaultArnoldRenderOptions.aovList’,s=1)

        customAOV = cmds.createNode(‘aiAOV’,n=’deep_scatter’, skipSelect=True)

        cmds.setAttr(customAOV+’.name’,'deep_scatter’, type=’string’)

        cmds.connectAttr(customAOV+’.message’, ‘defaultArnoldRenderOptions.aovList[' + str(aovListSize)+']‘,f=1)

        cmds.connectAttr(‘defaultArnoldDriver.message’, customAOV+’.outputs[0].driver’, f=1)

        cmds.connectAttr(‘defaultArnoldFilter.message’, customAOV+’.outputs[0].filter’, f=1)

        • lokesh

          Hello,
          I am able to create aovs by using the lines you have given but they are creating in custom aovs section. but i want to create builtin deep_scatter aov.

          • therenderblog

            My bad, better use native arnold functions:

            import mtoa.aovs as aovs

            # Create AOV

            aovs.AOVInterface().addAOV(‘deep_scatter’, aovType=’rgb’)

  • Macbeth R.

    Do you have an example for the customized renderview? (
    “renderWindowPanel.mel”) Thanks!

    • therenderblog

      You can find examples on creativecrash (for example RenderView Extension).

      • Pedro

        Hey Joao.
        Great job, this is awesome!
        Now, I’m also trying to get this to work with the renderWindowPanel.mel unfortunately no luck so far… I know a little python but pretty much no mel :-/ and not sure how to implement this… I had a look in other customised renderWindowPanel’s and other mel script examples etc. but can’t really find an example where someone would run a python script in mel – Is it even possible or would I have to script this ‘tool’ in mel? Some help would be massively apreciated!!
        Obrigado!

        • therenderblog

          Pedro, you can execute python in a mel script with the mel command “python command”. But for that specific task I would definitely go with mel. If you know a little “maya cmds python” you can easily translate it to mel. Just a matter of having the maya docs open for both mel and python version of the command you are working on and see the “pattern”. Also use the current mel document of the renderView to follow the rules and patterns of mel. You just have to learn the rules, the idea is the same.

          If you already have the code in python it will be easy to translate, give it a try, if you having any problem send me an email with the code and I will try to help.

          Boa sorte ;)

  • Macbeth R.

    Could you load a 32bit exr on renderview? and actually be able to use the exposure slider on it? for example for Z pass, or cputime?

    • therenderblog

      Yes you can, but keep in mind this is not using multi layer exr’s, we use it as separate files, otherwise the renderview won’t be able to filter the passes.

      • Macbeth R.

        Try it in LINUX, 32bit float exr is just black, got HDR on, and using exposure, what options are you using in the aov driver for the exr?

        • therenderblog

          Can’t try on linux unfortunately, I am just using default options (don’t merge aov’s). You can change that for the final renders, this is just for testing the passes inside maya while you’re developing the shaders/lighting.

  • lokesh

    Hi,
    Can any one please help me to figure out how to get the AOV actual name or its customAOV number on a shading Engine.

    or how can i connect a shader to any shadingEngine’s customAOV.
    for ex:
    i have 4 customAOVs (bag, belt, hair, shoe) in a scene. i want to connect a utility node to customAOV hair in a particular shadingEngine.

    • therenderblog

      Not sure if this is what you’re looking for:

      https://vimeo.com/63433605

      • lokesh

        Hi,
        i can able to create it manually but i want to create it dynamically using python scripting.

  • Nathan Boyd

    what would be the mental ray equivalent to “aiAOV”?

    • therenderblog

      Something like this?

      renderPass = cmds.shadingNode(‘renderPass’, n=’MY_BEAUTY_PASS’, asRendering=1 )

      cmds.setRenderPassType(renderPass, type=”BEAUTY”) # select type

      • Nathan Boyd

        Thanks for the response. I’m not quite sure how to implement this correctly. Im getting some strange drop down box options in my many failed attempts. Could you elaborate on the implementation? My coding skills aren’t quite up to the task but I’d love to get this working.

  • Daniel Osbourn

    hi, i try to make the UI but i don have any idea of python …. how that expect, i dont have any result …. you can do any video to explain how make this or anyone can copy the script for paste in script editor. thanks …. (your blog are very helpful)