I don’t like to use booleans at all, but I came across this video where the guys from chaosgroup show how they used the VRayDistanceTex to make render-time booleans, similar to the renderman feature. There is no cutting or “bad” geometry tesselation, this is render-time raytracing. In this post I will show you how to do this in maya, but if you are using 3ds max you can do it as well using the same aproach.
The 3D scene
The scene setup is really simple, position your geometry like the example image, intersecting each other like if you would do a geometry boolean. Next you will need to duplicate the “cutting” object (red one) and hide it (ctrl+h).
Create a VrayMtl and assign it to the sphere(your object). Next create a vrayDistanceTex and connect it to the “opacity map” of your material. Notice that the vrayDistanceTex created a selection set called “distTex_set1″. In 3ds max you have an option to add objects, in maya you have to use selection sets. Now drag your hidden duplicated object into the set1 in the outliner.
Change your settings in the vrayDistanceTex like the above image. Set an high distance like ’99999′, both far and near color to 100% white. and the inside color to black. Remember that we are sending this values to an opacity map, so the inside color set to black means that where the sphere is “inside” the cutting object it should be completlty transparent.
Now we will create the material for the visible cutting object. Connect a new vrayDistanceTex to the opacity map of this new shader. In the outliner drag the main object, in my scene is the sphere, to the new set created by the vrayDistanceTex. Copy the settings from your first vrayDistanceTex but invert the colors like you see in the above image.
I made a simple example to follow along this tutorial, as you can see, “perfect” raytrace booleans. It’s easy to make small mistakes in this setup, so make sure:
That’s it, not very complicated but you have to follow the steps to get the correct result. You will not be using this everyday but may come in handy in some situations. Use the comments below to let me know if you have any issues and if this was usefull.