Shader Programming 101

This tutorial will show you how to write simple mental ray material shaders for Softimage XSI.

What you will need

You need the latest version of the Microsoft C++ compiler - at the time of writing that is version 6. It's a good idea to be up to date with service packs and all that kind of stuff. You will also (obviously) need a copy of XSI, and you need to have installed the SDK. If you haven't, you can run setup (Programs>Softimage_Products >XSI_3.0>Setup), click Modify and tick the SDK box.

I'm afraid I've written the entire tutorial from a Windows perspective. If anyone wants to send me some additional instructions for Irix or Linux I'll happily include them.

How material shaders work

Mental ray works by shooting a series of eye rays from the camera into your scene and testing to see if a ray hits any of the scene objects. If so, the program works out where the ray touches the object, finds the objects material shader, and calls it passing a bunch of information about the state of the scene. The material shader then decides what the surface colour should be at that point, and passes the information back to mental ray.

A material shader typically does complex calculations involving the objects surface characteristics and the orientation of the point hit by the ray with respect to the scene lights. It may also fire new rays itself to simulate reflections and transparency. In this first tutorial the shader will return the same colour no matter where the ray hits the object. It will produce the same effect as the built-in constant shader. Then you will modify the code to produce some more interesting effects based on the scene state information passed by mental ray.

The Shader Wizard

Start by firing up the Shader Wizard. It's part of the SDK, and is located (at least on my machine) at C:\Softimage\XSI_3.0\XSISDK\wizards\shader\wizard.htm.

It will ask you if it's ok to install some java code, and then display the opening screen.

Start by filling in the shader name - call it basic_material. Leave the description, Help File, Help ID, leave the version set at 1, and ignore the GUID field. From the Shader Type checkboxes, select only Material. Leave all the other options and click Next.

XSI Shader Wizard

The next screen defines the parameters that the user can pass to the shader from the dialog box. In this simple shader there's only going to be one parameter - the object colour. type col in the parameter name field. A description is probably superfluous given that there's only one parameter, so leave it. Select color as the parameter type. The next sets the initial values that will be displayed in the dialog box when it's first displayed to the user. Set the red and alpha values to 1.0, the green and blue values to 0.0. Leave all the check boxes at their defaults, and click Add. The window will update to show the new parameter. You can go back and change any of these parameters later, but remember to click Update Changes afterwards. Click Next when you have finished.

XSI Shader Wizard

The next screen lets you set various options that affect the way the parameters are displayed. For now, click Next to accept the defaults.

XSI Shader Wizard

The final wizard screen lets you review the work you have done, and output the resulting files. The Save button doesn't follow the normal windows convention of popping up a File Save dialog box, you have to type a location for the files in the Save To: text box next to it. If the directory doesn't exist you'll be given the option to create it.

Click Save.

The function also doesn't report any errors - such as a blank Save To: box - so it's worth checking that your files have been created before you close the wizard.

Now fire up windows explorer and navigate to the directory you just saved in. Double click on basic_material.dsp - this is the shaders project file for MS Visual c++. The compiler will start up. In the left hand pane, click the File View tab, and expand the file tree to find basic_material.cpp. Double click on this and the file will open for editing in the right hand pane.

The file defines four functions - basic_material_version; basic_material_init; basic_material; and basic_material_exit. We are going to modify basic_material. This is the function called by mental ray every time a camera ray hits our object. It is worth having a look at this function in detail now.
DLLEXPORT miBoolean basic_material
(
    miColor * out_pResult,
    miState * state,
    basic_material_params * in_pParams
)
{
    /* put your code here */
    return miTRUE;
}
Shaders compile into DLLs, so the function type has to be DLLEXPORT, The return value is type miBoolean - generally you will return miTrue to mental ray if the shader completes without an error. The ray tracer passes three parameters to your shader - out_pResult is where you will store the surface colour you're going to compute. state is a huge structure containing all the information mental ray has about the state of the scene, the ray that hit the object, and the object itself. The final parameter is in_pParams - a structure containing the parameters passed from the shader dialog box. This structure is defined in basic_material.h - have a quick look at that file now. The relevant part is:
typedef struct
{
   miColor m_col;
} basic_material_params; 
As you can see, the structure contains one variable called m_col of type miColor. The shader wizard prepends m_ to the start of all the parameter names, so even though you named the parameter col in the shader wizard, you have to remember to call it m_col in the code. Change the basic_material function as follows:
DLLEXPORT miBoolean basic_material
(
    miColor * out_pResult,
    miState * state,
    basic_material_params * in_pParams
)
{
    *out_pResult = *mi_eval_color(&in_pParams->m_col);
    return miTRUE;
}
The line that does the work is:
 *out_pResult = *mi_eval_color(&in_pParams->m_col);
The shader calls the mental ray function mi_eval_colour() to read in the values from the dialog box. All input values are read in this way - the function decides if the dialog box slider values should be read directly, or if another shader is plugged into the parameter in the render tree, in which case that shader is retrieved and called. All this happens transparently without any further action by the shader programmer, and makes life very simple. The function takes as its parameter the address of our m_col variable and returns the address of the result of its calculation.

Note that any variables or temporary values created in the shader will be created on the stack, so the return values must be explicitly copied to out_pResult. It is not enough to simply copy the address of the mi_eval_color() return value, as the contents will be destroyed when the shader returns, leading to garbage in the shader output.

Compiling The Shader

You will have to set a couple of options in the Visual C++ GUI before the shader will compile. On the Tools menu select Options and click the Directories tab. Under Show Directories For: select Include files and double click the first blank line at the end of the list of directories. Click the [...] pushbutton, and enter the location of the SDK include files. On my system, this is:

C:\SOFTIMAGE\XSI_3.0\XSISDK\INCLUDE

Again under Show Directories For: select Library Files and add a line pointing the linker to the libraries directory. On my system this is:

C:\SOFTIMAGE\XSI_3.0\XSISDK\LIB\NT-X86

Next, switch off debug mode. The default for the compiler is to include about 200k of debug information in the library, but unless you have a real masochistic streak I would recommend not trying to debug DLLs in the debugger, so from the Build menu select Set_Active_Configuration>Win32_Release.

Finally, check that all optimisations are turned off. The optimiser can break your shader in a variety of subtle and interesting ways. Turn it off by opening the Project>Settings dialog box, selecting the C/C++ tab, and in the text box at the bottom find the option /O2 and delete it. You can always turn it back on later, but it pays to get the shader working without it first.

Now compile the project with Build>Rebuild_All. You may get a warning about conflicting libraries, but I have never found it to be a problem. If it annoys you, turn it off by adding /nodefaultlib:LIBC.LIB to the linker options.

Installing The Shader

Open an XSI command prompt (Start>Program>Softimage_Products>XSI_3.0>Command_Prompt), and change directory to wherever you saved the shader files. XSI can install shaders either as addons or by running XSI -i from the command line. We will do the latter, so type

XSI -i basic_material.spdl

This file is the one that describes the shader and its dialog box to XSI. It was produced automatically by the Shader Wizard, and must be in the same directory as the shader DLL. The install program opens up another command prompt window telling you what it has done. Make a note of where it puts the file basic_material.preset, and you will need to find it again in a moment. After a few seconds the installer will finish. Press a key to close the window again.

Using The Shader

Fire up XSI and create a default NURBS sphere. Select Get>Material>More... and in the file selector dialog box navigate to where the basic_material.preset file you noted earlier was stored and pick it. Drag a render region around the sphere and you should see something like the following:

Try moving the sliders to check that the sphere changes colour. Have you ever had so much fun?

In general it is a pain navigating through the user hierarchies trying to find shader presets. There are two ways round this - either drag the preset onto a custom toolbar - a large black square appears on the toolbar, and clicking it will apply the shader to whatever is selected at the time. The black square isn't particularly intuitive though, especially if you have several presets on the same toolbar. An alternative (and better) method is to create a script button running the command

ApplyShader "basic_material.Preset"

More on all that later.

The State Structure.

Earlier I touched on the state structure, and said it was how mental ray passes information about the state of the scene to your shader. The structure is defined in shader.h, which is #include-d at the beginning of your code. Highlight the text shader.h in the include statement, right click it, and select Open Document shader.h.. The structure miState is defined about half way down the file. There's no room here to describe the structure in detail - see Programming Mental Ray (Th.Drietmayer and R Herken) for more information.

You will need this book if you are serious about writing shaders. There is an html version in XSI_3.0>doc>mental_ray>manual but a printed version will save you a lot of time. I think there used to be a pdf version on the install disks - I'm not sure if it's still there.

We are going to modify the basic_material shader to read some information from the state structure and make pretty pictures with it.

Close down XSI, open up Visual C++ again, and re-load the project file basic_material.dsw. Modify the basic_material function as follows:
DLLEXPORT miBoolean basic_material
(
    miColor * out_pResult,
    miState * state,
    basic_material_params * in_pParams
)
{
    out_pResult->r = state->normal.x;
    out_pResult->g = state->normal.y;
    out_pResult->b = state->normal.z;
    out_pResult->a = 1.0f;
    return miTRUE;
}
The new code takes the surface normal direction of the object at the point hit by the ray, and copies it into the output colour returned by the shader. The normal vector is made up of x, y, and z components, each in the range -1.0 to +1.0. Mental ray treats any negative colour values as if they were zero, so I just copy the values straight into the r, g, and b components of the output colour.

The alpha is set to 1.0 to make the object completely opaque. The ray tracer always expects material shaders to return an alpha value, and may make decisions based on it. Make sure you always return something sensible.

Compiling and installing the new shader code

Compilation should now be very straightforward - simply hit F7 or pick Rebuild All from the Build menu. Before installing the shader you need to un-install the previous version. In an XSI command prompt window, change directory to the shader code folder, and type:
xsi -u basic_material.spdl
xsi -i basic material.spdl
to uninstall the old version and re-install the new one.

Make sure you have shut down XSI before doing the install / uninstall process. If you get errors saying that XSI is unable to delete or install libraries it probably means that you have the program still running.

Shader programming involves a lot of closing down and re-starting XSI, because it doesn't un-link the shader DLLs when it has finished rendering - perhaps for performance reasons. I have asked Softimage to consider adding an UnlinkShaderDLLs button somewhere, but it's a kind of obscure request, so I'm not holding my breath.

Using the new shader

Start up XSI again and create a default polygon sphere. Open up a script window (click the scroll button next to Playback on the bottom bar) and type:
ApplyShader "basic_material.Preset"
or if you use jscript (which if you're a programmer you really should do) type:
ApplyShader ("basic_material.Preset");
Drag a render region around the sphere and have a look from various angles.

As you can see, the sphere is red in the direction of positive x, green in the direction of positive y, and blue in the direction of positive z - check the XYZ arrows at the bottom left of the picture. Have a look on the back of the sphere as well.

While you have the script window open, select the command you have just typed in and drag it onto a custom toolbar - either the one below the palettes on the command bar, or create a new one using View>Custom_Toolbars>New_Toolbar. Name the button Basic Material and name the command basicMaterialCommand. Change the name of the script file as well, to basicmaterialcommand.vbs or basicmaterialcommand.js. This is my preferred method of using custom shaders, and if you package shaders as addons you can include the toolbar and buttons as well.

The normal vectors displayed by the shader above are actually the interpolated normals. Mental ray has assumed that the sphere is to be smooth shaded, and has 'guessed' the values after taking into account the normals of the actual polygons that make up the geometry, the discontinuity setting of the Geometry Approximation property, etc. If you want to see the actual normals of the polygons, Close down XSI, re-open the basic_material.cpp and change the basic_material function to:
DLLEXPORT miBoolean basic_material
(
    miColor * out_pResult,
    miState * state,
    basic_material_params * in_pParams
)
{
    out_pResult->r = state->normal_geom.x;
    out_pResult->g = state->normal_geom.y;
    out_pResult->b = state->normal_geom.z;
    out_pResult->a = 1.0f;
    return miTRUE;
}
Re-compile the shader, then back to the command prompt:
xsi -u basic_material.spdl
xsi -i basic material.spdl
Now start up XSI again, create a default sphere and click the Basic_Material button on your custom toolbar.

Now you can see each individual facet. Try it on NURBS objects too.

Notice how some of the polygons seem to be darker at the top and lighter at the bottom? In fact they're not - cover up the polygons immediately above and below a suspect one and you can see it's actually an even colour all over. It is an optical illusion called (incredibly) the Craik-Obrien-Cornsweet Effect. There's an explanation of it here.

A Third Variation.

Another easy change you can make involves barycentric coordinates. They are difficult to explain concisely, but are a by-product of the normal interpolation process. Mental ray converts all geometry into triangles before rendering actually begins, because triangles are much more efficient to process than other polygons or NURBS surfaces.You can define a point inside a triangle in terms of its distance from each of the three vertices - these distances are the barycentric coordinates. When a material shader is called the ray intersection points' barycentric coordinates inside its particular triangle are stored in the state structure in the array bary[0..2]. These coordinates are in the range 0.0 - 1.0, making them prime candidates for copying into a colour structure...

Again, open up basic_material.cpp, and find the basic_material function. Modify it as follows:
DLLEXPORT miBoolean basic_material
(
    miColor * out_pResult,
    miState * state,
    basic_material_params * in_pParams
)
{
    out_pResult->r = state->bary[0];
    out_pResult->g = state->bary[1];
    out_pResult->b = state->bary[2];
    out_pResult->a = 1.0f;
    return miTRUE;
}
Make sure XSI is closed down, and a quick re-compile, uninstall / install. Fire up XSI again and apply the shader to a default sphere:

As you might be able to see, the barycentric coordinate of a point for each triangles vertex is 1.0 at the vertex itself, and 0.0 at anywhere along the triangle edge opposite the vertex. This property is quite useful, because it provides an easy way of displaying the results of mental rays triangulation process and displaying a wireframe view of the resulting triangles.

A Wireframe Shader

Begin a new shader, by starting up the Shader Wizard again. If you still have the wizard open you can restart it by hitting the browser Reload button .
call the shader wireframe, and in the Description field try and write something more interesting than I did. The only other thing to set on this page is to tick the Material box under Shader Type. Click Next to move on.

There are three parameters for this shader.

Call the first wire_colour, leave the description but change the Type to color. set the Red, Green, Blue and alpha values to 1.0, and leave everything else set at the default. Click Add to add the parameter, which appears in the dialog window.

The second parameter is called object_colour. Leave the Description. If the name fully describes the parameter it's pointless adding an identical description. Set Type to color again, and this time set the Red, Green, and Blue values to 0.0, though set the Alpha to 1.0 for the moment. Leave everything else and click the Add button again.

Finally type wire_width for the parameter Name, but set the type to scalar. You can see that the options for the parameter have now changed to those appropriate for a scalar value. Set the Value to 0.1, Min and Max to 0.0 and 0.1 respectively. Click Add to finish off the parameters.

If you make any mistakes, you can always highlight the parameter in the window, and correct anything. Remember to click Update Changes to save your corrections. There is also a Remove button if you want to get rid of a parameter and start again. Click Next.

There's nothing to do on this page, so click Next again.

Type the location you want the files saved to in the Save To: box. The directory will be created for you if it doesn't exist.

Now open up an explorer window, navigate to the folder you just saved in, and double-click on the file wireframe.dsp. The Microsoft compiler will start up. Click on the File View tab under the left hand pane, and double click on wireframe.cpp in the source files branch to open it for editing.

Notice how all the function have been named to reflect the new shader name. Near the top of the file there is an include statement for wireframe.h. Right click on this filename, and select Open document wireframe.h to open the header file. Again, about half way down the file the parameters for this shader are defined:
typedef struct
{
    miColor m_wire_colour; 
    miColor m_object_colour; 
    miScalar m_wire_width; 
} wireframe_params;
Notice again that the parameter names have been prepended with m_.

Close the header file and locate the function called wireframe in the source file wireframe.cpp. change it as follows:
 DLLEXPORT miBoolean wireframe
 (
     miColor * out_pResult,
     miState * state,
     wireframe_params * in_pParams
 )
 {
     miColor wire_colour, object_colour;
     miScalar wire_width;
     bool close_to_the_edge;
     // read in the dialog box values
     wire_colour = *mi_eval_color(&in_pParams->m_wire_colour);
     object_colour = *mi_eval_color(&in_pParams->m_object_colour);
     wire_width = *mi_eval_scalar(&in_pParams->m_wire_width);
     // decide if the intersection point is close to an edge.
     close_to_the_edge = (state->bary[0] < wire_width ||
                          state->bary[1] < wire_width ||
                          state->bary[2] < wire_width);
     // if so, output the wire colour...
     if(close_to_the_edge) 
         *out_pResult = wire_colour;
                 
     // ...otherwise output the object colour
     else
         *out_pResult = object_colour;
     return miTRUE;
}
The shader starts by reading in the users input values from the dialog box. There are two calls to mi_eval_color() and one to mi_eval_scalar(). Each data type that can be passed to a shader has its own mi_eval_xxx function. The possible types are boolean, integer, scalar, vector, transform, color, and tag.

Next the code decides if the ray intersection point is close to the triangle edge, by testing to see if any of the three barycentric coordinates are close to zero. The user has a slider to control just how close to zero the edge is assumed to be - and so how thick the wire will be drawn.

If the edge test is passed, the shader returns the wire colour, if not, it returns the object colour.

Finally the shader returns miTRUE to tell mental ray that the code completed without error.

Compiling and Installing The Wireframe Shader

As you are working on a new project, some of the settings have been reset. Specifically the optimiser is turned on again - switch it off using Project>Settings>C/C++ and delete the /O2 option in the Project Options: window.

Compile the shader using Build>Rebuild All or hitting F7.

Open up an XSI command prompt, change directory to the location of the shader files, and install it with the command
xsi -i  wireframe.spdl

Using The Wireframe Shader

Start up XSI and create a default sphere. Open up the script window and type:
ApplyShader "wireframe.Preset"
or
ApplyShader ("wireframe.Preset");
If you use VBScript or JScript respectively. Drag a render region around the sphere.

Have a play with the parameters to get a feel for how it works. There is an obvious problem with the wire widths - they vary with triangle size - still, I quite like it.

The shader as it stands doesn't do any of the surface calculations you would expect from a material shader. It basically returns one of two preset colours. You could add code to shade the sphere properly - but why re-invent the wheel? The need to write huge monolithic shaders has passed with the advent of the render tree. Because you used mi_eval_xxx() functions to read in the shader parameters, the render tree approach of plugging and un-plugging shaders works automatically.

Open up a render tree window in XSI, and try plugging in phong or lambert shaders to the wire_colour and object_colour parameters. Try texturing the wire_width parameter.

Here's one I made earlier...

Improving The Wireframe Shader

There are a couple of problems with the wireframe shader. Firstly, the wire width problem, but that's really a limitation of the algorithm, so I'm not going to worry too much about it.

Secondly, the code is written rather inefficiently. Both the wireframe and object colours are read in at the start of the code, but as the shader only returns one colour or the other on each call, it only really needs to read the one it's going to use. It may seem like a small point, but there are a couple of shader performance considerations. A material shader is called every time a ray intersects with your geometry. This could run into tens of thousands of calls, so even a small performance hit could have drastic consequences for the scene rendering time. Also you have no idea what the users will plug into your parameters in the render tree- they may attach a huge shader network that takes ages to run, and won't appreciate it being called more than necessary.

Summary

In this tutorial I showed you how to write some simple material shaders. You looked at:

The Shader wizard
Setting up the Microsoft Visual C++ compiler
Compilation and installation
Using the new shaders in the render tree.
The next installment (whenever I get around to writing it) will look at the Shader Wizard in more detail, and introduce some different types of shaders.

Credits

This tutorial and all the associated bits and pieces were produced by David Rowntree. I would like to thank Justin King, ex of Softimage now of Electronic Arts who showed me how to do all this stuff in the first place, back in the Softimage 3D days. It was also his idea to use the barycentric coordinates to make a wireframe shader, and I'd like to thank him for letting me use it in this tutorial.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU Free Documentation License".

SOFTIMAGE® is a registered trademark of Softimage Inc., a wholly owned subsidiary of Avid Technology, Inc., in the United States, Canada, and/or other countries. Microsoft® and Windows NT® are registered trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks belong to their respective owners and are hereby acknowledged.

What this means is, if as a result of following these instructions your computer explodes, all your clients desert you for the competition, your business collapses, and your husband leaves you taking the kids, you can't sue me.