Shader Programming 102

This tutorial builds on the concepts introduced in part 1. It starts by showing a simple utility shader, and then examines the shader wizard in some detail. Finally the tutorial shows how a typical phong shader works, and expands on the code to produce a new effect.

What you will need

The basic requirements for this tutorial are the same as for part one.

Terminology

There's a basic problem with the terminology - XSI, Softimage 3D, and mental ray all call the same thing by different names. I've decided to go with the names that seem the most obvious to me, aware that this will just piss everyone else off. Here's what I use:

• Shader - I define this as a compiled plug-in called during rendering. I call it a shader regardless of weather it actually shades anything, or even deals with colour at all. XSI sometimes calls this a Node and sometimes a Shader, mental ray also calls this a Shader as does Soft 3D.
• Render Tree - This is the thing you build in the render tree view in XSI. It's one or more boxes plugged into each other. XSI calls this a Shader, mental ray calls it a Phenomenon. Soft 3D doesn't really have this concept.
• Material Shader, etc. - In the render tree view, if you have an object selected, XSI provides a connection point for all the rendering attributes it defines a being part of that objects material. I call this connection point the Material Shader and I call it and everything plugged into it the Material Render Tree or the Material Tree. Similarly, if you select a camera the render tree provides a Camera Shader into which you can plug Lens Shaders. I call the whole thing a Lens Tree or a Camera Tree - showing that I'm not immune from confusion myself.

A Raytracer Within A Raytracer.

By way of re-introduction to the shader programming tools I will run through a simple utility shader. It is going to cast a new ray in the scene in the same way that mental ray shoots eye rays from the camera. It will accept the rays origin and direction as parameters, and return the colour of whatever it hits. It is like having a raytracer within a raytracer, a concept that pushes all my geek buttons.

Start the shader wizard up again. If it's been a while since you did this, the wizard is an HTML file, part of the XSI SDK. On my system it's located at C:\Softimage\XSI_3.0\XSISDK\wizards\shader\index.html.

Type raycast in the Name field and leave the Description blank. The shader should work with most render tree types, so under Shader Type check Lens, Material, Shadow, Environment, Light, Texture, and Volume. Leave everything else on this page and click Next.

The first parameter is a vector representing the ray origin. Type origin in the name field, and change the Type drop-down list box to vector. The vector needs some kind of default value, so in the Value field type 1.0 0.0 0.0 - three floating point numbers separated by spaces. Click on Add. Next change the parameter name to direction, the type and value fields should have stayed at their previous values, so click Add again. You should now have a page looking like this:

 

 

Click Next again twice to reach the final page of the wizard. Type a location for your files in the Save To: box, and click Save.

Now start up Microsoft Visual C++ and open the file raycast.dsp - it's one of the files that the shader wizard has just saved. At the bottom of the workspace window, click the File View tab, and double-click on raycast.cpp to open the source code file.

 

 

The third function down, called simply raycast(...) will contain the guts of the shader. Modify it as follows:

 

DLLEXPORT miBoolean raycast
(
	miColor * out_pResult,
	miState * state,
	raycast_params * in_pParams
)
{
	miVector origin; // the origin of the new ray
	miVector direction; // the direction it will be cast in
	miVector saved_point; // the origin value in the state, so save it first


	// read in the dialog box parameters
	origin = *mi_eval_vector(&in_pParams->m_origin);
	direction = *mi_eval_vector(&in_pParams->m_direction);

	// make sure the vectors are of unit length
	mi_vector_normalize(&origin);
	mi_vector_normalize(&direction);

	// save the bit of the state we're going to overwrite.
	saved_point = state->point;

	// substitute our parameter value
	state->point = origin;

	// trace the ray
	mi_trace_reflection(out_pResult, state, &direction);

	// restore the state from the saved value
	state->point = saved_point;


	return miTRUE;
}

            

The line that does the work is :

	mi_trace_reflection(out_pResult, state, &direction);

The normal use of the function (as the name suggests) is to bounce rays off the surface of an object to simulate reflections. The function gets the new rays origin from state->point - the point the eye ray intersected with the object. The function lets the programmer choose the direction of the reflected ray - normally you would call the built-in function:

        mi_reflection_dir(...);

to calculate the physically correct reflection direction. In this case, we are going to fire the ray in the direction given by the direction parameter, and change the value of state->point to the position given in the origin parameter. You can change some state values and not others - the mental ray documentation gives the details - but whatever you do you have to return the state variable to the way you found it at the end of your function, or you risk crashing the ray tracer.

The mi_trace_reflection() function expects both the vectors it uses to be of unit length. Obviously you can't make any predictions about what values the user will plug into any shader, so the program forces the parameters to unit length (called normalising) before they are used, with the lines:

	mi_vector_normalize(&origin);
	mi_vector_normalize(&direction);

You need to make the same changes to the compiler options as last time - Build>Options>Set_Active_Configuration and pick Win32_Release. In the Project>Settings dialog pick the C/C++ tab. In the text window at the bottom where the command string is listed, switch off the optimiser by deleting the /O2 switch.

Compile the shader using Build>Rebuild_all or by hitting F7.

Install the shader files from an XSI command prompt: Start>Programs>Softimage_Products>Softimage_XSI_3.0>Command_Prompt. Change directory to where you keep the raycast files, for example CD C:\SHADERS\RAYCAST, and run the install command xsi -i raycast.spdl.

Using The Raycast Shader

Start up XSI and create a default sphere with Get>Primitive>Surface>Sphere. Open an explorer view (press 8) and change the scope to Passes using the leftmost button along the top row. Double-click on Default_Pass to bring up the pass options dialog. We are going to put an environment shader on the pass to show the effects of the new rays being cast. Click on the Environment_Shaders tab, click the Add button and add the shader called (amazingly enough) Environment. This shader allows you to pick a image file which will be wrapped into a sphere around the scene, and called every time a ray leaves the scene without hitting anything. If you don't select an image you get the default noicon_pic.pic file, which will be good enough for our purposes.

If you draw a render region around the sphere now you should see this -

 

If so, so far so good.

Make sure the sphere is selected, and open up a script window. Type:

applyshader("raycast.preset");                  

if you use JScript, or

        applyshader "raycast.preset"

if you use VBScript.

Hit Run. The sphere will promptly disappear from the render region - don't worry about it. Open up a render tree view (press 7) and you can see why. The shader has connected itself to all sorts of points on the material shader. Un-plug it from everywhere except the material node. The sphere will probably turn a constant colour, because the shader is using the default values, which always cast a ray from a fixed point in a fixed direction, and so return a fixed colour. Try double-clicking on the raycast box in the render tree view to bring up its dialog box. If you play with the direction parameters you can change the colour, though not in a very controllable way.

Now, in the render tree view again, get a Vector State shader - Nodes>State>Vector_State. The x_State shaders give render tree access to various state variables including ray intersection points and directions.Open the shaders dialog box, and change the State Parameter to Intersection Point. Plug it into the origin parameter of our raycast shader. Get a second Vector State node, change its State Parameter to Normal Vector and plug it into the direction parameter.

 

You should see the sphere reflecting the environment.

Lets have a quick re-cap about what's happening here The camera fires eye rays into the scene, and if a ray hits nothing, the environment shader is called. If a ray hits the sphere, the raycast shader is called, which fires a new ray from the intersection point along the surface normal. This ray hits nothing, and so the environment shader gets called anyway, giving the impression of the sphere reflecting the environment.

Note that these aren't 'real' reflections. in reality the reflection ray direction depends on the viewing angle as well as the surface normal.

Try changing the Vector State plugged into the direction parameter to Eye_Ray_Vector. Now the 'reflection rays' are being cast in the same direction as the original eye rays, as if the sphere wasn't there at all. The sphere obligingly disappears!

Now we are going to mess with the ray directions a little. XSI doesn't provide much in the way of vector maths in the render tree, but it does make it easy to work with colours, and you can convert between colours and vectors quite easily.

In the render tree view, get a mix_2_colors node (Nodes>Mixers>Mix_2_Colours) and plug in into the direction of the raycast shader. XSI helpfully disconnects the old vector_state and inserts a conversion shader after mix_2_colors .

Next get a ripple shader (Nodes>Texture>Ripple) plug it into the Color_1 parameter of mix_2_colors. Open the latter's dialog box, change the Mix Mode to RGB_Intensity and change the Weight R, G, B, and A values to roughly 0.15. If you hold down the control key while moving a slider then they all move together.

You can get some interesting 'Predator' or 'The Abyss' effects going like this.

Finally, change the State_Parameter of the 'direction' Vector_State (Vector_state_1 in the picture above) back to Normal_Vector and change the Weight sliders in the mix dialog box to around 0.3.

Of course, you can do these effects in other ways, you can probably think of half a dozen, but have a play around and get a feel for the raycasting process. Trust me - this is all going somewhere. I'm the teacher, you're the student. Stop picking your nose.

The Shader Wizard Re-Visited.

Fire up the shader wizard again. It's worth going through the options in some detail, so I'll talk about all the fields of each page before doing any more programming.

• Name - This is the base name for several different things in the shader set-up. It is the root name for all the files, as well as the compiled DLL, and is also the name of the code entry point.
• Description - Anything you type here gets copied into a comment block at the top of the c++ source file. I tend to leave the documentation until tomorrow (as you may have noticed) so I rarely fill this field in.
• Help File, Help ID - This is how you specify which help file gets opened when the user clicks the ? Button at the top-right-hand corner of your shader dialog. XSI supports the windows .HLP file format, and there is a program included with MS VC++ called HCW.EXE which produces these files. The Help File field specifies the file name, and Help ID tells XSI which part of the file to look in. If you don't fill anything in here, XSI provides a default "Sorry..." page.
• Version - Every shader DLL has to include two functions - the shader entry point itself and the version function. If the shader is called my_shader the entry point function is simply called my_shader(...) and the version function is called my_shader_version(...). When mental ray first loads your shader DLL it calls this function, which simply returns an integer.

DLLEXPORT int my_shader_version(void)
{
        return 1;
}

The returned value has to agree with whatever you've typed into the version field in the shader wizard, or the shader quits with an error.

The version mechanism is supposed to stop errors caused by shader dialog boxes picking up old incompatible versions of shader DLLs and feeding them the wrong parameters. In Soft 3D I could (just about) imagine this being a problem, but XSI won't let you install a new shader of the same name without un-installing the old one, so these days versioning is a cure looking for a problem. Unfortunately it's not optional, but now you know where to look if your shader suddenly refuses to run, and the error messages talk about shader versions.

• GUID (optional) - GUID stands for Globally Unique Identifier. It's a long hex string that XSI and mental ray use to keep track of your shader and its parameters in the registry, and it is generated in such a way that it is guaranteed to be unique. You don't have to worry about them because the shader wizard will fill them all in for you. GUIDs are machine stuff, beyond the ken of us mere humans, so don't ever edit them. If you accidentally make two of them the same you may mess your registry up so badly you will have to re-install windows.
• Shader Type - This isn't actually asking what type of shader you are writing. Rather it wants you to list all the render tree types that your shader will work in. For example, if your shader uses the variable state->normal then it will only work when used in material trees - this variable is only filled in when a ray hits an object. On the other hand, some of the conversion shaders (like Color2scalar for example) will work in all kinds of render trees. The key point is to tick all the kinds of trees in which your shader will work, not just those where you intend it to be used. That way you leave the door open for people to use your work in ways you didn't anticipate.
• Shader Options - These options allow you to specify things such as your shader will only be called if a ray hits the back faces of polygons. They are provided for compatibility with Soft 3D, and I would leave them alone. It's all better done in the code itself.
• Shader Output Parameter -You can change this name and type of the shaders output here. I would recommend leaving the name. The Incremental Rendering switch is found on all shader parameters - input and output. If a parameter marked as incremental changes, and the user is using a render region, mental ray will only re-render those parts of the scene to which that parameters shader is attached. It can speed up render region previews quite dramatically, but can also cause some odd effects. It's pretty safe for material-type shaders though, and has no effect on the final rendering.

Click Next to move on to the next page.

 

The parameter page has some display bugs on my system, but anyway...

You have used this page quite a bit so far, but for the sake of completeness I'll run through the fields.

• Name - Er, the parameter name.
• Description - This isn't used anywhere, so ignore it.
• Type - Most of the types available here map onto an equivalent mental ray variable type - for example a scalar is read into the shader as an miScalar. The exceptions are:
  • filename which arrives in the shader as a char* pointer to a string containing the file name.
  • string which arrives as an miTag. Tags are pointers to mental ray database items, and mental ray stores the string there. The database is shared between all machines on the rendering network, which could contain many architecture types. Tags are handily not machine-specific, so you don't have to do any conversions. You get a database item from its tag using mi_db_access(tag) followed by mi_db_unpin(tag). See Programming mental ray for details.
  • texture which also arrives as an miTag
  • texturespace which arrives as an integer. You get the corresponding texture space from the state variable

    state->tex_list[texturespace]
    

    The user interface of the various types is illustrated below. I'd like to see what this shader does. Must be AMAZING.

  
  

 

• Value - The default that will be displayed in the dialog. If possible pick some defaults that mean the shader visibly does something when it is first applied.
• Min and Max - Many shaders controls have sliders that allow you to pick values between two limits. You can generally pick a value outside these extremes by typing the value directly into a text box along side. The Min and Max values here are the absolute minimum and maximum values that the shader will allow you to enter, even if you type into the text box. If you want a different range for the slider you can set it int a later screen. If you leave them blank, (generally a good idea unless some particular values are going to crash the shader ) the Min and Max will be automatically set according to the variable type.
• Parameter Options - The block of switches is slightly messed up in my browser, and none of them seem to be doing anything at the moment. However, I'll say what they should do in the hope that someone will fix them... The options can be changed at run-time by inserting VBScript code in the shader definition file (called a SPDL file), but that's beyond the scope of this tutorial.
  • Texturable - Displays a widget on the right hand side of the parameter allowing the user to connect a texture.
  • Writable - if un-ticked, the parameter is read-only and cannot be changed by the user.
  • Animatable - Displays a green 'set key' button to the left of the parameter.
  • Readable - I assume this is talking about whether the shader can read the parameter. I have no idea why this would be useful.
  • Persistable - If on, the parameter value is saved when you save the scene. If off, it is reset to the default each time the scene is loaded.
  • Inspectable - If on the parameter is displayed in the shader dialog. If off, it is hidden from the user.
  • Incremental Rendering - If on, when the user previews the scene using the render region, only models to which the parameters shader are connected are re-rendered.
• GUID - Another box best filled in by a computer.
• Add Button - Adds the parameter to the list.
• Reset Values - Resets all of the fields back to their starting values, but leaves the parameter list box as it is.
• Update Changes - When you select a parameter from the list in the list box, the values are displayed in the corresponding fields for editing. When you have finished you must press Update Changes to copy the fields back into the list box.
• Remove - Removes the selected parameter from the list box
• UI Info - This button opens up a scone screen of parameter options that control how the parameter is displayed:

  

  • Name - you can give the parameter a more descriptive name here for display in the dialog box.
  • Description - This is copied into the shader definition file (the SPDL file) for documentation purposes, but doesn't appear in the interface anywhere.
  • Mapping - I have no idea what this does, as it isn't documented anywhere. Perhaps someone could tell me?
  • Min, Max, Incr - These are the values that are used to set the minimum, maximum, and unit increment of the slider of there is one for the control.
  • Disabled - causes the item to be greyed out and stops the user being able to use it.
  • Exclusive - if you have grouped together a set of radio buttons or check boxes, (see later) this switch only allows one to be switched on at a time.
  • Hidden - This switch stops the parameter being displayed.
  • Type - You can set the interface to display a different type of control than the one you picked in the previous screen. Obviously some settings aren't going to make any sense (for example re-assigning a boolean value as a file browser) and the results will be undefined. I suspect the mapping string could come into play here. A common use for this is to set a colour widget to RGB rather than RGBA if the alpha slider is not used. Some types (check, combo and radio) pop up an Items listbox as shown here:

   

  

   

  These control types allow the user to pick from a list of options, presented in various ways. Unfortunately this feature is buggy and introduces an error into the SPDL file that will stop the shader installing. I don't want to go into too much detail about SPDL file hacking here, it's a topic for a tutorial in its own right. But here is what you have to do -

  Fill the Items listbox by typing the option names in the Items window and clicking Add. If you make a mistake you can get rid of on option with Remove. For Example:

   

  

   

  Then carry on with the rest of the wizard until you have saved the shader files.

  Open up the shader SPDL file (saved in the shader directory as [shadername].spdl) with a text editor and change it as follows:

   

  Defaults                         <-- look in the Defaults section
  {
          parameter                <-- this is the parameter name
          {
                  Name = "My Parameter";
                  Description = "";
                  Commands = "{F5C75F11-2F05-11d3-AA95-00AA0068D2C0}";
                  UIType = "Combo" <-- Wrongly always set to 'Combo'. change to 
                                       either 'Radio' or 'Check' if necessary,
                                       and then add a semicolon to the line end.
                  Items
                  {
                       "option_1" = 0,
                       "option_2" = 1,
                  }
          }
  }

  The Combo, Check and Radio controls allow the user to pick from a list of options which you specify in the Items listbox. For the Check and Radio types each option you enter shows up as an exclusive checkbox or radio box which the user can tick. The controls return a value of 0 if the first options box is selected, 1 for the second, etc.

  The Combo control returns a value in the same way, but displays the options differently as shown below. If you have a large number of options to display the Combo control can save you a lot of space.

  
  

   

  If you're as pedantic as me, you will have spotted that the combo control isn't actually a combo box at all. It's a drop down list box.

   



 

The next page allows you to set some more options for how the parameters will be displayed, specifically you can set up the parameters into tabbed pages, and also visually group them within a page. It's a bit twitchy, so instead of just describing the buttons, I'll give you a step-by-step example.

Here is the end result I am trying to achieve - the dialog has two tab, the first with two groups:



 

In the Layout page the Layout Name is set as Default and you can't edit it, but as it isn't displayed anywhere in the interface, so it's not a problem.

There are five parameters (named parameter_1 to parameter_5) in the shader, so the Layout page initially looks like this:

 



 

The list box on the left is where you build up the interface, the list box on the right shows the parameters you have available.

In the left hand box, pick each of the parameters in turn and delete them (using the Remove key) to give the following:

 



 

The first job is to set up the dialog tabs - for this example we'll create two, called First_tab and Second_tab.

Enter First_tab in the New Tab / Group Name Box, and click Add Tab. Do the same for Second_tab.

 



 

Next enter the groups. In the left listbox, click on the first tab line to select it, and type Group_1 in the Tab/Group Name box. Click Add Group, and do the same for a second group, Group_2. You should have the following:



 

Now, in the left listbox click on the Group "Group 1" line to highlight it, and in the right listbox select parameter_1. Click on Add Parameter. Do the same for parameter_2. Select Group 2 and add parameter_3 and parameter_4. Finally select the Tab 2 line in the left box, parameter_5 from the right, and click Add Parameter again.

The end result should look like this:

 



 

The final page of the shader wizard is the Output page.

 

 

The Shader spdl preview window shows what the final shader SPDl file will look like. This is quite useful where you're for example, writing a tutorial on the shader wizard. Apart from that, I can't see much use for it.

The Shader files to generate window lists the files that the wizard will output. As all that changes is the filenames, you are unlikely to find any surprises here.

Finally the Save To box is where you type the location for your shader files. Save them by pressing the Save button.

Wraplights.

In Siggraph 2000 there was a presentation from Disney about their (then) new film Dinosaur. During the presentation the speaker introduced the concept of wraplights to solve a problem in digital lighting.

 

 

As you can see above, there is a dark unlit band around the center of the sphere. In the real world, reflections and scattering of light would cause the lighting to tend to 'wrap' around the sphere and eliminate the dark band. Disney found that they could solve a lot of compositing problems by re-writing their renderman light shaders to simulate this wrapping, and called them wraplights.

Unfortunately in mental ray this work-around isn't possible because of a built-in optimisation. Before a light shader is called mental ray first checks the surface normal at the intersection point. If it is facing away from the light it doesn't call the light shader at all, so no wrapping will take place whatever the light shader does.

The solution is to fake the effect in the objects material. Mental ray never actually calls any light shaders itself - it relies entirely an objects material shader to do that, and a material shader can make up all sorts of lies to fool mental ray...

 

A Basic Phong Shader

The shader will be built up in stages, starting with a basic phong algorithm. Next I will add the wrapping effect, and finally add reflections and refractions using the techniques from the first part of the tutorial.

Start by firing up the shader wizard. Call the shader wrapmaterial tick the material type checkbox, but leave all the other options at their defaults.

Click Next to move on to the next page.

 

 

Add the following parameters:

Name	Type	Parameter Settings
ambient	color	red = 0.3, green = 0.3, blue = 0.3
diffuse	color	red = 0.7, green = 0.7, blue = 0.7
specular	color	red = 1.0, green = 1.0, blue = 1.0
spec_size	integer	value = 50
reflection	color	red = 0, blue = 0, green = 0
transparency	color	red = 0, blue = 0, green = 0
ior	scalar	value = 1.0
wrap_factor	scalar	value = 0

Now select each color type parameter in turn and click the UI Info button. Change the Type to RGB, and, for all except the ambient parameter, enter colour in the Name field. Hit OK.

Select the spec_size parameter, click UI Info and change the Name to the more user-friendly specular size, change the Min, Max, and Incr to 0, 300 and 1 respectively. Press OK.

Next select the wrap_factor parameter, click UI Info and change the Name to wrap factor, Min to 0, Max to 3.0, and Incr to 0.01. Hit OK.

Finally select ior, and press UI Info again. Change the Name to index of refraction, and change Min to 0, Max to 5.0 and Incr to 0.01. Press OK to finish setting the parameters.

Whenever you set minimum and maximum values in this page, always set an increment as well, or XSI gets very confused.

You should have something like this:

 

 

Click Next>> to move on to the Layout page.

The Dialog box layout will broadly copy the XSI phong dialog. In the left hand pane select the top parameter and press Remove until they have all gone. The shader dialog will have two tabs - for the first type Illumination into the New Tab/Group Name box and press Add Tab. Do the same for a second tab, called Transparency / Reflection.

Now select the Tab "Illumination" line in the left hand pane, type Diffuse in the New Tab/Group Name box, and click Add Group. Add two more groups the same way, called Specular, and Wrapping.

Select the Tab "Transparency / Reflection" line, and add two groups called (wait for it) Transparency and Reflection.

Now to add the parameters back into the dialog.

• Select the Group "Diffuse" line in the left hand pane, select the ambient parameter in the right, and click Add parameter. Do the same for the diffuse parameter.
• Select Group "Specular" and add parameters specular and spec-size.
• In Group "Wrapping" add wrap-factor.
• In the Transparency group add transparency and ior.
• In Reflection add the reflection parameter.

You should end up with this:

 

Click Next to move onto the final page, type a location for your saved files, and click Save.

 

Coding the Phong Shader

In the shader directory you have just created double-click on wrapmaterial.dsp to open up the project file in MS VC++. Double-click on wrapmaterial.cpp in the fileview of the left hand pane to open the source file.

The third function down is called wrapmaterial(...) - change it as follows:

 

DLLEXPORT miBoolean wrapmaterial
(
    miColor * out_pResult,
    miState * state,
    wrapmaterial_params * in_pParams
)
{
    // read in the dialog box values
    miColor ambient = *mi_eval_color(&in_pParams->m_ambient);
    miColor diffuse = *mi_eval_color(&in_pParams->m_diffuse);                  
    miColor specular = *mi_eval_color(&in_pParams->m_specular);                  
    miInteger spec_size = *mi_eval_integer(&in_pParams->m_spec_size);                  
    miColor reflection = *mi_eval_color(&in_pParams->m_reflection);                  
    miColor transparency = *mi_eval_color(&in_pParams->m_transparency);                  
    miScalar ior = *mi_eval_scalar(&in_pParams->m_ior);
    miScalar wrap_factor = *mi_eval_scalar(&in_pParams->m_wrap_factor);

    miColor sample_colour; // returned light colour sample
    miColor this_light; // colour contribution by this light
    miScalar spec_strength; // result of specular calculation

    miScalar dot_nl; // working variable - see text
    miVector dir; // ditto

    // Start the running total with the ambient value
    // XSI scales its ambient values by a global multiplier. We don't.
    out_pResult->r = ambient.r;
    out_pResult->g = ambient.g;
    out_pResult->b = ambient.b;

    // get the number of scene lights
    int n_lights;
    mi_query(miQ_NUM_GLOBAL_LIGHTS, state, miNULLTAG, &n_lights);
                 
    // if there are any, get an array of light tags
    miTag *light;
    if (n_lights > 0)
        mi_query(miQ_GLOBAL_LIGHTS, state, miNULLTAG, &light);
   
    // for each light
    for (int i=0; i < n_lights; i++) {
                 
        // zero the totals for this light
        this_light.r = this_light.g = this_light.b = 0;
        int samples = 0;

        // keep sampling the light shader until the function returns miFalse
        while (mi_sample_light(&sample_colour, &dir, &dot_nl, state, light[i], &samples)) {

           // use the built-in function to get the specular strength
           spec_strength = mi_phong_specular((float)spec_size, state, &dir);

           // add up the diffuse and specular components
           this_light.r += sample_colour.r * (dot_nl * diffuse.r + spec_strength * specular.r);
           this_light.g += sample_colour.g * (dot_nl * diffuse.g + spec_strength * specular.g);
           this_light.b += sample_colour.b * (dot_nl * diffuse.b + spec_strength * specular.b);
        }

        // if there was more than one light sample, divide the results back down
        if (samples > 1) {
            this_light.r /= samples;
            this_light.g /= samples;
            this_light.b /= samples;
        }

        // add the results for this light to the running total
        out_pResult->r += this_light.r;
        out_pResult->g += this_light.g;
        out_pResult->b += this_light.b;
    }

    // for now set the alpha to opaque.
    out_pResult->a = 1;

    return(miTRUE);
}

You should be able to cut and paste this directly into the file over the existing function.

The phong calculation is a lot simpler than it looks - its a straight addition of three parts

phong = ambient + diffuse + specular.

The ambient value is fixed for the object, so it is just added to the running total at the start of the function. The diffuse and specular values have to be calculated for each light in the scene, so first the function calls some mental ray database functions to get both the number of lights, and an array of database tags, one per light:

    // get the number of scene lights
    int n_lights;
    mi_query(miQ_NUM_GLOBAL_LIGHTS, state, miNULLTAG, &n_lights);
                 
    // if there are any, get an array of light tags
    miTag *light;
    if (n_lights > 0)
        mi_query(miQ_GLOBAL_LIGHTS, state, miNULLTAG, &light);

Then for each light, the function calculates the specular strength and diffuse colour of the object at the ray intersection point. This is complicated a little by mental rays 'area lights' capability. For an area light, the program samples the light value repeatedly, each time moving the intersection point slightly in 3D space ( 'jittering' ). Mental ray does much of this behind the scenes, using the function mi_sample_light(). The shader writer must call this function repeatedly, until it returns miFALSE. Each time it is called, mi_sample_light casts a light ray towards the light in question. Providing the ray doesn't hit anything along the way, the function calculates the light colour and returns:

  // keep sampling the light shader until the function returns miFalse 
  while (mi_sample_light(&sample_colour, &dir, &dot_nl, state, light[i], &samples)) 

mi_sample_light() also fills in some other handy values - dir is a vector pointing to the light, dot_nl is a measure of how much the object's surface is pointing at the light at the intersection point. Obviously if the surface is facing directly the light it will be illuminated much more than if it is facing away. The shader must multiply the returned sample colour by dot_nl to get the final diffuse colour. Lastly, mi_sample_light keeps a running total of how many times it has been called for this light in samples. The shader averages out all the returned values by adding them together and dividing by samples.

The specular strength is calculated by the function:

   // use the built-in function to get the specular strength
   spec_strength = mi_phong_specular((float)spec_size, state, &dir);

It uses the dir vector returned by mi_sample_light, and the spec_size value from the dialog box and performs its voodoo. The return value is then multiplied by the users specular colour to make the final specular term.

The diffuse and specular colours are added up for each sample by the block:

   // add up the diffuse and specular components
   this_light.r += sample_colour.r * (dot_nl * diffuse.r + spec_strength * specular.r);
   this_light.g += sample_colour.g * (dot_nl * diffuse.g + spec_strength * specular.g);
   this_light.b += sample_colour.b * (dot_nl * diffuse.b + spec_strength * specular.b);

Then the result is divided back down by the number of samples:

   // if there was more than one light sample, divide the results back down
   if (samples > 1) {
       this_light.r /= samples;
       this_light.g /= samples;
       this_light.b /= samples;
   }

Finally the values for each light are added to the running total:

   // add the results for this light to the running total
   out_pResult->r += this_light.r;
   out_pResult->g += this_light.g;
   out_pResult->b += this_light.b;

Using the Phong Shader

To compile and install the shader:

1. Save the shader program and set the Visual C++ options needed (Set Active Configuration to Win32_Release and remove the /O2 switch)
2. Install the files with xsi -i wrapmaterial.spdl
3. Fire up XSI and create a default sphere.
4. Use applyshader("wrapmaterialt.preset"); or applyshader "wrapmaterial.preset" to apply the shader to the sphere, and drag the code onto a toolbar to create a button.

You should see that the shader behaves pretty much like the built-in XSI phong shader. You may notice that the wrapmaterial is quite a bit lighter than the XSI shader if you use exactly the same values. This is because XSI scales all its ambient values by a global multiplier (set in Render>Modify>Ambience). We would have to add some extra code to take advantage of this, but it's beyond the scope of this tutorial.

 

Adding The Wrap

Now to add in the wrapping effect. Open up the wrapmaterial project in Visual C++ again. All the changes will be in the wrapmaterial.cpp file, in the function wrapmaterial().

DLLEXPORT miBoolean wrapmaterial
(
	miColor * out_pResult,
	miState * state,
	wrapmaterial_params * in_pParams
)
{
	// read in the dialog box values
	miColor ambient = *mi_eval_color(&in_pParams->m_ambient);
	miColor diffuse = *mi_eval_color(&in_pParams->m_diffuse); 
	miColor specular = *mi_eval_color(&in_pParams->m_specular); 
	miInteger spec_size = *mi_eval_integer(&in_pParams->m_spec_size); 
	miColor reflection = *mi_eval_color(&in_pParams->m_reflection); 
	miColor transparency = *mi_eval_color(&in_pParams->m_transparency); 
	miScalar ior = *mi_eval_scalar(&in_pParams->m_ior);
	miScalar wrap_factor = *mi_eval_scalar(&in_pParams->m_wrap_factor);

	miColor sample_colour;	// returned light colour sample
	miColor this_light;		// colour contribution by this light
	miScalar spec_strength; // result of specular calculation

	miScalar dot_nl;		// working variable - see text
	miVector dir;			// ditto

	void *saved_pri;		// saved value of state->pri
	miVector viewing_vec;	// vector pointing from the hit point to the camera
	miVector reflection_vec;// half way between the viewing direction and light vectors

    // Start the running total with the ambient value
    // XSI scales its ambient values by a global multiplier. We don't.
    out_pResult->r = ambient.r;
    out_pResult->g = ambient.g;
    out_pResult->b = ambient.b;

	// get the number of scene lights
	int n_lights;
	mi_query(miQ_NUM_GLOBAL_LIGHTS, state, miNULLTAG, &n_lights);
	
	// if there are any, get an array of light tags
	miTag *light;
	if (n_lights > 0)
        mi_query(miQ_GLOBAL_LIGHTS, state, miNULLTAG, &light);
	
	// save hit primative value from the state, and then set it to zero
	// this tells mental ray to ignore surface orientations
	saved_pri = state->pri;
	state->pri = NULL;

	// for each light
	for (int i=0; i < n_lights; i++) {
 
		// zero the counters for this light
		this_light.r = this_light.g = this_light.b = 0;
		int samples = 0;
		
		// keep sampling the light shader until the function returns miFalse
        while (mi_sample_light(&sample_colour, &dir, &dot_nl, state, light[i], &samples)) {

			// calculate the viewing vector
			viewing_vec = state->dir;
			mi_vector_neg(&viewing_vec);
			
			// calculate the reflection vector
			mi_vector_add(&reflection_vec, &dir, &viewing_vec);
			mi_vector_normalize(&reflection_vec);

			// specular strength
			spec_strength = mi_vector_dot(&state->normal, &reflection_vec);
			spec_strength = powf(spec_strength, (float)spec_size);

			// find the real dot_nl
			dot_nl = mi_vector_dot(&state->normal, &dir);

			// factor in the wrap
			dot_nl = (dot_nl + wrap_factor) / (1.0f + wrap_factor); 

			// discard negative values
			if (dot_nl < 0) dot_nl = 0;

			// add up the diffuse and specular components
            this_light.r += sample_colour.r * (dot_nl * diffuse.r + spec_strength * specular.r);
            this_light.g += sample_colour.g * (dot_nl * diffuse.g + spec_strength * specular.g);
            this_light.b += sample_colour.b * (dot_nl * diffuse.b + spec_strength * specular.b);
        }

		 // if there was more than one sample, divide the results back down
        if (samples > 1) {
            this_light.r /= samples;
            this_light.g /= samples;
            this_light.b /= samples;
        }

	    // add the results for this light to the running total
		out_pResult->r += this_light.r;
		out_pResult->g += this_light.g;
		out_pResult->b += this_light.b;
    }

	// restore the saved values
	state->pri = saved_pri;

	// for now set the alpha to opaque.
    out_pResult->a = 1;

    return(miTRUE);
}

			  

The changes from the previous version are in bold. Some extra variables are declared at the start, but the first
procedural change is :

 
		// save hit primitive value from the state, and then set it to zero
		// this tells mental ray to ignore surface orientations
		saved_pri = state->pri;
		state->pri = NULL; 

state->pri is a state variable which holds a pointer to which group of objects was hit by the ray - the groups are given the unlikely name of 'Hit Boxes'. However, plenty of shader calls are made without a ray hitting a primitive - for example a ray-marching volume shader calls mi_sample_light() repeatedly whilst inching along a ray path through a volume. In these circumstances, the shader sets state->pri to NULL which tells mental ray not to bother doing surface orientation calculations. This allows our shader to effectively switch off the offending optimisation which prevented light wrapping, but also means we have to do some of the donkey-work in the shader instead of leaving it to mental ray:

			// calculate the viewing vector
			viewing_vec = state->dir;
			mi_vector_neg(&viewing_vec);
			
			// calculate the reflection vector
			mi_vector_add(&reflection_vec, &dir, &viewing_vec);
			mi_vector_normalize(&reflection_vec);

			// specular strength
			spec_strength = mi_vector_dot(&state->normal, &reflection_vec);
			spec_strength = powf(spec_strength, (float)spec_size);

			// find the real dot_nl
			dot_nl = mi_vector_dot(&state->normal, &dir);

			// factor in the wrap
			dot_nl = (dot_nl + wrap_factor) / (1.0f + wrap_factor); 

			// discard negative values
			if (dot_nl < 0) dot_nl = 0;

The first half of this code calculates the specular strength. First the reflection vector is calculated. This is a vector half way between the camera direction (the viewing vector) and the light direction (the light vector which is returned by mi_sample_light() in the dir variable). To find the half-way-vector the shader simply adds the two together and normalises the result. The shader takes a measure of how close the surface normal is to the reflection vector (using mi_vector_dot() ) and then uses spec_size, the specular size variable from the shader dialog box, to adjust the specular fall-off.

The other thing the shader needs to calculate is dot_nl which would be automatically worked out by mental ray if it thought a primitive was hit, and is used in the diffuse computation. Dot_nl is the dot product of the Normal and Light vectors, and it is where the shader adds in the fiddle factor to make the lighting wrap around.

 

Using the Wrapping Shader

Save and compile the new code, and from an XSI command prompt un-install the old DLL and install then new one:

XSI -u wraplights.spdl

XSI -i wraplights.spdl

Start up XSI, create a default sphere and a couple of lights, and have a play with the wrapping effect. Here's one I did earlier. With the wrap_factor set to 0 the shader works the same as an ordinary phong. The dreaded black band is still there. Set wrap_factor to 0.4 and the lighting becomes much more natural.

 

..

 

 

Adding Reflection and Transparency

 

The final task of the shader is to deal with reflections and transparency of the object surface. The final code is shown below, with the changes marked in bold:

 

DLLEXPORT miBoolean wrapmaterial
(
	miColor * out_pResult,
	miState * state,
	wrapmaterial_params * in_pParams
)
{
	// read in the dialog box values
	miColor ambient = *mi_eval_color(&in_pParams->m_ambient);
	miColor diffuse = *mi_eval_color(&in_pParams->m_diffuse); 
	miColor specular = *mi_eval_color(&in_pParams->m_specular); 
	miInteger spec_size = *mi_eval_integer(&in_pParams->m_spec_size); 
	miColor reflection = *mi_eval_color(&in_pParams->m_reflection); 
	miColor transparency = *mi_eval_color(&in_pParams->m_transparency); 
	miScalar ior = *mi_eval_scalar(&in_pParams->m_ior);
	miScalar wrap_factor = *mi_eval_scalar(&in_pParams->m_wrap_factor);

	miColor sample_colour;	// returned light colour sample
	miColor this_light;		// colour contribution by this light
	miScalar spec_strength; // result of specular calculation

	miScalar dot_nl;		// working variable - see text
	miVector dir;			// ditto

	void *saved_pri;		// saved value of state->pri
	miVector viewing_vec;	// vector pointing from the hit point to the camera
	miVector reflection_vec;// half way between the viewing direction and light vectors

	miVector ref_dir;		// reflection direction vector
	miColor ref_colour;		// returned reflection ray colour
	miScalar trans_average; // average transparency of the colour channels


    // Start the running total with the ambient value
    // XSI scales its ambient values by a global multiplier. We don't.
    out_pResult->r = ambient.r;
    out_pResult->g = ambient.g;
    out_pResult->b = ambient.b;

	// get the number of scene lights
	int n_lights;
	mi_query(miQ_NUM_GLOBAL_LIGHTS, state, miNULLTAG, &n_lights);
	
	// if there are any, get an array of light tags
	miTag *light;
	if (n_lights > 0)
        mi_query(miQ_GLOBAL_LIGHTS, state, miNULLTAG, &light);

	// save hit primative value from the state, and then set it to zero
	// this tells mental ray to ignore surface orientations
	saved_pri = state->pri;
	state->pri = NULL;
	
	// for each light
	for (int i=0; i < n_lights; i++) {
 
		// zero the counters for this light
		this_light.r = this_light.g = this_light.b = 0;
		int samples = 0;
		
		// keep sampling the light shader until the function returns miFalse
        while (mi_sample_light(&sample_colour, &dir, &dot_nl, state, light[i], &samples)) {

			// calculate the viewing vector
			viewing_vec = state->dir;
			mi_vector_neg(&viewing_vec);
			
			// calculate the reflection vector
			mi_vector_add(&reflection_vec, &dir, &viewing_vec);
			mi_vector_normalize(&reflection_vec);

			// specular strength
			spec_strength = mi_vector_dot(&state->normal, &reflection_vec);
			spec_strength = powf(spec_strength, (float)spec_size);

			// find the real dot_nl
			dot_nl = mi_vector_dot(&state->normal, &dir);

			// factor in the wrap
			dot_nl = (dot_nl + wrap_factor) / (1.0f + wrap_factor); 


			// discard negative values
			if (dot_nl < 0) dot_nl = 0;

			// add up the diffuse and specular components
            this_light.r += sample_colour.r * (dot_nl * diffuse.r + spec_strength * specular.r);
            this_light.g += sample_colour.g * (dot_nl * diffuse.g + spec_strength * specular.g);
            this_light.b += sample_colour.b * (dot_nl * diffuse.b + spec_strength * specular.b);
        }

		// if there was more than one sample, divide the results back down
        if (samples > 1) {
            this_light.r /= samples;
            this_light.g /= samples;
            this_light.b /= samples;
        }

		// add the results for this light to the running total
		out_pResult->r += this_light.r;
		out_pResult->g += this_light.g;
		out_pResult->b += this_light.b;
    }

	// restore the saved values
	state->pri = saved_pri;
	
	// reflection - if any reflection colour set 
	if(reflection.r + reflection.g + reflection.b > 0) {

		// reduce the lighting to compensate for the reflectivity
		out_pResult->r *= 1.0f - reflection.r;
		out_pResult->g *= 1.0f - reflection.g;
		out_pResult->b *= 1.0f - reflection.b;

		// calculate the reflection direction
		mi_reflection_dir(&ref_dir, state);

		// trace a reflection ray - if nothing is hit, trace an environment ray
		if(mi_trace_reflection(&ref_colour, state, &ref_dir) || 
			mi_trace_environment(&ref_colour, state, &ref_dir) ) {

			// add the reflection results into the running total
			out_pResult->r += reflection.r * ref_colour.r;
			out_pResult->g += reflection.g * ref_colour.g;
			out_pResult->b += reflection.b * ref_colour.b;
		}
	}

	
	// transparency - if any transparent colour set 
	if(transparency.r + transparency.g + transparency.b > 0) {

		// reduce the lighting to compensate for the transparency
		out_pResult->r *= 1.0f - transparency.r;
		out_pResult->g *= 1.0f - transparency.g;
		out_pResult->b *= 1.0f - transparency.b;

		// trace a refraction ray - if nothing is hit, trace an environment ray
		if(mi_refraction_dir(&ref_dir, state, 1.0f, ior) &&
			mi_trace_refraction(&ref_colour, state, &ref_dir) || 
			mi_trace_environment(&ref_colour, state, &ref_dir) ) {

			// add the results into the running total
			out_pResult->r += transparency.r * ref_colour.r;
			out_pResult->g += transparency.g * ref_colour.g;
			out_pResult->b += transparency.b * ref_colour.b;

			// set the alpha to something sensible
			trans_average = (transparency.r + transparency.g + transparency.b) /3.0f;
			out_pResult->a = (1 - trans_average) + (trans_average * ref_colour.a);
		}
	} 
	else {

		// if the object isn't transparent, it's opaque
		out_pResult->a = 1;
	}
    return(miTRUE);
}
			  

The code for reflection and transparency is very similar, so I'll go through the reflection part in detail and then touch on the differences with transparency. The start of the reflection code is an optimisation:

// reflection - if any reflection colour set
if(reflection.r + reflection.g + reflection.b > 0) ...

The code skips reflections altogether if the sum of the reflection colour sliders is zero. Normally this will be because they are all set at zero, but will also cause reflections to stop working if for some reason the sliders are set at RGB(-1, -1, 2). If this is a problem you should change the code to specifically test for zeros. Next, a more reflective surface contributes less of its surface colour to the mix, so the illumination colour is reduced proportionately:

		// reduce the lighting to compensate for the reflectivity
		out_pResult->r *= 1.0f - reflection.r;
		out_pResult->g *= 1.0f - reflection.g;
		out_pResult->b *= 1.0f - reflection.b;

The reduction is done on the red, green and blue channels separately. I have no idea if this has any basis in reality, and it does produce some odd colouring of reflections, but it's the way the built-in shaders work, so I've followed suit.

The next line calls a mental ray function to calculate the direction of the reflection ray, which the code then fires. If the raycasting function returns miFALSE it means that no objects were hit, so the shader fires an environment ray to see if there is an environment shader. Again, a return value of miFALSE means there was no suitable shader, so the rest of the block is skipped.

If either ray returns a colour, it is added to the running total:

	// trace a reflection ray - if nothing is hit, trace an environment ray
	if(mi_trace_reflection(&ref_colour, state, &ref_dir) || 
		mi_trace_environment(&ref_colour, state, &ref_dir) ) {

		// add the results into the running total
		out_pResult->r += reflection.r * ref_colour.r;
		out_pResult->g += reflection.g * ref_colour.g;
		out_pResult->b += reflection.b * ref_colour.b;
	}

Again the colours are added channel by channel. I don't know if this mirrors reality or not, but it's the way the XSI does it. It make the process of colouring reflections quite counter-intuitive. If you want to redden the reflection you have to turn the red slider down rather than up. It's one of my (very few) criticisms of the XSI interface that the dialog boxes parameters generally make sense from a programmers view, but less so from a users point of view. It steepens the learning curve.

The transparency code is very similar, transparency rays are called refraction rays. This time the direction calculation is included in the initial if() test, because it returns miFALSE if the ray is reflected back internally into the object, in which case it will be reflected internally repeatedly, and can be ignored. If that doesn't happen, a refraction ray is fired, if that doesn't hit anything an environment ray is fired. If neither of those hit anything the rest of the block is skipped.

		// trace a refraction ray - if nothing is hit, trace an environment ray
		if(mi_refraction_dir(&ref_dir, state, 1.0f, ior) &&
			mi_trace_refraction(&ref_colour, state, &ref_dir) || 
			mi_trace_environment(&ref_colour, state, &ref_dir) )

The other main difference with a transparent object is that it is expected to return a non-opaque alpha value. If any transparency rays hit other objects, their alpha values will have to be composited. The code works out the average alpha value of the three transparent colour channels, and uses that to do the composite. I have no justification for this, I simply made it up.

	
		// set the alpha to something sensible
		trans_average = (transparency.r + transparency.g + transparency.b) /3.0f;
		out_pResult->a = (1 - trans_average) + (trans_average * ref_colour.a);

Further Work

You should be able to compile and install the finished shader without any more prompting from me. You can use this code as a foundation for other kinds of shaders. The simplest is the lambert shader, which is exactly the same but without the specular component. Several other kinds of specular calculations are also available as mental ray functions, and can be substituted for mi_phong_specular() in the basic phong code devoped earlier.

 

Conclusion

In this tutorial I developed many of the themes introduced in part 1. In particular it covered:

• A simple utility shader
• A detailed look at the shader wizard
• A full-featured 'Wrapmaterial' shader based on the phong model.

I had no idea that the tutorial was going to be so long when I started it. Had I known I would probably have done something else with the time... It is bound to have mistakes, bad grammar and inadequate explanations. Please feel free to let me know at the email address on the contact page.

Credits

This tutorial and all the associated bits and pieces were produced by David Rowntree. I would like to thank Justin King, ex of Softimage now of Electronic Arts who showed me how to do all this stuff in the first place, back in the Softimage 3D days.

(C) Copyright Nanomation Limited 2003

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

SOFTIMAGE® is a registered trademark of Softimage Inc., a wholly owned subsidiary of Avid Technology, Inc., in the United States, Canada, and/or other countries. Microsoft® and Windows NT® are registered trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks belong to their respective owners and are hereby acknowledged.

What this means is, if as a result of following these instructions your computer explodes, all your clients desert you for the competition, your business collapses, and your husband leaves you taking the kids, you can't sue me.