Generating Outlines in OpenGL

• Download source files - 239 Kb
• Download demo project - 11.9 Kb

Introduction

I recently had to write some code to outline 3D objects using OpenGL. I hunted around the Internet for methods to do this, and eventually came across two main techniques. The first involves the use of Polygon Offsets, and the second involves the use of the Stencil Buffer. This article explains how these two methods work, and explains in a bit more detail what the stencil buffer is.

Background

There is often a requirement in 3D graphics to outline objects on the screen. This can be used for a variety of different reasons. For example, a CAD system might outline an object in red to show that it is currently selected. Also, densely triangulated objects do not often draw well in wireframe, and an outline mode can be a way of portraying the shape of an object to a user without overcomplicated wireframe rendering. A final application could be within engineering drawing, where an outline of a 3D model could be used to automatically generate particular black and white projections of a model.

The generation of outlines uses multiple passes through the scene geometry to achieve its goal. Multi-pass rendering techniques are becoming more and more common, and are often used in games software to generate effects such as shadows. Shadows cannot be created with conventional z-buffer techniques, but can with multi-pass techniques. I'll briefly explain how this would be done at the end of the article; although I won't show any code for shadows.

The first technique I found is quick and dirty. It involves drawing the object in wireframe with the GL line style set to use very thick lines, then over-drawing the image using the background color. This leaves the edges of the thick wireframe lines that were protruding the object visible, and gives you your required outline. One of the rendering passes must be offset. I chose to pull the surfaces forward out of the screen, hence ensuring that the polygons render over the top.

The second technique uses a feature of the graphics card called the Stencil Buffer. The Stencil Buffer is like the Z-Buffer, but allows you to do per pixel tests during rendering passes. In this instance, I draw the object in the background color, writing a value into the stencil buffer every time I successfully write a pixel to the screen. Then the object is redrawn in wireframe, using thick lines, but only writing when the stencil buffer is empty.

In order to quickly get some OpenGL code into existence, I have used a library and demo application from another Code Project article. OGLTools by Jonathan de Halleux provides an object oriented framework for creating GL render contexts and handling all the Windows specific OpenGL stuff. Note that in order to use the stencil buffer, you have to make sure that you get a stencil buffer in your pixel format. I did that by hacking Jonathan's projects slightly.

Using the code

The demo application is based on Jonathan de Halleux's application. I have edited the draw loop so that it can draw in one of three ways. The first using Polygon Offsets, the second using the Stencil Buffer, and the third using the Stencil Buffer along with display lists. In the third algorithm, I have drawn the objects in a dark blue rather than the background color so that you can see the effect from that. You can select the mode from a standard menu in the application.

The first block of code shows how to draw the object using polygon offsets:

// Push the GL attribute bits so that we don't wreck any settings

glPushAttrib( GL_ALL_ATTRIB_BITS );
// Enable polygon offsets, and offset filled polygons forward by 2.5

glEnable( GL_POLYGON_OFFSET_FILL );
glPolygonOffset( -2.5f, -2.5f );
// Set the render mode to be line rendering with a thick line width

glPolygonMode( GL_FRONT_AND_BACK, GL_LINE );
glLineWidth( 3.0f );
// Set the colour to be white

glColor3f( 1.0f, 1.0f, 1.0f );
// Render the object

RenderMesh3();
// Set the polygon mode to be filled triangles 

glPolygonMode( GL_FRONT_AND_BACK, GL_FILL );
glEnable( GL_LIGHTING );
// Set the colour to the background

glColor3f( 0.0f, 0.0f, 0.0f );
// Render the object

RenderMesh3();
// Pop the state changes off the attribute stack

// to set things back how they were

glPopAttrib();

The second block of code demonstrates the use of the stencil buffer:

// Push the GL attribute bits so that we don't wreck any settings

glPushAttrib( GL_ALL_ATTRIB_BITS );
glEnable( GL_LIGHTING );
// Set the clear value for the stencil buffer, then clear it

glClearStencil(0);
glClear( GL_STENCIL_BUFFER_BIT );
glEnable( GL_STENCIL_TEST );
// Set the stencil buffer to write a 1 in every time

// a pixel is written to the screen

glStencilFunc( GL_ALWAYS, 1, 0xFFFF );
glStencilOp( GL_KEEP, GL_KEEP, GL_REPLACE );
// Render the object in black

glPolygonMode( GL_FRONT_AND_BACK, GL_FILL );
glColor3f( 0.0f, 0.0f, 0.0f );
RenderMesh3();
glDisable( GL_LIGHTING );
// Set the stencil buffer to only allow writing

// to the screen when the value of the

// stencil buffer is not 1

glStencilFunc( GL_NOTEQUAL, 1, 0xFFFF );
glStencilOp( GL_KEEP, GL_KEEP, GL_REPLACE );
// Draw the object with thick lines

glLineWidth( 3.0f );
glPolygonMode( GL_FRONT_AND_BACK, GL_LINE );
glColor3f( 1.0f, 1.0f, 1.0f );
RenderMesh3();
// Pop the state changes off the attribute stack

// to set things back how they were

glPopAttrib();

Explaining the key GL calls in a little more detail: glClearStencil is used to set the clear value for a call to glClear( GL_STENCIL_BUFFER_BIT ). This allows you to initialize the stencil buffer to any value you want. glEnable( GL_STENCIL_TEST ) is pretty self explanatory, turning on the use of the stencil buffer. The key functions are glStencilFunc and glStencilOp. glStencilFunc allows you to set a test function, specify a reference value, and a mask that will be applied to both the stencil buffer and the reference value before any test is done. The test function can consist of a variety of comparison operators, or the generic GL_NEVER and GL_ALWAYS. glStencilOp explains what to do in the event of the stencil buffer test failing, the stencil buffer test passing but the z-buffer test failing, and both the stencil buffer test passing and the z-buffer test passing. Your options here include keeping the existing value, zeroing the buffer or replacing, inverting, incrementing or decrementing the value.

Points of Interest

Finally, I would like to explain one of the many methods for using the stencil buffer and multi-pass rendering to produce shadows. The following process will produce a shadow on a ground plane from a simple object:

1. Load the view transform into the modelview matrix. This sets the system to be looking from the eye point.
2. Draw the object and ground plane, with lighting enabled.
3. Push the modelview matrix with glPushMatrix().
4. Construct a light view matrix and multiply it into the modelview matrix. The light view matrix is generated in a specific way using the equation of the ground plane, and the light position.

   void shadowMatrix(Glfloat m[4][4],
   GLfloat plane[4],
   GLfloat light[4])
   {
   GLfloat dot = plane[0]*light[0] + plane[1]*light[1] +
   plane[2]*light[2] + plane[3]*light[3];
   m[0][0] = dot - light[0]*plane[0];
   m[1][0] = - light[0]*plane[1];
   m[2][0] = - light[0]*plane[2];
   m[3][0] = - light[0]*plane[3];
   m[0][1] = - light[1]*plane[0];
   m[1][1] = dot - light[1]*plane[1];
   m[2][1] = - light[1]*plane[2];
   m[3][1] = - light[1]*plane[3];
   m[0][2] = - light[2]*plane[0];
   m[1][2] = - light[2]*plane[1];
   m[2][2] = dot - light[2]*plane[2];
   m[3][2] = - light[2]*plane[3];
   m[0][3] = - light[3]*plane[0];
   m[1][3] = - light[3]*plane[1];
   m[2][3] = - light[3]*plane[2];
   m[3][3] = dot - light[3]*plane[3];
   }

5. Set up a polygon offset so that the rendered shadow polygons do not interleave with the ground plane.
6. Render the ground plane to the stencil buffer - setting a value every time the ground plane is written.
7. Disable lighting, set the color to be dark grey, and render the object again (not the ground plane). Only render the object points if the stencil buffer is set, and whenever you render, zero the contents of the stencil buffer. This slightly complex process will ensure that shadows are only cast on the ground plane, and that the shadow polygons are only rendered once.
8. Pop the matrix and disable the polygon offset to tidy things up.

This technique, and others using multi-pass rendering, is explained in considerably more detail in an article by Mark J Kilgard of nVidia.

History

• Version 1 issued on 7 Oct 04.