Loading and Rendering Milkshape 3D Models with Animation and Skinning

• Download ViewAppletCode - 622.09 KB
• Download MilkshapeViewerXNA (C# version with XNA) - 984.6 KB

Introduction

Skeletal animation is a technique used in 3D rendering which uses an exterior shell composed of vertices and an internal skeleton for the animation. To avoid breaking the model, each vertex is assigned a set of bones and weights used to compute its final position based on the animation key frames.

In modern application, this technique is used frequently. Autodesk Media and Entertainment is one of the companies that produces software used in films, commercials and computer games. Among its current products, we can find Maya and 3DStudio MAX. The max file format is not very well documented and the 3Ds is just there for legacy purposes. In some cases, if you export and re-import from 3DSMax, the animation and models simply break. Autodesk offers developers SDK support for implementing their own exporters and importers but this takes a lot of time. On the other hand, MS3D is a complete documented file format with source code examples, in Milkshape 3D is a very rapid and easy to use application, making it easy to load and render their native file format. So if you want 3D modeling for your application, you can simply turn to Milkshape for creating and animating your objects.

The code is written in JAVA, but I also have a C# version with XNA or DirectX Managed. If you want the code, just let me know and I’ll email it or maybe just make it public.

Further reading on skeletal animation and skinning can be done at http://en.wikipedia.org/wiki/Skeletal_animation. I recommend that you familiarize yourself more with this technique before you get started on the code.

Topics Covered by this Article

• JOGL (java opengl)
• Skeletal animation
• Milkshape 3D model loading and rendering
• GLSL
• Hardware skinning

Specifications

The application includes the following features:

• Vertex manipulation in GLSL
• Loading and rendering Milkshape files
• Skeletal animation
• OpenGl rendering

Technical Information

• Code written in JAVA Netbeans
• Uses JOGL for rendering
• Uses GLSL for hardware skinning

Requirements

• Netbeans
• OpenGL enabled video card with GLSL support (tested on Nvidia, hardware skinning fails on ATI and older Nvidia chipsets)
• Milkshape 3D
• JOGL

Using the Application

A demo of the application can be found at http://inline.no-ip.org/testing/java/ViewApplet2 (software skinning version) or http://inline.no-ip.org/testing/java/ViewApplet3 (hardware skinning version).

Using the Code

The provided code is meant to work in Java enabled browsers as a Java applet. The skeletal animation class is called MilkshapeModel and the constructor receives an absolute URL from which to download the required files. Textures have to be in the same directory as the model. The Boolean parameter represents the hardware skinning flag. If it’s set to true, it tries to download and compile the GLSL VertexShader for hardware skinning.

model = new MilkshapeModel(new URL
	("http://localhost:8080/WebApplication/dwarf2.ms3d"), drawable.getGL(), true);

The Basic Milkshape 3D File Format is:

• Header
  • "MS3D000000" followed by version number (ver. 3 or 4)
• Vertex Data
  • Coordinates for the vertices
• Triangle Data
  • Pointers to vertices, as well as surface normals
• Group Data (object/mesh)
  • Group name and pointers to triangles
• Material Data
  • Color details
• Bone data
  • Animation data

The supported file format is version 4. The model loader is based on the C++ binary milkshape loader example provided by http://chumbalum.swissquake.ch/.

Skeletal Animation

In this example, skeletal animation takes place in two steps. The bone setup in which the key frame interpolation is computed and matrices are updated and the vertex update, where vertex positions are obtained using attached bones and weights. This is done either software or hardware by using a shader. Software takes a lot of computing power and is dependent on the number of vertices. Also if we add more models in the same scene, the slowdown is visible. Shaders execute much faster and use GPU power, so we can use the CPU for other operations.

In our case, we have a hierarchy of bones, each having its own set of key frames. These give rotation and translation data. The first step is to locate two key frames that contain the given frame and interpolate the result. Also we need to concatenate results with the parent joint. Joints are preordered in the array so now when we evaluate a Joint, we know we have already evaluated its parent.

//bone evaluation
for (int i = 0; i < numJoints; i++) {
    EvaluateJoint(i, frame);
}

private void EvaluateJoint(int index, float frame) {
    MilkshapeJoint joint = Joints[index];

    //
    // calculate joint animation matrix, this matrix will animate matLocalSkeleton
    //
    float[] pos = {0.0f, 0.0f, 0.0f};
    int numPositionKeys = (int) joint.positionKeys.length;
    if (numPositionKeys > 0) {
        int i1 = -1;
        int i2 = -1;

        // find the two keys, where "frame" is in between for the position channel
        for (int i = 0; i < (numPositionKeys - 1); i++) {
            if (frame >= joint.positionKeys[i].time &&
        frame < joint.positionKeys[i + 1].time) {
                i1 = i;
                i2 = i + 1;
                break;
            }
        }

        // if there are no such keys
        if (i1 == -1 || i2 == -1) {
            // either take the first
            if (frame < joint.positionKeys[0].time) {
                pos[0] = joint.positionKeys[0].key[0];
                pos[1] = joint.positionKeys[0].key[1];
                pos[2] = joint.positionKeys[0].key[2];
            } // or the last key
            else if (frame >= joint.positionKeys[numPositionKeys - 1].time) {
                pos[0] = joint.positionKeys[numPositionKeys - 1].key[0];
                pos[1] = joint.positionKeys[numPositionKeys - 1].key[1];
                pos[2] = joint.positionKeys[numPositionKeys - 1].key[2];
            }
        } // there are such keys, so interpolate using hermite interpolation
        else {
            MilkshapeKeyFrame p0 = joint.positionKeys[i1];
            MilkshapeKeyFrame p1 = joint.positionKeys[i2];
            MilkshapeTangent m0 = joint.tangents[i1];
            MilkshapeTangent m1 = joint.tangents[i2];

            // normalize the time between the keys into [0..1]
            float t = (frame - joint.positionKeys[i1].time) /
        (joint.positionKeys[i2].time - joint.positionKeys[i1].time);
            float t2 = t * t;
            float t3 = t2 * t;

            // calculate hermite basis
            float h1 = 2.0f * t3 - 3.0f * t2 + 1.0f;
            float h2 = -2.0f * t3 + 3.0f * t2;
            float h3 = t3 - 2.0f * t2 + t;
            float h4 = t3 - t2;

            // do hermite interpolation
            pos[0] = h1 * p0.key[0] + h3 * m0.tangentOut[0] +
        h2 * p1.key[0] + h4 * m1.tangentIn[0];
            pos[1] = h1 * p0.key[1] + h3 * m0.tangentOut[1] +
        h2 * p1.key[1] + h4 * m1.tangentIn[1];
            pos[2] = h1 * p0.key[2] + h3 * m0.tangentOut[2] +
        h2 * p1.key[2] + h4 * m1.tangentIn[2];
        }
    }

    float[] quat = {0.0f, 0.0f, 0.0f, 1.0f};
    int numRotationKeys = (int) joint.rotationKeys.length;
    if (numRotationKeys > 0) {
        int i1 = -1;
        int i2 = -1;

        // find the two keys, where "frame" is in between for the rotation channel
        for (int i = 0; i < (numRotationKeys - 1); i++) {
            if (frame >= joint.rotationKeys[i].time &&
        frame < joint.rotationKeys[i + 1].time) {
                i1 = i;
                i2 = i + 1;
                break;
            }
        }

        // if there are no such keys
        if (i1 == -1 || i2 == -1) {
            // either take the first key
            if (frame < joint.rotationKeys[0].time) {
                AngleQuaternion(joint.rotationKeys[0].key, quat);
            } // or the last key
            else if (frame >= joint.rotationKeys[numRotationKeys - 1].time) {
                AngleQuaternion(joint.rotationKeys[numRotationKeys - 1].key, quat);
            }
        } // there are such keys, so do the quaternion slerp interpolation
        else {
            float t = (frame - joint.rotationKeys[i1].time) /
    (joint.rotationKeys[i2].time - joint.rotationKeys[i1].time);
            float[] q1 = new float[4];
            AngleQuaternion(joint.rotationKeys[i1].key, q1);
            float[] q2 = new float[4];
            AngleQuaternion(joint.rotationKeys[i2].key, q2);
            QuaternionSlerp(q1, q2, t, quat);
        }
    }

    // make a matrix from pos/quat
    float matAnimate[][] = new float[3][4];
    QuaternionMatrix(quat, matAnimate);
    matAnimate[0][3] = pos[0];
    matAnimate[1][3] = pos[1];
    matAnimate[2][3] = pos[2];

    // animate the local joint matrix using: matLocal = matLocalSkeleton * matAnimate
    RecConcatTransforms(joint.matLocalSkeleton, matAnimate, joint.matLocal);

    // build up the hierarchy if joints
    // matGlobal = matGlobal(parent) * matLocal
    if (joint.parentIndex == -1) {
        //memcpy(joint.matGlobal, joint.matLocal, sizeof(joint.matGlobal));
        for (int k = 0; k < joint.matLocal.length; k++) {
            System.arraycopy(joint.matLocal[k], 0,
    joint.matGlobal[k], 0, joint.matLocal[k].length);
        }
    } else {
        MilkshapeJoint parentJoint = Joints[joint.parentIndex];

        RecConcatTransforms(parentJoint.matGlobal, joint.matLocal, joint.matGlobal);
    }
}
private void RecConcatTransforms(float in1[][], float in2[][], float out[][]) {
    out[0][0] = in1[0][0] * in2[0][0] + in1[0][1] *
        in2[1][0] + in1[0][2] * in2[2][0];
    out[0][1] = in1[0][0] * in2[0][1] + in1[0][1] * in2[1][1] + in1[0][2] * in2[2][1];
    out[0][2] = in1[0][0] * in2[0][2] + in1[0][1] * in2[1][2] + in1[0][2] * in2[2][2];
    out[0][3] = in1[0][0] * in2[0][3] + in1[0][1] *
        in2[1][3] + in1[0][2] * in2[2][3] + in1[0][3];
    out[1][0] = in1[1][0] * in2[0][0] + in1[1][1] * in2[1][0] + in1[1][2] * in2[2][0];
    out[1][1] = in1[1][0] * in2[0][1] + in1[1][1] * in2[1][1] + in1[1][2] * in2[2][1];
    out[1][2] = in1[1][0] * in2[0][2] + in1[1][1] * in2[1][2] + in1[1][2] * in2[2][2];
    out[1][3] = in1[1][0] * in2[0][3] + in1[1][1] *
        in2[1][3] + in1[1][2] * in2[2][3] + in1[1][3];
    out[2][0] = in1[2][0] * in2[0][0] + in1[2][1] * in2[1][0] + in1[2][2] * in2[2][0];
    out[2][1] = in1[2][0] * in2[0][1] + in1[2][1] * in2[1][1] + in1[2][2] * in2[2][1];
    out[2][2] = in1[2][0] * in2[0][2] + in1[2][1] * in2[1][2] + in1[2][2] * in2[2][2];
    out[2][3] = in1[2][0] * in2[0][3] + in1[2][1] *
        in2[1][3] + in1[2][2] * in2[2][3] + in1[2][3];
}

Hardware Skinning

After this, each vertex needs to be transformed according to the bones and weights. Milkshape uses a maximum of 4 bones per vertex, which is perfect for our vertex shader because we can store both bone ids and bone weights in two vec4 structures as vertex attributes.

GLSL Vertex Shader

attribute vec3 normal;
attribute vec4 weight;
attribute vec4 index;
attribute float numBones;
uniform mat4 matGlobal[100];
uniform mat4 matGlobalSkeleton[100];
uniform vec4 color;
uniform vec4 lightPos;
float DotProduct(in vec4 x, in vec4 y) {    
return x[0] * y[0] + x[1] * y[1] + x[2] * y[2];
}
void VectorRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]);
    rez[1] = DotProduct(in1, in2[1]);
    rez[2] = DotProduct(in1, in2[2]);
}
void VectorIRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = in1[0] * in2[0][0] + in1[1] * in2[1][0] + in1[2] * in2[2][0];
    rez[1] = in1[0] * in2[0][1] + in1[1] * in2[1][1] + in1[2] * in2[2][1];
    rez[2] = in1[0] * in2[0][2] + in1[1] * in2[1][2] + in1[2] * in2[2][2];
}
void VectorTransform(in vec4 in1,in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]) + in2[0][3];
    rez[1] = DotProduct(in1, in2[1]) + in2[1][3];
    rez[2] = DotProduct(in1, in2[2]) + in2[2][3];
}
void VectorITransform(in vec4 in1, in mat4 in2, out vec4 rez) {
    vec4 tmp; 
    tmp[0] = in1[0] - in2[0][3];
    tmp[1] = in1[1] - in2[1][3];
    tmp[2] = in1[2] - in2[2][3];
    VectorIRotate(tmp, in2, rez);
}

void main(){
    vec4 transformedPosition = vec4(0.0);
    vec3 transformedNormal = vec3(0.0);
    vec4 curIndex = index;
    vec4 curWeight = weight;
    for (int i = 0; i < int(numBones); i++)
    {
        mat4 g44 = matGlobal[int(curIndex.x)];
        mat4 s44 = matGlobalSkeleton[int(curIndex.x)];
        vec4 vert = vec4(0);
        vec4 norm = vec4(0);
        vec4 tmpNorm = vec4(0);
        vec4 tmpVert = vec4(0);
        VectorITransform(gl_Vertex, s44, tmpVert);
        VectorTransform(tmpVert, g44, vert);//g
        vert[3]=1;
        transformedPosition += vert * curWeight.x;

        vec4 preNormal=vec4(normal.xyz,1);
        VectorIRotate(preNormal, s44, tmpNorm);
        VectorRotate(tmpNorm, g44, norm);
        norm[3]=1;
        transformedNormal += vec3(norm.xyz) * curWeight.x;

        // shiftam sa avem urmatoarele valori pentru pondere si indice de os
        curIndex = curIndex.yzwx;
        curWeight = curWeight.yzwx;
    }
    gl_Position = gl_ModelViewProjectionMatrix * transformedPosition;
    transformedNormal = normalize(transformedNormal); 
    gl_FrontColor = dot(transformedNormal, lightPos.xyz) * color;
} 

The code above is also implemented in software version for those who don’t have GLSL support.

The GLSL is just for demonstration purposes. It doesn’t handle very well and wastes a lot of memory on the uniform matrices, failing on a lot of video cards because of this. However, if you want to test it, use a model with fewer bones and adjust the size of the matrices to suit your needs.
We animate the model using AdvanceAnimation:

if (lastTick == 0) {
            lastTick = Calendar.getInstance().getTimeInMillis();
        }
        long currentTick = Calendar.getInstance().getTimeInMillis();
        long delta = currentTick - lastTick;
        lastTick = currentTick;
        //model.AnimationFPS=120;
        model.AdvanceAnimation((float) delta / 1000);

Conclusion

This article is an introduction to OpenGL software and hardware skinning using Java and Milkshape3D model loading. It’s not optimized, but you can use a lot of it in your applications. Hope this helps and if you make any good changes to the code, please let me in on it :).