Introduction to glTF for WPF 3D

• Download source code - 55.7 MB

Introduction

I got a little sidetracked on my project of solving differential equations. I really wanted to show how to calculate the trajectories of the planets in the solar system, with corrections for general relativity. Some planets won't actually have correct paths if you only use Newtons gravitational law so you need to solve some non-linear differential equations with relatively huge timesteps, which is perfect for the backwards Euler integration method.

In any event, I wanted to show the resulting planets in 3D so you could see the orbit. But different planets have different colors and makeup, so in order to separate them, I would need some coloring or equivalent ImageBrushes of textures if that was going to happen. That is when I stumbled upon NASA's 3D resources. They have all the 3D images stored in a glTF file format (*.glb) that you could download and use, but how to show these files in WPF 3D and what are these files really?

I am only going to use this tool to get some very simple 3D shapes and you have to rewrite the code for it to work in a general way.

Background

glTF stands for graphics language Transmission Format and seems to be the new standard that everyone is implementing when it comes to storing and sending 3D graphical components. The current version spec is maintained by the Khronos Group, which also publishes a lot of developer material on their Github account.

The documentation that describes the new standard 2.0 version of the exchange format is given here. A quick guide in the format can be viewed in the following quick guide pdf document.

The document type is actually quite simply arranged, taken from the quick guide:

So the code for loading in a *.glb is actually quite simple:

// Load all byte arrays from the Binary file glTF version 2
using (var stream = File.Open(filename, FileMode.Open))
{
    using (var reader = new BinaryReader(stream, Encoding.UTF8, false))
    {
        // Reading the initial data that determines the file type
        Magic = reader.ReadUInt32();
        Version = reader.ReadUInt32();
        TotalFileLength  = reader.ReadUInt32();
        
        // Read the JSON data
        JsonChuckLength = reader.ReadUInt32();
        UInt32 chunckType = reader.ReadUInt32();
        // Should be equal to JSON_hex 0x4E4F534A;         
        string hexValue = chunckType.ToString("X");
        
        JSON_data = reader.ReadBytes((int)JsonChuckLength);
        
        // Read the binary data
        BinChuckLength = reader.ReadUInt32();
        UInt32 chunckType2 = reader.ReadUInt32();
        // Should be equal to BIN_hex 0x004E4942;
        string hexValue2 = chunckType2.ToString("X");
        
        BIN_data = reader.ReadBytes((int)BinChuckLength);
    }
}

We now have extracted the JSON data and the Binary data in two different arrays. However, there is already a tool created for extracting all the information from the JSON that is available both on github and as a NuGet package. This will however only extract information on how the file is organized, what is where and so on. The actual data is either in the binary part, or in separate files.

There are a lot of resources on KhoronosGroup Github account but they are mostly for programming languages other than C#.

Extract the 3D Model

Any 3D model will have some position data that is usually organized with triangle indices, and each of these triangles will have a normal vector that gives its direction. Additionally, there might also, as in my case, be images that have some texture coordinates.

Describing what data is used in each object is given in the Meshes object that is extracted from the glTF JSON. Since I only want to show a planet, each file will only contain one mesh, but in general, as with the planet Saturn and its rings, there could be many meshes for each glb file. But for the simplicity and showing the principle, I have excluded the more difficult file types.

Each of the files will have so-called Accessors that will point to the position in the binary file where the actual information is stored. So here, I extract the information for each of the meshes (or mesh in this case).

for (int i = 0; i < glTFFile.Accessors.Count(); i++)
{
    Accessor CurrentAccessor = glTFFile.Accessors[i];
    
    // Read the byte positions and offsets for each accessors
    var BufferViewIndex = CurrentAccessor.BufferView;
    BufferView BufferView = glTFFile.BufferViews[(int)BufferViewIndex];
    var Offset = BufferView.ByteOffset;
    var Length = BufferView.ByteLength;
    
    // Check which type of accessor it is
    string type = "";
    if (AttrebutesIndex.ContainsKey(i))
        type = AttrebutesIndex[i];
        
    if (type == "POSITION")
    {
        // Used to scale all planets to +/- 1
        float[] ScalingFactorForVariables = new float[3];
        
        if (CurrentAccessor.Max == null)
            ScalingFactorForVariables = new float[3] { 1.0f, 1.0f, 1.0f };
        else
            ScalingFactorForVariables = CurrentAccessor.Max;
            
        // Upscaling factor
        float UpscalingFactor = 1.5f;
        
        Point3DCollection PointsPosisions = new Point3DCollection();
        
        for (int n = Offset; n < Offset + Length; n += 4)
        {
            float x = BitConverter.ToSingle(BIN_data, n) / 
                                   ScalingFactorForVariables[0] * UpscalingFactor;
            n += 4;
            float y = BitConverter.ToSingle(BIN_data, n) / 
                                   ScalingFactorForVariables[1] * UpscalingFactor;
            n += 4;
            float z = BitConverter.ToSingle(BIN_data, n) / 
                                   ScalingFactorForVariables[2] * UpscalingFactor;
            
            PointsPosisions.Add(new Point3D(x, y, z));
        }
        MaterialPoints = PointsPosisions;
    }
    else if (type == "NORMAL")
    {
        Vector3DCollection NormalsForPosisions = new Vector3DCollection();
        for (int n = Offset; n < Offset + Length; n += 4)
        {
            float x = BitConverter.ToSingle(BIN_data, n);
            n += 4;
            float y = BitConverter.ToSingle(BIN_data, n);
            n += 4;
            float z = BitConverter.ToSingle(BIN_data, n);
            
            NormalsForPosisions.Add(new Vector3D(x, y, z));
        }
        
        NormalPoints = NormalsForPosisions;
    }
    else if (type.Contains("TEXCOORD"))
    {
        // Assuming texture positions
        PointCollection vec2 = new PointCollection();
        for (int n = Offset; n < Offset + Length; n += 4)
        {
            double x = (double)BitConverter.ToSingle(BIN_data, n);
            n += 4;
            double y = (double)BitConverter.ToSingle(BIN_data, n);
            
            vec2.Add(new Point(x, y));
        }
        
        TexturePoints = vec2;
    }
    else
    {
        if (CurrentAccessor.ComponentType == Accessor.ComponentTypeEnum.UNSIGNED_SHORT)
        {
            for (int n = Offset; n < Offset + Length; n += 2)
            {
                UInt16 TriangleItem = BitConverter.ToUInt16(BIN_data, n);
                Indecies.Add((Int32)TriangleItem);
            }
        }
    }
}

If you have texture coordinates, there will also be images that you could load from either the binary part or in a separate file.

foreach (glTFLoader.Schema.Image item in glTFFile.Images)
{
	//var ImageType = item.MimeType;
	int BufferViewIndex = (int)item.BufferView;
	BufferView BufferView = glTFFile.BufferViews[BufferViewIndex];
	var Offset = BufferView.ByteOffset;
	var Length = BufferView.ByteLength;

	// Copy the relevant data from binary part
	byte[] ImageBytes = new byte[Length];
	Array.Copy(BIN_data, Offset, ImageBytes, 0, Length);

	// Convert to image
	MemoryStream ms = new MemoryStream(ImageBytes);
	BitmapImage Img = new BitmapImage();
	Img.BeginInit();
	Img.StreamSource = ms;
	Img.EndInit();
	Images.Add(Img);             
}

Generate WPF 3D glTF Viewer

Adding all this information to a ModelVisual3D that can be used for showing it in a Viewport3d is relatively straightforward.

// Construct the WPF 3D ViewPort model
Model3D = new MeshGeometry3D();
Model3D.TriangleIndices = Indecies;
Model3D.Positions = MaterialPoints;
Model3D.Normals = NormalPoints;
Model3D.TextureCoordinates = TexturePoints;

// Geometry model
GeoModel3D.Geometry = Model3D;
GeoModel3D.Material = new DiffuseMaterial() { Brush = new ImageBrush(Images[0]) };

// ModelVisual3D for showing this component
Visualisation.Content = GeoModel3D;

The Viewport3D is very simple and all I need is to position the camera and give some lights to the scene. All the planets are centered in origin (0,0,0).

<Viewport3D Name="viewport3D1" Width="400" Height="400">
	<Viewport3D.Camera>
		<PerspectiveCamera x:Name="camMain" 
		Position="6 5 4" LookDirection="-6 -5 -4">
		</PerspectiveCamera>
	</Viewport3D.Camera>
	<ModelVisual3D>
		<ModelVisual3D.Content>
			<DirectionalLight x:Name="dirLightMain" Direction="-1,-1,-1">
			</DirectionalLight>
		</ModelVisual3D.Content>
	</ModelVisual3D>    
</Viewport3D>

I stole some ideas on simple zooming and rotating from this site.

History

• 2^nd May, 2023: Initial version