Named Binary Tag serialization

 

• Download SourceCode - 70 KB
• Download DLL - 18.7 KB
• Download Phonebook Sample - 44.5 KB
• Download Sample - 43.7 KB

Introduction

The Named Binary Tag (also known as NBT) is a very simple file format that use binary tags to store data. This file format was created by Markus Persson for storing game data MineCraft.

Unfortunately, the NBT format is present in several compression formats:

1. Uncompressed.
2. Compressed with GZIP.
3. Compressed with ZLIB.

 

Standard tag types

TagID	Name	Payload size (Bytes)	Description
0	TagEnd	0	The propouse of this tag is indicates the end of the opened TagCompound.
1	TagByte	1	A single unsigned byte.
2	TagShort	2	A single signed short.
3	TagInt	4	A single signed integer.
4	TagLong	8	A single signed long.
5	TagFloat	4	A single signed float.
6	TagDouble	8	A single signed double.
7	TagByteArray	Variable	Byte array. This tag is prefixed with a single signed integer that indicates the size of array.
8	TagString	Variable	A UTF-8 string. The string is prefixed with a single unsigned short that indicates the size of string.
9	TagList	Variable	List of nameless tags, all tags must be same tag type. The list is prefixed with the TagID of the items it contains (just one byte), and the length of the list with a single signed integer. This list is sortable.
10	TagCompound	Variable	List of named tags. This list is not sortable and his size is variable (without prefixed length)
11	TagIntArray	Variable	Integer Array. This tag is prefixed with a single integer that indicates the size of array.

My custom tag types

To give more options to NBT file format, i created new tag types.

252	TagImage	Variable	For storing images. (System.Drawing.Image)
253	TagIP	Variable	For storing a IPAddress.
254	TagMAC	Variable	For storing a Physical Address.
251	TagSByte	1	For storing a signed Byte.
250	TagUShort	2	For storing a unsigned Short.
249	TagUINT	4	For storing a unsigned Integer.
248	TagULong	8	For storing a unsigned Long.
247	TagShortArray	Variable	Short array. This tag is prefixed with a single integer that indicates the size of array.
246	TagDateTime	8	For storing a date time value.
245	TagTimeSpan	8	For storing a time span value.
244	TagLongArray	Variable	Long array. This tag is prefixed with a single integer that indicates the size of array.
243	TagFloatArray	Variable	Float array. This tag is prefixed with a single integer that indicates the size of array.
242	TagDoubleArray	Variable	Double array. This tag is prefixed with a single integer that indicates the size of array.
241	TagSByteArray	Variable	SByte array. This tag is prefixed with a single integer that indicates the size of array.
240	TagUShortArray	Variable	UShort array. This tag is prefixed with a single integer that indicates the size of array.
239	TagUIntArray	Variable	UInt array. This tag is prefixed with a single integer that indicates the size of array.
238	TagULongArray	Variable	ULong array. This tag is prefixed with a single integer that indicates the size of array.
237	TagImageArray	Variable	Image array. This tag is prefixed with a single integer that indicates the size of array.

File format rules

1. Everything is in big-endian.
2. All NBT files must begin with TagCompound.
3. All tags begin with a single byte that indicates his tag type.
4. All tags (except TagEnd and the items in TagList), begins with a TagString.
5. All tags of TagCompound must be closed by TagEnd.

Format specifications and samples

The tags contains the following format:

TagType (TagID)

TagString (Name)

Payload

The following sample show how this format store TagShort inside a TagCompound

Theory:

TagCompound: ('Test')
{
TagShort: ('sample') : 123
}

Data on disk (hex format):

(1) 10

(2) 00 04

(3) 54 65 73 74

(4) 02

(5) 00 06

(6) 73 61 6D 70 7C 65

(7) 00 7B

(8) 00

(1) ID of TagCompound.

(2) Length of the TagCompound name.

(3) UTF-8 String ("Test").

(4) Tag ID, in this case 2 because our tag is a TagShort.

(5) Length of his name.

(6) UTF-8 String ("sample").

(7) Payload.

(8) TagEnd (indicates the end of the TagCompound).

Using the code

To use my library, it's necessary to import the following name space:

1. NBT.IO (This name space contains everything to do with the file and its compression)
2. NBT.Tags (Contains all supported tag types)

There are two namespaces (NBT.Exceptions and NBT.Compression).

NBT.Exceptions contains all exception that can throw the library, and NBT.Compression contains all related with the compression.

Inside the library - Part 1 (NBT.IO)

The namespace NBT.IO is responsible for matters relating to the treatment of the file, reading, writing, exceptions, ...

to read a file is necessary to create an instance of the class NBTFile. The NBTFile class provide the main methods for the administration of the file. It also detects automatically the compression format.

//
// Part of the code from NBTCompression Headers
//
public static NBTCompressionTypes.enumNBTCompressionTypes CompressionType(string filePath)
{
    NBTCompressionTypes.enumNBTCompressionTypes result = 
                NBTCompressionTypes.enumNBTCompressionTypes.Uncompressed;
    //We open the file and check if file have the header of GZIP
    using (Stream fStream = new FileStream(filePath, FileMode.Open, FileAccess.Read))
    {
        result = NBTCompressionHeaders.CompressionType(fStream);
    }
    return result;
}

public static bool IsGzipStream(Stream stream)
{
    bool result = false;
    if (stream == null)
    {
        throw new NBT_InvalidArgumentNullException();
    }
    if (stream.CanSeek == false)
    {
        throw new NBT_IOException("Can't seek in the stream");
    }
    //we keep the current position within the stream.
    long initialOffset = stream.Seek(0, SeekOrigin.Current);
    stream.Seek(0, SeekOrigin.Begin);
    //Check if the stream is a gzip stream.
    if ((stream.ReadByte() == GZIP_Header[0]) && (stream.ReadByte() == GZIP_Header[1]))
    {
        result = true;
    }
    //Set the position to the initial position
    stream.Seek(initialOffset, SeekOrigin.Begin);
    return result;
}

//
// Part of the code from NBTFile
//
public void Load(Stream stream)
{
    try
    {
        //Indicates if the stream will be closed after this function.
        bool closeAuxStream = false;
        if (stream == null)
        {
            throw new NBT_IOException();
        }
        //Determine the compression
        NBTCompressionType fileCompression = NBTCompressionHeaders.CompressionType(stream);
        Stream auxStream = null;
        switch (fileCompression)
        {
            case NBTCompressionType.Uncompressed:
                {
                    auxStream = stream;
                    closeAuxStream = false;
                    break;
                }
            case NBTCompressionType.GZipCompression:
                {
                    auxStream = new GZipStream(stream, CompressionMode.Decompress, true);
                    closeAuxStream = true;
                    break;
                }
        }
        if (auxStream == null)
        {
            throw new NBT_IOException();
        }
        byte firstTag = (byte)auxStream.ReadByte();
        if (firstTag != TagTypes.TagCompound)
        {
            throw new NBT_IOException("The first tag must be a TagCompound");
        }
        this.fileType = fileCompression;
        this.rootTagName = TagString.ReadString(auxStream);
        this.rootTagValue.readTag(auxStream);
        if (closeAuxStream == true)
        {
            //Close the auxStream, but the original stream still opened
            auxStream.Close();
        }
    }
    catch (Exception ex)
    {
        throw new NBT_IOException("Load exception", ex);
    }
}
public void Load(string filePath)
{
    try
    {
        if (File.Exists(filePath) != true)
        {
            throw new NBT_IOException("File not found");
        }
        using (Stream stream = File.OpenRead(filePath))
        {
            this.Load(stream);
            this.filePath = filePath;
        }
    }
    catch (Exception ex)
    {
        throw new NBT_IOException("Load exception", ex);
    }
}

Inside the library - Part 2 (NBT.Tags)

This namespace contains all tag types available. The main idea is that all tags inherit from the abstract class Tag.

Is so because the Tag class provide the minimum functions that must have all tags.

Because TagCompound is the first tag of a NBT file, NBTFile.Load() call a readTag (this function is in the TagCompound class).

Here is the explanation and the code:

internal override void readTag(Stream stream)
{
    if (stream == null)
    {
        throw new NBT_InvalidArgumentNullException();
    }
    //Clear the current content in the dictionary
    this.Clear();
    bool exit = false;
    while (exit != true)
    {
        //Read the tag ID
        byte id = TagByte.ReadByte(stream);
        //If tagID = 0 (TagEnd), is the end of the list and we close the list.
        if (id == TagTypes.TagEnd)
        {
            exit = true;
        }
        if (exit != true)
        {
            //Read the Key (unique name of the tag in this TagCompound list)
            string tagEntry_Key = TagString.ReadString(stream);
            //Read the value (the tag)
            //See bellow to see the ReadTag code
            Tag tagEntry_Value = Tag.ReadTag(stream, id);
            //Add the tag with its key to the dictionary inside the TagCompound
            this.value.Add(tagEntry_Key, tagEntry_Value);
        }
    }
}

This function is in the abstract class Tag, and his function is simply, create a instance of the tag that match with the id parameter.

internal static Tag ReadTag(Stream stream, byte id)
{
    switch (id)
    {
        case TagTypes.TagEnd:
            return new TagEnd();

        case TagTypes.TagByte:
            return new TagByte(stream);

        case TagTypes.TagShort:
            return new TagShort(stream);
                  .
                  .
                  .
                  .
    }
}

The idea is simple, each tag is responsible for loading your data from the input stream, and also to save them.

Sample code

frmMain.cs

//
// the following sample show how you can store
// a undefined number of TagStrings into the main TagCompound.
//

//We need import the library.
using NBT.Tags;
using NBT.IO;

public partial class frmMain : Form
{
    //We create a instance of NBTFile to manage the data file.
    NBTFile nbtFile = new NBTFile();
    //Path where we found the nbt file.
    string filePath = Application.StartupPath + @"\test.nbt";

    public frmMain()
    {
        InitializeComponent();
    }

    private void frmMain_Load(object sender, EventArgs e)
    {
        if (File.Exists(this.filePath) == true)
        {
            //We open the file using the function LoadFromFile, if you don't use
            //a file, because you use a stream, you can use the function LoadFromStream.
            this.nbtFile.Load(this.filePath);
            //Reload the list
            this.ReloadList();
        }
    }

    private void btnSave_Click(object sender, EventArgs e)
    {
        //Save the current data into the specified file.
        this.nbtFile.Save(this.filePath);
    }

    private void btnAdd_Click(object sender, EventArgs e)
    {
        //Save a new TagString into the main TagCompound. (The key must be unique)
        this.nbtFile.RootTag.Add(this.txtKey.Text, new TagString(this.txtValue.Text));
        this.ReloadList();
    }
    private void ReloadList()
    {
        this.lstItems.Items.Clear();
        //Retrieve all items stored in the main TagCompound
        foreach (KeyValuePair<string,> item in this.nbtFile.RootTag)
        {
            //Check if the tag is a TagString to retrieve its value,
            //but it isn't necessary if you use ToString()
            if (item.Value.GetType() == typeof(TagString))
            {
                ListBoxItem lstItem = new ListBoxItem();
                lstItem.Text = ((TagString)item.Value).value;
                lstItem.Tag = item.Key;
                this.lstItems.Items.Add(lstItem);
            }
        }
    }

    private void btnDelete_Click(object sender, EventArgs e)
    {
        if (this.lstItems.SelectedItems.Count > 0)
        { 
            ListBoxItem selectedItem = (ListBoxItem)this.lstItems.SelectedItem;
            //Delete the selected key
            this.nbtFile.RootTag.Remove((string)selectedItem.Tag);
            this.ReloadList();            
        }
    }
}

public class ListBoxItem
{
    private string visibleText = "";
    private object itemTag = null;
    public string Text
    {
        get
        {
            return this.visibleText;
        }
        set
        {
            this.visibleText = value;
        }
    }
    public object Tag
    {
        get
        {
            return this.itemTag;
        }
        set
        {
            this.itemTag = value;
        }
    }
    public ListBoxItem()
    {

    }
    public override string ToString()
    {
        return this.visibleText;
    }
}

My free graphical tool to edit any NBT file.

You can download directly following this link to test your own NBT files. Download NBT Maker from my Skydrive (It's freeware)

Possible usages

I recently made a number of programs that use this format to store data. Among them a wake on lan program, that store the computers in directories using the TagCompound.

Conclusion

I think this format, although very simple, is quite powerful because it allows any data store. Furthermore, the fact that it is organized by name and sub ​​directories is a great feature that should be taken present to store data hierarchically.

History

• 27 July, 2012:
  • Initial release
• 11 August, 2012:
  • Added new 4 tag types (TagSByte | TagUShort | TagUInt | TagULong)
• 13 October, 2012:
  • Added new 3 tag types (TagShortArray | TagDateTime | TagTimeSpan)
  • The Load/Save functions are overloaded
• 26 December, 2012
  • Added new 8 tag types: (TagLongArray | TagFloatArray | TagDoubleArray | TagSByteArray | TagUShortArray | TagUIntArray | TagULongArray | TagImageArray)
  • Fixed minor bugs
• 26 January, 2013:
  • New sample added (NBT Phonebook - with contact image support)
• 31 March, 2013:
  • Fixed the TagImageArray bug
• 25 May, 2013:
  • Added the equality function
  • Added null protection in each tag array while writing in the nbt file
• 18 Oct, 2014:
  • Fixed TagCompound clone
  • Fixed TagEnd equals
  • Updated to Microsoft .Net 4.5.1
  • Added ZLib Compression
• 16 Feb, 2015:
  • Fixed problem when compressing data using ZLib with an emtpy tagString
  • Some minor changes

Related links

• Minecraft Coalition Wiki - NBT