A Spritesheet Library and Editor for Win2D

Disclaimer

Introduction

Within the Universal Windows Platform world, there exists a Windows Runtime API called Win2D that provides access to Direct2D within .NET applications. If you’re looking to build a game and don’t mind navigating the difficulties of building an app within the comparatively inflexible requirements of UWP (such as being limited to specific minimum versions of Windows 10), you’ll likely find Win2D a useful addition to your project.

However, while Win2D provides several useful classes to perform drawing and transforms, it still requires a lot of work on your part to tie it into a game’s logic engine. In this article, I detail the library I’ve written to help make it easier to go from a ‘logic-only game’ to displaying your game objects on-screen, as well as the editor I’ve built to help transition from a bitmap spritesheet into usable in-game graphics resources.

The project downloads and sideloadable app can be found on my Github repository.

Definitions

This library is most helpful if your use-case fits the following description of a ‘logic-only game’:

You have the logic engine written, with classes representing different in-game objects. Your engine is likely capable of solving conflicts, performing actions, synchronizing across server and clients, writing and loading from save states, and simulating time passing either via an external or internal timer.

Importantly, your engine maintains state information about all objects that are active, and there’s some way to access this state information from outside your engine. You may or may not have logic that can help dictate how the state should be represented in a visual engine.

Your game does not depend on live ray-casting or other graphics tricks to run. This sprite engine does not support collisions natively, but you could add it in yourself.

Your engine is compatible with the Universal Windows Platform runtime, either via .NET Standard 2.0+ or some other conversion.

Notably, your game does not have to exist explicitly within a 2D space; however, the engine does not have any 3D graphics built-in.

Project Origin

To help you better understand if this is a library, you can immediately use, or if you should heavily modify it to suit your purposes, I’d like to take a moment to explain why I developed this library.

I’m in the process of writing a game of my own, from absolute zero upward. This has been a decade-long trial, starting from when I barely understood the difference between Classes and Modules (in VB.NET) until now. Over time, I found that I focused too hard on trying to make the game visually playable, leading to faulty game logic and lots of dead ends. As the 2020 pandemic hit full-force, I took the opportunity to start over (again!) and build a thorough, complete engine to run the game on first.

My game’s ‘logic-only’ engine is structured as such:

I have one class that serves as the base for every in-game object, whether it be the grid-based floor tiles, the vegetation and environment, or the player. Unique sub-classes describe important object distinctions and specifications. All objects are accessible at the same level (to a degree) i.e., within a ‘Region’, without requiring navigation through nested children (although it’s preferred). A ‘Region’ contains anywhere from single-digit to six-digit numbers of tiles, each containing a theoretically infinite number of child objects. Every child object is assigned an unsigned long as its unique identifier.

With the logic in place to run the world, I needed a way to start displaying my object states. Extending the base object to include drawing logic was untenable; I wanted to use this logic engine for both the server and the client, and having all the extra overhead was an immediate disqualification. Because I started with the logic first and not the graphics, I couldn’t use the Unity approach of having the visual object ‘own’ the logic states, i.e., a prefab with a script containing fields and members. I needed a sprite engine that could load in a bunch of sprites along with their metadata, so that I didn't have to hard-code in the different filenames, locations of different images, and so on.

The SpriteAsset Engine

The engine is comprised of just three working parts: the SpriteBundle, the Tokens, and the Sprites.

There’s more going on in the background, especially with the editor, but we’ll get to that later.

The SpriteBundle contains the logic required to select one of possibly several frames in order to best describe the state of the object. Whether your sprite is an animated series of frames, the same object viewed from different angles, or just a single static image, the SpriteBundle internally solves for the state you request and gives you the correct image. This means you don’t have to save the images under complex filenames or reference them through long strings – you simply use Tokens.

From every SpriteBundle, you can generate Tokens. Each Token contains state information about what image should be displayed, and should be tied to any in-game object that is be displayed. For animated sprites, the Token maintains the timing and frame selection; similarly, for directional and overlay sprites, the Token contains information that dictates which of the many frames within the Bundle should be shown. In this way, the texture/sprite/image is only loaded once into memory, and every in-game object simply references them when they’re required.

In order to maintain the link between Tokens and their game object, an abstract Sprite class is introduced. There are two ways you can implement it: either by creating your own derivative Sprite class that contains logic for matching the object’s state with the Token, or by extending your base game object’s class by inheriting Sprite, so that there’s direct access to the states and state changes.

For my game, I created a derivative that has a field containing the ‘Source’ in-game object where the state would be extracted from, and the Token that contained the image to be drawn. Before every frame refresh, the derivative Sprite checks if the Source has had a significant state change and modifies the Token in accordance before returning the correct image.

Within the examples I’m providing, I’ve included some of the code that shows how I implemented this, from which you should be able to adapt to fit the shape of your engine.

Win2D’s Canvas

While Win2D has a lot of documentation on their github.io page, as a novice, I found myself struggling to understand the overall structure of the API at the start. There are a lot of classes to work with, and some of their constructors aren’t well-documented enough to intuitively explain how to use them. I’ll attempt to assuage some of those concerns here as I explain how to tie in the sprite engine to the Win2D controls.

The basic control you’ll need to use is the Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl. This provides immediate mode 2D rendering, and is a good starting point. A few events are emitted by this control that will be useful to us, namely the CreateResources event and Draw event. I’ve hooked up the CreateResource event to my sprite loading code, which reads the saved sprites and loads them into memory. Draw, on the other hand, is where you can start placing bitmaps onto the canvas to be shown. I’ll go over the sprite files a little later in this article.

You’ll notice that the abstract Sprite class provides four useful properties you can use; the Token, the IntendedRect, Image, and Clickable. Depending on how complex you want to make your drawing process, you can implement these however you want – the simplest approach is simply to make IntendedRect return a rectangle equivalent to your logic engine’s position state, and for Image to poll a Token’s Bundle (see below). Clickable lets you define a region for the sprite to respond to if it was clicked – its implementation should be pretty straightforward.

For example, here’s a summarized version of my ESprite (no in-game object equivalent) and EOSprite (tied to in-game EObject) implementation:

Public Class ESprite
    Inherits Sprite

    Public Overrides Property Image As CanvasBitmap
    Public Overrides Property Clickable As Clickable
    Public Property Token As Token

    Private _Intended As Rect

    Protected Sub New()
    End Sub

    Public Sub New(sprite As Token, unit As Integer, Optional click As Clickable = Nothing)
        Token = sprite
        Image = sprite.Source.ApplyToken(sprite)
        ' modify click bounds
        If click IsNot Nothing Then
            click.ClickBounds = IntendedRect(unit)
            Clickable = click
        End If
    End Sub

    Public Overridable Sub SetRect(intended As Rect)
        _Intended = intended
    End Sub

    Public Overrides ReadOnly Property IntendedRect(unit As Integer) As Rect
        Get
            Dim ret = _Intended
            If Clickable IsNot Nothing Then Clickable.ClickBounds = ret
            Return ret
        End Get
    End Property

    Public Overrides Sub Invalidate()
        Image = Token.Source.ApplyToken(Token)
    End Sub
End Class

Public Class EOSprite
    Inherits ESprite

    Public ReadOnly Source As EObject

    Public Sub New(source As EObject, sprite As Token, unit As Integer,
                   Optional click As Clickable = Nothing)
        ' uses the protected sub new() with no parameters to prevent accessing 
        'Source before assignment.
        MyBase.New()
        If source Is Nothing Then
            Throw New Exception("EOSprite cannot have a null source. Use ESprite instead.")
        End If
        Me.Source = source
        Token = sprite
        Image = sprite.Source.ApplyToken(sprite)
        ' modify click bounds
        If click IsNot Nothing Then
            click.ClickBounds = IntendedRect(unit)
            Clickable = click
        End If
    End Sub

    ''' <param name="unit">The pixels that represent one in-game Address unit.</param>
    Public Overrides ReadOnly Property IntendedRect(unit As Integer) As Rect
        Get
            Dim ret = New Rect(Source.Address(False).X * unit,
                            Source.Address(False).Y * unit,
                            Image.Bounds.Width, Image.Bounds.Height)
            If Clickable IsNot Nothing Then Clickable.ClickBounds = ret
            Return ret
        End Get
    End Property
End Class

You’ll note that the method to retrieve an image from the SpriteBundle is as simple as doing Token.Source.ApplyToken(Token). This call is required because the SpriteBundle does not maintain a collection of all the Tokens it has created, in order to prevent memory leaks. Attempting to apply a token on a SpriteBundle that wasn’t created by that Bundle's CreateToken() will throw an exception, so don’t try to sneak in a swap that way.

The return for ApplyToken() is of a CanvasBitmap, which is immediately compatible with your CanvasControl. In order to draw a CanvasBitmap onto your control, hook into your control's Draw event:

Public Sub Canvas_Draw(sender As CanvasControl, args As CanvasDrawEventArgs)

You can trigger a new Draw event by using a DispatcherTimer to repeatedly call CanvasControl.Invalidate() to force it to refresh its contents.

The second parameter, a CanvasDrawEventArgs, contains a single CanvasDrawingSession under CanvasDrawEventArgs.DrawingSession. You can access the DrawImage() method in this session to draw your image directly onto the canvas with a few different options. Notice that one of the parameters you can provide is the destinationRectangle – you can grab this from Sprite.IntendedRect(units), if you’ve implemented it properly; otherwise, you might need to apply some math to your IntendedRect to move and scale it properly.

The intention behind IntendedRect (no pun inte- wait!) is to have the Sprite automatically feed information about where on the canvas it expects its Image to be drawn. If your game uses a coordinate system, the units should usually be the size of your sprites. If you want to scale the size of all of your sprites, use the CanvasDrawingSession.Transform property to do so instead.

Included in the library, additionally, is the SceneSession class. If you only have a couple of sprites dancing around your screen at a time like Pong (two paddles and a ball, maybe some text), you probably won’t need to use the SceneSession. However, if your game is like mine and has potentially thousands of sprites active, you’ll probably want to keep those sprites organized in some other way, especially if you have layers to think about. This is where SceneSession comes in.

The SceneSession

Within the SceneSession class is an internal sorted dictionary that maintains a number of SingleLayer objects. Each SingleLayer is, as named, a single layer. While the sprites within the layer may overlap with one another based on the order of their addition, they can be grouped to draw above or below other layers.

Each layer maintains its own internal canvas, drawing to that first before yielding the entire canvas as a CanvasRenderTarget (compatible with CanvasDrawingSession.DrawImage()) to be drawn onto the main canvas.

To use a SingleLayer, you can either use your own collection to draw on the SingleLayer’s Session property, or you can extend the class to have its own List/Dictionary to hold your sprites. After instantiating a SingleLayer, add it to the SceneSession to have it automatically draw in the correct order – afterward, you can simply modify the sprite’s state to change its appearance. The sprite’s Clickable should also be added to the SingleLayer, if you’re looking to use the click functionality I’ve provided.

Finally, within the Draw event handler method, simply make a call to the SceneSession’s Start() method by passing in the aforementioned CanvasControl’s DrawingSession:

Public Sub Canvas_Draw(sender As CanvasControl, args As CanvasDrawEventArgs)
    Session.Start(args.DrawingSession, New Rect(0, 0, Canvas.ActualWidth, Canvas.ActualHeight))
    ...
End Sub

This tells the Session to dump each layer, in order, onto the Canvas’ DrawingSession. If you make an extended SingleLayer class, make sure that you override its Reset() method to paint your sprites onto the Layer’s Session.

Finally, you’ll note that both SceneSession and Sprite implement the ITakesTime interface. This interface simply exposes a single method, Step(), that is used to let every sprite know that a certain amount of time has passed between ticks, allowing timing to be propagated. If your sprite has an animation sequence of, say, 50 milliseconds per frame, you’ll need to provide this delta-time to inform the Token that time has passed.

The resultant logic-graphics structure should look something like this. Your logic engine will perform ticks, track the amount of time that has passed, and pass that information into your Sprite implementation - either through a direct Step() method call, or through an extended SingleLayer (i.e., a new layer class that maintains its own collection of sprites) - to register animations and movements. For any sprites that should have a visual change, you apply its Token to the Token's source SpriteBundle to retrieve an updated image. This image is then drawn onto the Canvas' DrawingSession for that particular frame.

The Editor

Now, how do we go about creating the sprites for your game?

The first step is to create a spritesheet. For the elusive programmer-artist (like myself), this is no big task – but even if you have no drawing experience, pixel art isn’t going to be that tough. First, you’ll want to determine your pixel size – say, 16x16 (always keep them square when possible). Using software like Paint or Photoshop, or some freeware like Krita or Paint.NET, you can zoom all the way in and draw your tiny little frames. You can organize the spritesheet in whatever manner you’d like – the editor I’ve built can handle it.

Once you have a few images on your spritesheet, you can open the image in the SpriteEditor’s Bounds Editor view via Load Sprite. The spritesheet is displayed on the left, with the ‘Bounds’ listed on the right. You can add Bounds by using the menu on the right, making sure to hit the ‘Modify Bounds’ button to commit changes, or you can simply double-click on the image to start a Bounds. Do this to mark out all of your individual frames. The order which you do this in does not matter to the editor – just do it however you feel comfortable.

Once you’ve marked all your Bounds, you can flip over to the Sprite Editor view.

You’ll see an empty list of Sprites on the left, the same Bounds you highlighted previously, and an empty box of Assignments-Properties.

Create a sprite by clicking Add Sprite. Then, select a Bounds you want to include in the sprite, and click Insert Selected.

The Bounds is now added as a known image within the sprite.

There are four (at the time of writing) ‘MultiSprite’ types; Single, Directional, Neighbor, and Animated. The Description box explains how to fill out the Assignment/Property boxes for each bounds within the sprite; read these carefully, as a faulty sprite will throw exceptions and the editor does not (yet) perform these checks automatically.

• Single sprites are a single, static image associated with the Class Identifier. Adding additional bounds will not affect how this sprite behaves.
• Directional sprites utilize the SpriteAsset.DirectionalSpriteDirections enumerator to select one of several images that best represents the sprite. If you use your own ‘direction’ class in your game, a translation will be required. A limitation to using this is that only 8 directions are supported; if you need more, you'll have to implement your own adaptation.

  

• Neighbor sprites add overlays to a base sprite based on what’s nearby; for instance, an image of a cobblestone path can be overlaid with grass particles on the left if there is a grassy patch there. This helps blend the environment a little better.

  

• Animated sprites contain a number of subset animations, based on their assignment names. As such, they can be used to express objects with multiple states, by naming each state within the sprite and assigning any arbitrary duration. The primary use, however, is to display a sprite that appears animated.

  

Once you’re satisfied with your settings, you can Save the sprite into a .cmplx file. This is a JSON-formatted document that contains all the details you’ve specified, as well as the byte data of the image. If you update your spritesheet, you can use the Swap Image button to reload an updated or new spritesheet without having to redraw all the Bounds and Sprites.

Token Selection

So how do you go about setting a Token’s state to get the right image?

Tokens contains several methods and a few fields, all of which are visible at all times. In order to keep things relatively incomplex, I opted to keep all fields under a single Token class rather than create yet more classes. The methods provided are what I call ‘Selector’ methods.

Each Selector method is prepended with the letter of their multitype-counterpart; the Animation-type SpriteBundle should make use of A_Select() and A_Time(), the Neighbor-type uses N_Select(), and so on. If you accidentally use an incorrect selector method, the Token will throw an exception. Due to this, it is advised that you create unique Sprite-derivative classes for each multitype, or at a minimum ensure that you’re cross-referencing your code to the ComplexSheet.

Names and Variants

As an aside, I wanted to quickly explain the rationale behind having two names.

Consider if you had a Tree object with some complex animations in the TREE sprite. You then add a field within your Tree class that determines how tall it is; say, an enumerator of Short, Medium and Tall. Instead of going into the TREE sprite and adding in all of the animation frames under difficult assignment names like ‘waving_tall’ or ‘waving_short’, you can simply apply the variant name to get a TREE:TALL sprite instead. This way, when you’re internally trying to call for an animation via Token, you can keep your old string constants.

It’s the difference between (using a theoretical function FindBundle()):

Token = FindBundle("TREE")
Token.A_Select("waving" + 
    If(Height = TreeHeight.Tall, "_tall", 
        If(Height = TreeHeight.Short, "_short", "")))

and:

Token = FindBundle("TREE", TreeHeight.ToString().ToUpper())
Token.A_Select("waving")

Granted, you might like the look of an If… structure (if that’s the case, you’re free to do that, too!), but otherwise, the variant option helps clean things up a bit by reducing the number of bounds contained within a sprite.

The benefit of using variants, in addition, is that if a variant isn’t found, you can simply fall back on the original. Otherwise, depending on what type of multisprite you’ve set, the SpriteBundle may throw an exception if an invalid assignment name is selected.

The Files and Classes

A ComplexSheet contains the image bounds you selected, as well as the list of sprites. However, at this stage, the sprites are not actually serialized as sprites. Rather, the data you specified during the Sprite Editor phase is stored within a package called MultiSpriteData.

Each MultiSpriteData object contains the Bounds you selected per sprite, the assignments and properties, and the names. These are all compiled together when saving the ComplexSheet. The result is a JSON document that can then be deserialized back into a ComplexSheet.

To retrieve the sprites in your game, you need to first re-build the ComplexSheet from file by loading the file into your app, read the entire file as a string to parse it back into JSON, and then pass the parsed JSON object into a ComplexSheet’s constructor. To do this, you’ll have to import Newtonsoft’s JSON library – if you have a library that you prefer, you can modify the deserialization code to fit your project.

Public Async Function LoadFromFiles(fromDir as String) As Task(Of Boolean)
    Dim appFolder = Windows.ApplicationModel.Package.Current.InstalledLocation
    ' note that fromDir must be below the InstalledLocation by UWP limitation.
    Dim assetsFolder = Await appFolder.GetFolderAsync(fromDir)
    Dim assets = Await assetsFolder.GetFilesAsync()
    For Each file In assets
        If file.FileType <> ".cmplx" Then
            Continue For ' skip unknown file
        End If
        Dim data = Await Windows.Storage.FileIO.ReadTextAsync(file)
        ' Newtonsoft.JSON.Linq.JObject
        Dim json = JObject.Parse(data)

        Dim sheet As New ComplexSheet(json)

        For Each msdata In sheet.Sprites
            Dim bundle As New SpriteBundle(msdata, sheet)
            ... ' handle the bundle however you need to.
        Next

    Next
    Return True   
End Function

After instantiating the ComplexSheet, you can extract the SpriteBundles from it by accessing the Sprites field of the ComplexSheet, and passing those into the SpriteBundle’s constructor, as shown above.

The SpriteBundle of the given MultiSpriteData builds itself based on the type of MultiSprite you selected, populates the image collection by examining the Image within the ComplexSheet, and is now ready to generate useful Tokens.

From there, you can decide how you want to organize your SpriteBundles. I’ve wrapped mine around a SpriteHandler class that provides fallback options for when a particular sprite isn’t found.

You can also load multiple ComplexSprites without issue; just do the same extraction process as above and make sure you can organize all the different SpriteBundles.

Depending on how your logic engine is organized, you may be able to rely purely on your class names to seek a particular sprite; my Player class, during resource loading, is tied to the sprite named PLAYER. You can use any string you want to match with the sprites, but note that the Class name and Variant name of sprites are always capitalized.

Considerations

Aside from the points mentioned in the Definitions section, there are a couple of more considerations I want to raise.

The performance of this engine is, as far as I can tell, acceptable. The ten-odd 16x16 sprites placed on a region of 40 by 40 tiles (1600 spaces) consumes about 200MB of memory in my debug build, including my game engine. Win2D does a really good job of keeping draw times down, and I was achieving about 4ms per frame on an NVidia 960M. If you require incredibly high performance, you may need to tweak the Token and SpriteBundle classes to reduce their footprint as much as possible - as they are, I've done my best within my limited knowledge to clean things up, but I'm sure you could find things I've missed.

The editor stores a copy of the imported bitmap within the saved .cmplx file. It does this by converting the raw bitmap bytes of the entire image, uncompressed, as a string. A .png with ten-odd 16x16 images on a 1024x1024 bitmap only costs about 6KB, and at no compression costs 20KB. The editor's .cmplx file costs over 5MB. A bitmap of this size can store 1024 separate 16x16 images, and the Bounds/MultiSpriteData within the JSON document won't add a significant amount of data in comparison - but if your spritesheets are much larger than this, you may have to deal with very slow JSON parsing.

The Clickable class currently only supports Left/Right mouse clicks.

Finally, I wrote this editor and sprite engine in order to fulfill the needs of my game. It likely will not provide everything you may need for your own game, but I hope that, if you're still in a drafting or prototyping stage, it can help accelerate your development.

Final Summary

In this article, I’ve described a spritesheet engine for Win2D that can help organize your image resources, as well as the SpriteEditor for creating complex sprite packages from a bitmap file. I’ve gone over the structure for implementing the engine into your game, as well as some design choices and patterns you should follow.

I will continue to work on refining the editor and library, but I have hopefully established the basis enough that future development and additions won't break your code, if you decide to try it.

You can find the library and editor project files and sideloadable app at my Github repository.

Bonus Hints

When working with UWP and Win2D, I came across a bunch of weird circumstances that didn’t have explanations or sufficient documentation online. I’m going to provide them here, in case you find it useful.

• UWP’s FilePicker class can’t be displayed unless there is at least one filetype filter. If you don’t add at least one filter, a weird COM exception is thrown.
• CanvasBitmap can’t load from filenames that aren’t ‘below’ the executable’s directory, so load a StorageFile and use its stream instead.
• In order for a SoftwareBitmap to be used as a source for creating a CanvasBitmap, you must specify the alpha mode parameter when instantiating the SoftwareBitmap to Premultiplied. If you don’t do this, CanvasBitmap throws an exception that says that the format is not supported. There’s also a list of supported pixel formats in the documentation; I suggest BGRA8 for best compatibility.

History

• 8^th February, 2021: Initial version