Silverlight Database Deep Zoom

• Download source - 383.01 KB

Acknowledgement

This code has grown out of two other projects/articles: the DecosDeepZoom by Berend Engelbrecht, and an article from Mike Ormond about the MultiTileSource. I used code from Berend Engelbrecht to generate the tiles.

Introduction

This article shows how to store a Deep Zoom image in a database, and how to retrieve the data for display in a Silverlight app. For simplicity, I use a Microsoft Access database, but the code can easily be adapted for Microsoft SQL Server or Oracle. As simple as this might sound, it takes quite a lot to make it work. The article covers the following ground:

• How to create a Deep Zoom image from a single (large) image. (Or more exactly, how to create the tiles that make up a Deep Zoom image.)
• How to store the created tiles in a database.
• The database model that is required.
• How to read the tiles from the database and pass it to the MultiScaleImage control.

Along the journey, you will see some other useful stuff, for example, how to store a bitmap in an Access database, or how you can use LINQ to convert a DataSet into a list of objects that can be the data source for a Silverlight control.

Background

I have always been fascinated by how you can zoom in endlessly in Google Earth. When Silverlight came with the Deep Zoom feature, I was quite excited. However, I found it awkward that you would have to use a separate tool (Deep Zoom Composer) to create such an image. I also found the files that it generates on the file system impractical. It certainly works if you have to deploy one or two Deep Zoom images onto a website. But what do you do, if you have to create thousands of them?

I work for a company called Scope Solutions, located in Basle, Switzerland. We do software for "historical" archives, which means that we mainly manage meta data about the archival holding, which includes images, maps, and building plans. Of course, the archives scan their interesting items to make them available to the public. And, if an archivist is going to scan an old map, he will do that in the highest possible quality so he has a "backup" that matches the original as close as possible. Just in case the original is damaged/lost. This can lead to very large files and bitmaps. For example, a bitmap with 13722 x 9737 pixels is not uncommon. Such a bitmap uses, depending on the format, up to 400 MB on the disk. And having said this, it should be no surprise to hear that some archives have map collections with ten thousands of maps or over a quarter million images.

Wouldn't it be cool if hobby archivists from all over the world could view "deep zoomable" images of these historical maps? Using the Deep Zoom Composer and a file based system is surely not the way to go. That is why I investigated and wanted to find out how to store the images in the database.

Using the Code

The demo presented here consists of two applications:

• A Windows Forms application that generates the tiles for a Deep Zoom image and stores them in a Microsoft Access database.
• A Silverlight web application that displays the images from the database.

Both applications are contained in the same solution. To compile and run the solutions, you will need VS 2008 SP 1 (required for Silverlight development) with .NET 3.5. (for simplicity, I decided to wrap both applications in one solution). For that reason, you will have to change the startup project, depending on which app you want to run.

Important: Before you can run the solution on your machine, you have to change the connection string in the configuration to match your path location. The connection string for the Windows Forms application can be changed in the settings file. The connection string for the ASP.NET application can be changed in the web.config file. The (empty) sample database is located in the App_Data directory of the ASP.NET application.

The four projects found within the solution are:

• DatabaseDeepZoom: A simple Windows Forms application that provides the UI for creating the tiles of a Deep Zoom image.
• DbDzComposer: This library is used by both projects, and does most of the work. It contains a class for generating the tiles, and a class for storing and retrieving the data from the database.
• DeepZoomSilverlightProject: The Silverlight app that displays the images. This is mainly the default project that you get when you use Deep Zoom Composer. It has been extended by a list for displaying the thumbnails to the right of the Deep Zoom image.
• DeepZoomSilverlightWeb: The ASP.NET app that hosts the Silverlight app. It is an extended version of the default project that Deep Zoom Composer creates. Two HttpHandlers have been added that return the tiles and the thumbnails that the Silverlight app requires, and a WCF Web Service for returning the data about the images.

In this article, I have omitted most of the code comments. The code you can download is better documented.

DatabaseDeepZoom Project

This simple app generated the tiles for a Deep Zoom image. The user provides a filename and a name for the image. The code calls the function GenerateFromFile from the DeepZoomComposer class that is part of the DbDzComposer library. The DeepZoomGenerator class is instantiated with the default constructor which instructs the class to create 256 x 256 pixel tiles and save them as 24bit color JPEG tiles with a quality of 90. You can change that behavior using the second constructor or by setting properties.

The most important fact is that we also instantiate the database access class. This class inherits the interface from an abstract base class that defines the basic interface. Using that interface, it is easy to write an additional provider that would store the data in an Oracle or SQL Server database instead of the Access class we use here.

public MainForm()
{
    InitializeComponent();

    dz = new DeepZoomGenerator {DatabasePersister = GetAccessDb()};
    dz.CreationProgress += dz_CreationProgress;
}

private DzDbAccess GetAccessDb()
{
    string connectString = Settings.Default.ConnectionString;
    var cn = new OleDbConnection(connectString);
    cn.Open();

    var accessPersistence = new DzDbAccess();
    accessPersistence.DbConnection = cn;
    return accessPersistence;
}

The rest of the code just deals with opening the image file, and some other UI related stuff like progress bars.

DbDzComposer Project

This library is used by the Windows Forms application and the ASP.NET application. It contains classes for creating the Deep Zoom image as well as a class for storing the generated tiles in a Microsoft Access database. The DzDbAccess class inherits from an abstract interface that defines the methods for storing and retrieving the images from the database. I have chosen to create this interface so I could easily add another database storage provider for a different database system. As you can see from the code, the interface is quite simple. It provides a method to save the information about an image and a method to save the individual tiles. Then, it has methods to return a list with the information about the images stored in the database, and methods to return a thumbnail preview and the individual tiles back to the caller. And, of course, it needs a property with the information about the database connection.

namespace DbDzComposer
{
    public abstract class IDzDbPersistance
    {
        public abstract int SaveImageInfo(string imageName, int width, int height,
            int tileSize, int overlap, string mimeType, Bitmap thumbnail);
        public abstract List<ImageInfo> GetImageInfo(Uri fromUri);
        public abstract void SaveImageTile(int imageId, int level, int x, int y,
            Bitmap bitmap);
        public abstract Bitmap GetImageTile(int imageId, int level, int x, int y);
        public abstract Bitmap GetThumbnail(int imageId);
        public abstract IDbConnection DbConnection { get; set; }
    }

    public class ImageInfo
    {
        public int ImageId { get; set; }
        public string ImageName { get; set; }
        public int Width { get; set; }
        public int Height { get; set; }
        public int TileSize { get; set; }
        public int Overlap { get; set; }
        public string MimeType { get; set; }
        public string ThumbnailUrl { get; set; }
    }
}

The concrete database persister class is used by the DeepZoomGenerator class for storing the images. This DeepZoomGenerator class is based on the work by Berend Engelbrecht, and is a stripped down version, as my goal was to create tiles for single images only. This class takes a large image, and creates the tiles required for creating the Deep Zoom image. It does, however, not support the more advanced stuff like image collections that the Deep Zoom technology provides and that Berend in his original class provided.

DeepZoomSilverlight Project

This Silverlight project consists of a single page that displays the Deep Zoom image and displays the available images from the database in a listbox. The question is, of course, how does the MultiScaleImage control display the tiles that are stored in the database? Usually, one would assign the Source property of that control the URI to an XML file, e.g., something like msi.Source="../GeneratedImages/dzc_output.xml". But, this property actually is of type MultiScaleTileSource.

So, what we have to do is create a class that inherits from MultiScaleTileSource and implement its interface. The main method of that class is called GetTileLayers, and is called from the MultiScaleImage control for every tile that is needed. Inside that method, you add each tile to a list, but not in the form of a bitmap, but in the form of an URI. The URI itself must then stream back the image, and that is why we use a HttpHandler in the web project.

/// <summary>
/// Gets a collection of the tiles that comprise the Deep Zoom image.
/// </summary>
/// <param name="tileLevel">Level of the tile.</param>
/// <param name="tilePositionX">X-coordinate position of the tile.</param>
/// <param name="tilePositionY">Y-coordinate position of the tile.</param>
/// <param name="tileImageLayerSources">Source of the tile image layer.</param>
protected override void GetTileLayers(int tileLevel, int tilePositionX,
                                      int tilePositionY,
                                      IList<object> tileImageLayerSources)
{
    string source =
        string.Format(
                      "TileHandler.ashx?tileLevel={0}&tilePositionX={1}&" +
                      "tilePositionY={2}&imageId={3}&contentType={4}",
                      tileLevel, tilePositionX, tilePositionY, imageId, mimeType);

    var uri = new Uri(HtmlPage.Document.DocumentUri, source);

    tileImageLayerSources.Add(uri);
}

The HttpHandler is not part of the Silverlight project, but of the ASP.NET application that hosts the Silverlight app. This kind of architecture is required because Silverlight itself is not capable of accessing the database as it runs in a secure sandbox environment.

DeepZoomSilverlight Web Project

This is the ASP.NET application that hosts the Silverlight control. This application also provides the Silverlight control with the data. There are two HttpHandlers in this application, one that returns the bitmap tiles, and one that returns a thumbnail for a specific image. Besides that, there is a WCF service defined that the Silverlight app uses to retrieve the information about the images.

public void ProcessRequest(HttpContext context)
{
    if (context.Request.QueryString.HasKeys())
    {
        // Being lazy...
        int imageId = int.Parse(context.Request.QueryString.Get("imageId"));
        int tileLevel = int.Parse(context.Request.QueryString.Get("tileLevel"));
        int tilePosX = int.Parse(context.Request.QueryString.Get("tilePositionX"));
        int tilePosY = int.Parse(context.Request.QueryString.Get("tilePositionY"));
        string contentType = context.Request.QueryString.Get("contentType");

        context.Response.ContentType = contentType;
        System.Drawing.Bitmap bitmap = GetTile(imageId, tileLevel, tilePosX,
            tilePosY);

        if (bitmap != null)
        {
            System.IO.MemoryStream ms = new System.IO.MemoryStream();
            if (contentType.Contains("png"))
                bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Png);
            else
                bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
            ms.WriteTo(context.Response.OutputStream);
        }
    }
}

private Bitmap GetTile(int imageId, int level, int x, int y)
{
    try
    {
        DzDbAccess accessDb = DeepZoomSilverlightWeb.Helper.GetAccessDb();
        Bitmap bmp = accessDb.GetImageTile(imageId, level, x, y);
        accessDb.DbConnection.Close();
        return bmp;
    }
    catch (Exception)
    {
        return null;
    }
}

Points of Interest

After having given an overview about the different projects involved in this solution and cleared the most important points, I would like to go deeper and give you additional insight on the code.

Deep Zoom File Format

Of course, this article would not be complete without some words about the Deep Zoom file format. But, I am not going to repeat what Microsoft has described in detail in MSDN.

Database Design

One of the questions I had to answer was the database structure that is needed to store the image tiles. It turned out that this is easier than I first thought after looking at all the files the Deep Zoom Composer generates. The database design consists of two tables. One table is needed to store information about the image, most importantly about the width and height, the tile size, and the overlap, as this information is required to correctly initialize the MultiScaleTileSource. The other table stores the generated tiles, and on which level and on which coordinates they are used.

Storing the Bitmaps in the Microsoft Access Database

Even though I do quite a lot of database programming, I'm not storing bitmaps in a BLOB field every day. To achieve the goal, you have to use a parameterized command object. The parameter for the bitmap must be of type OleDbType.LongVarBinary, and you pass this parameter a byte array. To convert the bitmap into a byte array, you will serialize the bitmap into a memory stream using the BinaryFormatter.

public override void SaveImageTile(int imageId, int level, 
                     int x, int y, Bitmap bitmap)
{
    // Create the SQL
    string sql = "INSERT INTO DeepZoomTile(imageId, [level], x, y,
        bitmap) values (?, ?, ?, ?, ?)";
    var cmd = new OleDbCommand(sql, dbConnection);

    //serialise the image
    byte[] bytes = BitmapToByteArray(bitmap);

    // Add the parameters
    cmd.Parameters.Add("@imageid", OleDbType.Integer).Value = imageId;
    cmd.Parameters.Add("@level", OleDbType.Integer).Value = level;
    cmd.Parameters.Add("@x", OleDbType.Integer).Value = x;
    cmd.Parameters.Add("@y", OleDbType.Integer).Value = y;
    cmd.Parameters.Add("@bitmap", OleDbType.LongVarBinary,
        bytes.Length).Value = bytes;

    // Execute it
    cmd.ExecuteNonQuery();
}

/// <summary>
/// Serializes a bitmap to an byte array
/// </summary>
/// <param name="bitmap">The bitmap.</param>
/// <returns></returns>
private static byte[] BitmapToByteArray(Bitmap bitmap)
{
    var memStream = new MemoryStream();
    var formatter = new BinaryFormatter();
    formatter.Serialize(memStream, bitmap);

    byte[] bytes = memStream.GetBuffer();
    memStream.Close();
    return bytes;
}

Returning the Information About Images from the Database to the Silverlight Listbox

The web front-end shows a listbox to the right of the Deep Zoom image. This listbox displays data from the database. As Silverlight cannot deal with DataTables or DataSets, we cannot pass one of those objects as we would normally do. Therefore, the database access class returns a list of ImageInfo objects. Here, LINQ does a great job in simplifying the developer's work, once you have understood the syntax that it requires. You can use this example every time you have to return a list of objects from a DataSet. One thing that is special about this method is the passed parameter fromUri that is required to return the ThumbnailUrl. As the Silverlight app can be deployed anywhere on the net, and thus the path to the HttpHandler that returns the thumbnail image changes, I pass in the URI from the calling page. Like that, I am sure that the address for the ThumbnailUrl is always correct.

/// <summary>
/// Gets a list with the information about the images.
/// </summary>
/// <param name="fromUri">The Url from which the call is made.</param>
/// <returns></returns>
public override List<ImageInfo> GetImageInfo(Uri fromUri)
{
    string sql = "Select * from DeepZoomImage order by imageName";
    var dataAdapter = new OleDbDataAdapter(sql, dbConnection);
    var ds = new DataSet();

    dataAdapter.Fill(ds, "ImageInfo");

    var imageInfos = from image in ds.Tables["ImageInfo"].AsEnumerable()
                     select new ImageInfo
                    {
                        ImageId = image.Field<int>("ImageId"),
                        ImageName = image.Field<string>("ImageName"),
                        Height = image.Field<int>("Height"),
                        Width = image.Field<int>("Width"),
                        TileSize = image.Field<int>("TileSize"),
                        Overlap = image.Field<int>("Overlap"),
                        MimeType = image.Field<string>("MimeType"),
                        ThumbnailUrl = new Uri(fromUri,
                               "ThumbnailHandler.ashx?ImageId=" +
                               image.Field<int>("ImageId")).ToString()
                    };

    return imageInfos.ToList();
}

The returned list of ImageInfo objects is then passed to the Silverlight application using a WCF Service. Here is how the Service, that is defined in the ASP.NET application, looks like:

namespace DeepZoomSilverlightWeb
{
    [ServiceContract(Namespace = "")]
    [AspNetCompatibilityRequirements(RequirementsMode =
        AspNetCompatibilityRequirementsMode.Allowed)]
    public class ImageListService
    {
        /// <summary>
        /// Gets a list of images from the database.
        /// </summary>
        /// <returns>A list of ImageInfo objects</returns>
        [OperationContract]
        public List<ImageInfo> GetImageList()
        {
            DbDzComposer.DzDbAccess accessDb = Helper.GetAccessDb();
            return accessDb.GetImageInfo(HttpContext.Current.Request.Url);
        }
    }
}

In the Silverlight app that consumes the returned list, we call the GetImageList method from within the constructor of the page. As this is an async call, we bind it in the completed event handler of that method.

First, initialize imageListClient...

public Page()
{
    InitializeComponent();

    // Setting the datasource for the list
    var imageListClient = ServiceClientFactory.GetImageListServiceClient();
    imageListClient.GetImageListCompleted +=
      new EventHandler<DeepZoomSilverlightProject.
          ImageListClient.GetImageListCompletedEventArgs>(
    imageListCliet_GetImageListCompleted);
    imageListClient.GetImageListAsync();

Bind the result to the imageList and select the first item in the list...

/// <summary>
/// Handles the GetImageListCompleted event of the imageListCliet control.
/// </summary>
/// <param name="sender">The source of the event.</param>
/// <param name="e">The <see cref="DeepZoomSilverlight
///                 Project.ImageListClient.GetImageListCompletedEventArgs"/>
/// instance containing the event data.</param>
void imageListCliet_GetImageListCompleted(object sender,
    DeepZoomSilverlightProject.ImageListClient.GetImageListCompletedEventArgs e)
{
    // Set the datasource for the image list control
    imageList.ItemsSource = e.Result;

    if (imageList.Items.Count > 0)
        imageList.SelectedIndex = 0;
}

And, in the SelectionChanged event of the listbox, we pass the MultiScaleImage a MultiScaleTileSource object...

private void imageList_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
    ImageInfo imageInfo = imageList.SelectedItem as ImageInfo;

    if (imageInfo != null)
    {
        // Setting the source of the multi scale image
        msi.Source = new DeepZoomTileSource(imageInfo.ImageId,
            imageInfo.Width, imageInfo.Height, imageInfo.TileSize,
            imageInfo.Overlap, imageInfo.MimeType);
    }
}

That's all. I hope you find the code useful and easy to follow. If you have any comments or suggestions, just let me know.

History

• 04 February 2009 - Version 1.0 created
• 25 March 2009 - Article updated