Contents
Introduction
One of the most "magical" features of Silverlight is Deep Zoom [^]. Through the use of clever partitioning of images, Deep Zoom allows users to pan and zoom smoothly through immense (and potentially infinite) images with excellent performance.
Since the release of Deep Zoom with Silverlight 2, it has been one of the most requested missing features from WPF - at the time this article was written, it was ranked within the top 10 requests on the UserVoice site for WPF [^]:
Well, your requests have been answered! This article presents a fully-working implementation of Deep Zoom for WPF, complete with Deep Zoom Composer and Zoom.it support, multi-touch support, and more. In the first part, we'll see how to use the control and its current limitations; after that, we'll see how it's done in detail. Let's start!
Using the control
The Deep Zoom for WPF project is an Open Source project available on CodePlex [^] under the MS-PL [^]. You can download the latest version at http://deepzoom.codeplex.com/releases [^].
To use the control, all you have to do is add a reference to DeepZoom.dll and add it to your XAML file as if it were a normal image:
[MainWindow.xaml]: (modified)
<Window x:Class="DeepZoom.TestApplication.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:dz="clr-namespace:DeepZoom.Controls;assembly=DeepZoom"
Title="MainWindow" Height="800" Width="1280">
<Border Background="Black">
<dz:MultiScaleImage
Source="http://static.seadragon.com/content/misc/color-flower.dzi" />
</Border>
</Window>
It doesn't get any simpler than this :)
The control also supports images generated with Deep Zoom Composer [^] (except for collections) and Zoom.it [^], as well as custom tile sources.
Limitations and known issues
The current implementation of the control is a simplified version of the Silverlight MultiScaleImage, supporting only its most essential features. Its limitations and known issues are:
- The control only supports Deep Zoom images (including sparse images), with no support for Deep Zoom Collections yet.
- Unsupported members from Silverlight:
- Properties:
AllowDownloadingBlurFactorIsDownloadingIsIdleSkipLevelsSubImagesUseSpringsViewportOriginViewportWidth
- Events
ImageFailedImageOpenFailedImageOpenSucceededMotionFinishedViewportChanged
- The control only supports panning and zooming with mouse dragging/wheel and multi-touch, with no additional buttons for navigation (though they can be easily added).
- The maximum number of tiles (total) is
int.MaxValue (~2.1 billion). This is due to the use of IList/ItemsContainerGenerator for virtualization (more on that below). Therefore, the maximum zoom level will be ~24 (depending on your image's aspect ratio).
- The pan and zoom performance gets worse after level ~21. If you have any idea on how to fix this, please let me know!
Acknowledgements
This project uses code from the following libraries:
Special thanks to IntuiLab, my employer when this project was born, for allowing me to publish this project and article as Open Source!
Show me how it's done
How does Deep Zoom work?
The first step for building Deep Zoom is to understand how it works. From Jaime Rodriguez's Deep Zoom Primer [^]:
Deep Zoom accomplishes its goal by partitioning an image (or a composition of images) into tiles. While tiling the image, the composer also creates a pyramid of lower resolution tiles for the original composition.
Image from MSDN Library
The image above shows you what a pyramid looks like; the original image is lowest in the pyramid, notice how it is tiled into smaller images; also notice, the pyramid creates lower resolution images (also tiled). (...) All of this tiling is performed at design-time and gets accomplished using the Deep Zoom Composer.
At run-time, a MultiScaleImage downloads a lower resolution tile of the image first and downloads the other images on demand (as you pan or zoom); Deep Zoom ensures the transitions from lower to higher resolution images are smooth and seamless.
In Deep Zoom, all the tiles are square and have the same size (by default, 256x256). Although, if you look at the output folder of a Deep Zoom Composer project on your computer, you'll see that tiles have different sizes because they also overlap each other (by default, the overlap is 1px in each side).
In this article, I won't delve into the mathematical details of Deep Zoom - please refer to "Inside Deep Zoom - Part II: Mathematical Analysis" [^] on Daniel Gasienica's blog for a very interesting explanation of multi-scale images from a mathematical perspective.
Deep Zoom object model
To follow the Silverlight object model for Deep Zoom as closely as possible, the project provides a class library that mimics the Silverlight one. The public object model is represented below:
The only difference (besides members that are not implemented) is that the GetTileLayers method was simplified. In the original Silverlight version, the method had the following signature:
protected abstract void GetTileLayers(int tileLevel,
int tilePositionX,
int tilePositionY,
IList<object> tileImageLayerSources);
The class that implemented this method had to add the Uri for the new tile to the tileImageLayerSources list. Since this list of Uris is not used in this project, the method has the following signature:
protected abstract object GetTileLayers(int tileLevel,
int tilePositionX,
int tilePositionY);
The method now supports both Uris and Streams as valid return values, allowing for custom tile sources that dynamically generate tiles in memory.
Enter ZoomableCanvas
Since multi-scale images can have billions of tiles, the main challenge of Deep Zoom lies in virtualization. We need to perform virtualization in two ways:
- Data must be virtualized so that only tiles that are being used are stored in memory.
- UI must be virtualized so that we only load and render images when they're needed.
To accomplish both types of virtualization, I've used Kael Rowan's ZoomableCanvas [^]. This clever control allows us to virtualize items while panning and zooming. The basic idea is that at any moment, the ZoomableCanvas must be able to know which elements are visible in the current viewbox, so that it can appropriately load and "realize" the elements.
The magic happens when you create a class that implements an interface called ISpatialItemsSource:
public interface ISpatialItemsSource
{
Rect Extent { get; }
event EventHandler ExtentChanged;
event EventHandler QueryInvalidated;
IEnumerable<int> Query(Rect rectangle);
}
The Query method above should return a list of indexes of the elements that are visible inside the rectangle passed in as parameter, while the Extent property represents the bounds of the canvas. Additionally, your class should use the ExtentChanged and QueryInvalidated events to notify the canvas that it should re-query or change its extent (usually when the item collection changes).
When this interface is implemented, the ZoomableCanvas manages the elements to be added and removed from the screen by querying the visible items whenever the view changes (through panning or zooming). Then, if the ZoomableCanvas is used as an ItemsPanel on an ItemsControl (which means the ItemsSource implements both ISpatialItemsSource and IList), the canvas will query the List by index to get the items and show them on the screen.
Because of that dependency in IList, the ZoomableCanvas is limited to int.MaxValue (~2.1 billion) tiles, which might seem to be a huge number, but it can be surpassed by some large Deep Zoom sparse images.
The implementation used in this project is in the MultiScaleImageSpatialItemsSource class, based on Kael Rowan's demo called "ZoomableApplication2 - A million items" [^]. In this demo, Kael uses a simple algorithm to determine visible tiles in a uniform grid and order from the center to the borders of the visible area. The implementation in this project is the following:
from [MultiScaleTileSource.cs]:
private IEnumerable<Tile> VisibleTiles(Rect rectangle, int level)
{
rectangle.Intersect(new Rect(ImageSize));
var top = Math.Floor(rectangle.Top / TileSize);
var left = Math.Floor(rectangle.Left / TileSize);
var right = Math.Ceiling(rectangle.Right / TileSize);
var bottom = Math.Ceiling(rectangle.Bottom / TileSize);
right = right.AtMost(ColumnsAtLevel(level));
bottom = bottom.AtMost(RowsAtLevel(level));
var width = (right - left).AtLeast(0);
var height = (bottom - top).AtLeast(0);
if (top == 0.0 && left == 0.0 && width == 1.0 && height == 1.0)
yield return new Tile(level, 0, 0);
else
{
foreach (var pt in Quadivide(new Rect(left, top, width, height)))
yield return new Tile(level, (int)pt.X, (int)pt.Y);
}
}
private static IEnumerable<Point> Quadivide(Rect area)
{
if (area.Width > 0 && area.Height > 0)
{
var center = area.GetCenter();
var x = Math.Floor(center.X);
var y = Math.Floor(center.Y);
yield return new Point(x, y);
var quad1 = new Rect(area.TopLeft, new Point(x, y + 1));
var quad2 = new Rect(area.TopRight, new Point(x, y));
var quad3 = new Rect(area.BottomLeft, new Point(x + 1, y + 1));
var quad4 = new Rect(area.BottomRight, new Point(x + 1, y));
var quads = new Queue<IEnumerator<Point>>();
quads.Enqueue(Quadivide(quad1).GetEnumerator());
quads.Enqueue(Quadivide(quad2).GetEnumerator());
quads.Enqueue(Quadivide(quad3).GetEnumerator());
quads.Enqueue(Quadivide(quad4).GetEnumerator());
while (quads.Count > 0)
{
var quad = quads.Dequeue();
if (quad.MoveNext())
{
yield return quad.Current;
quads.Enqueue(quad);
}
}
}
}
Some points to note:
- The
Tile struct identifies a tile by its Level, Column, and Row (all of type int).
- The
Quadivide algorithm simply divides a rectangle in 1x1 "cells", ordering them from the center to the edges.
- This code uses some of the MathExtensions [^] extension methods by Kael Rowan, such as
AtMost and AtLeast, to make the code more readable.
An important point to note is that this algorithm has to be run at each level, from 0 to the current level, in order to achieve the Deep Zoom "progressive loading" effect. The code that does that is the following:
from [MultiScaleTileSource.cs]: (modified)
internal IEnumerable<Tile> VisibleTilesUntilFill(Rect rectangle, int startingLevel)
{
var levels = Enumerable.Range(0, startingLevel + 1);
return levels.SelectMany(level =>
{
var levelScale = ScaleAtLevel(level);
var scaledBounds = new Rect(rectangle.X * levelScale,
rectangle.Y * levelScale,
rectangle.Width * levelScale,
rectangle.Height * levelScale);
return VisibleTiles(scaledBounds, level);
});
}
This LINQ query takes a visible rectangle in the coordinate system of the full image (passed in by the ISpatialItemsSource Query method), scales it to each level, and calculates the cells inside it through the algorithm described above. Note that this algorithm becomes slower as startingLevel increases (the user is zooming deeper in the image). According to Daniel Gasienica's article [^], this algorithm downloads up to 33% more data than what would be needed to display only the largest layer.
Displaying the tiles
After we know which tiles must be shown, we need to download them if necessary. To do that, the MultiScaleImageSpatialItemsSource's implementation for this[int index] (from IList) must be able to get a Tile from its index, get the image source from the MultiScaleTileSource, and download the image. In this implementation, the download is done asynchronously using Parallel Extensions for .NET. The class also manages a local cache for the tiles.
After downloading, we must display the tiles on the screen with a reasonable performance. Since we'll be showing up to 200 tiles on the screen and the tiles don't need to be interactive, it's not a good idea to use Image objects or other complex interactive FrameworkElements.
The solution implemented in this project is a very simple class that inherits from FrameworkElement that renders the image in a Visual object:
[TileHost.cs]: (modified)
public class TileHost : FrameworkElement
{
private DrawingVisual _visual;
private static readonly AnimationTimeline _opacityAnimation =
new DoubleAnimation(1, TimeSpan.FromMilliseconds(500))
{ EasingFunction = new ExponentialEase() };
public TileHost()
{
IsHitTestVisible = false;
}
public static readonly DependencyProperty SourceProperty =
DependencyProperty.Register("Source",
typeof(ImageSource),
typeof(TileHost),
new FrameworkPropertyMetadata(null,
new PropertyChangedCallback(RefreshTile)));
public static readonly DependencyProperty ScaleProperty =
DependencyProperty.Register("Scale",
typeof(double),
typeof(TileHost),
new FrameworkPropertyMetadata(1.0,
new PropertyChangedCallback(RefreshTile)));
private static void RefreshTile(DependencyObject d,
DependencyPropertyChangedEventArgs e)
{
var tileHost = d as TileHost;
if (tileHost != null && tileHost.Source != null && tileHost.Scale > 0)
tileHost.RenderTile();
}
private void RenderTile()
{
_visual = new DrawingVisual();
Width = Source.Width * Scale;
Height = Source.Height * Scale;
var dc = _visual.RenderOpen();
dc.DrawImage(Source, new Rect(0, 0, Width, Height));
dc.Close();
CacheMode = new BitmapCache(1 / Scale);
Opacity = 0;
BeginAnimation(OpacityProperty, _opacityAnimation);
}
protected override int VisualChildrenCount
{
get { return _visual == null ? 0 : 1; }
}
protected override Visual GetVisualChild(int index)
{
return _visual;
}
}
Some interesting points:
- The
TileHost can calculate its size from its Scale property.
- Using
DrawingVisual and DrawingContext is an efficient way of printing an image on a FrameworkElement, if interactivity is not necessary.
- The "blending" animation is executed when the tile finishes rendering.
Putting it all together
So now we know how to download and print individual tiles on the screen. To bind everything together:
- An
ItemsControl with a MultiScaleImageSpatialItemsSource as ItemsSource will display the tiles.
- This
ItemsControl's ItemsPanel is a ZoomableCanvas that knows how to virtualize UI and data, thanks to the ISpatialItemsSource implementation.
- The
TileHost defined in the previous section will be used as an ItemTemplate, displaying the images on the screen.
The class that does all that is the MultiScaleImage, which is the entry point for the developer. The default template for this class looks like this:
from [Themes/Generic.xaml]:
<ControlTemplate TargetType="{x:Type local:MultiScaleImage}">
<ItemsControl x:Name="PART_ItemsControl"
Background="Transparent" ClipToBounds="True">
<ItemsControl.ItemContainerStyle>
<Style TargetType="ContentPresenter">
<Setter Property="Canvas.Top" Value="{Binding Top}"/>
<Setter Property="Canvas.Left" Value="{Binding Left}"/>
<Setter Property="Panel.ZIndex" Value="{Binding ZIndex}"/>
</Style>
</ItemsControl.ItemContainerStyle>
<ItemsControl.ItemTemplate>
<DataTemplate>
<local:TileHost Source="{Binding Source}" Scale="{Binding Scale}"/>
</DataTemplate>
</ItemsControl.ItemTemplate>
</ItemsControl>
</ControlTemplate>
Where is the ZoomableCanvas, you might ask. Since we need to access it from the class, if we had put it as ItemsControl.ItemsPanel, we wouldn't have access to its namescope. To work around that issue, the MultiScaleImage class injects the ZoomableCanvas and keeps a reference to it:
from [MultiScaleImage.cs]: (modified)
public override void OnApplyTemplate()
{
base.OnApplyTemplate();
_itemsControl = GetTemplateChild("PART_ItemsControl") as ItemsControl;
if (_itemsControl == null) return;
_itemsControl.ApplyTemplate();
var factoryPanel = new FrameworkElementFactory(typeof(ZoomableCanvas));
factoryPanel.AddHandler(LoadedEvent, new RoutedEventHandler(ZoomableCanvasLoaded));
_itemsControl.ItemsPanel = new ItemsPanelTemplate(factoryPanel);
}
private void ZoomableCanvasLoaded(object sender, RoutedEventArgs e)
{
_zoomableCanvas = sender as ZoomableCanvas;
}
Panning and zooming
The MultiScaleImage in Silverlight doesn't come with any interactivity by default. The developer must implement panning and zooming support to enable users to manipulate the image. In this version, I've included mouse and multi-touch panning and zooming in the control to make it easier to use directly.
To do that, I've overridden the OnManipulationDelta method and added multi-touch pan and zoom support. OnManipulationInertiaStarting was also overridden to make the animations more fluid.
The problem is that by using only manipulation events, the control wouldn't support the mouse. To overcome this issue, I've used Josh Blake's MouseTouchDevice from his excellent Blake.NUI [^] library. This class enables the mouse to act as a single-touch device, making it much easier to create mouse and multi-touch interactions with a single entry point.
The final touch for mouse support is the mouse wheel. By overriding OnPreviewMouseWheel, we're able to capture the mouse wheel and animate the zoom level.
One interesting point in the manipulation code is the use of throttling for zoom events. Since the user might zoom in and out very quickly, the level will change quickly, forcing the ZoomableCanvas to recalculate the tiles multiple times for no reason, while the user is moving from a level to another. To solve that, we use a DispatcherTimer that throttles the level changes to a predefined interval:
Excerpts from [MultiScaleImage.cs]:
private int _desiredLevel;
private readonly DispatcherTimer _levelChangeThrottle;
public MultiScaleImage()
{
_levelChangeThrottle = new DispatcherTimer
{
Interval = TimeSpan.FromMilliseconds(ThrottleIntervalMilliseconds),
IsEnabled = false
};
_levelChangeThrottle.Tick += (s, e) =>
{
_spatialSource.CurrentLevel = _desiredLevel;
_levelChangeThrottle.IsEnabled = false;
};
}
private void ScaleCanvas()
{
if (newLevel != level)
{
if (newLevel > level)
ThrottleChangeLevel(newLevel);
else
_spatialSource.CurrentLevel = newLevel;
}
}
private void ThrottleChangeLevel(int newLevel)
{
_desiredLevel = newLevel;
if (_levelChangeThrottle.IsEnabled)
_levelChangeThrottle.Stop();
_levelChangeThrottle.Start();
}
Adding a TypeConverter
The last step to make it easier for developers to use this control is to enable them to use it directly in XAML. Without a TypeConverter, the code would look something like this:
<dz:MultiScaleImage>
<dz:MultiScaleImage.Source>
<dz:DeepZoomImageTileSource
UriSource="http://static.seadragon.com/content/misc/color-flower.dzi" />
</dz:MultiScaleImage.Source>
</dz:MultiScaleImage>
By using TypeConverters, we're able to convert strings to other types directly, allowing for code like this:
<dz:MultiScaleImage Source="http://static.seadragon.com/content/misc/color-flower.dzi" />
What's the secret? To do that, we must first create a class that derives from TypeConverter that looks like this:
[DeepZoomImageTileSourceConverter.cs]:
public class DeepZoomImageTileSourceConverter : TypeConverter
{
public override bool CanConvertFrom(ITypeDescriptorContext context,
Type sourceType)
{
if (sourceType == typeof(string))
return true;
return base.CanConvertFrom(context, sourceType);
}
public override bool CanConvertTo(ITypeDescriptorContext context,
Type destinationType)
{
if (destinationType == typeof(string))
return true;
return base.CanConvertTo(context, destinationType);
}
public override object ConvertFrom(ITypeDescriptorContext context,
CultureInfo culture, object value)
{
var inputString = value as string;
if (inputString != null)
{
try
{
return new DeepZoomImageTileSource(
new Uri(inputString, UriKind.RelativeOrAbsolute)
);
}
catch (Exception ex)
{
throw new Exception(string.Format(
"Cannot convert '{0}' ({1}) - {2}",
value,
value.GetType(),
ex.Message
), ex);
}
}
return base.ConvertFrom(context, culture, value);
}
public override object ConvertTo(ITypeDescriptorContext context,
CultureInfo culture, object value,
Type destinationType)
{
if (destinationType == null)
throw new ArgumentNullException("destinationType");
var tileSource = value as DeepZoomImageTileSource;
if (tileSource != null)
if (CanConvertTo(context, destinationType))
{
var uri = tileSource.UriSource;
return uri.IsAbsoluteUri ? uri.AbsoluteUri : uri.OriginalString;
}
return base.ConvertTo(context, culture, value, destinationType);
}
}
Finally, we must declare that this converter can be used to convert a string to a class of type MultiScaleTileSource:
[MultiScaleTileSource.cs]:
[TypeConverter(typeof(DeepZoomImageTileSourceConverter))]
public abstract class MultiScaleTileSource : DependencyObject
{
}
And that's all!
Bonus: Creating a custom tile source
To finish, we'll see how easy it is to implement a custom tile source for Deep Zoom. In this sample, we'll create a tile source for OpenStreetMap.
Note: Accessing the OSM tiles directly is not recommended in production. Please don't abuse this service!
In order to implement a custom MultiScaleTileSource, all we have to know is how to generate a Uri from a tile level, X and Y coordinates. According to the OpenStreetMap wiki, the URL format for the Osmarender map is http://tah.openstreetmap.org/Tiles/tile/{zoom}/{x}/{y}.png.
Also note that the zoom level for OSM is not the same as the zoom level for Deep Zoom; a zoom level of 0 for OSM corresponds to one visible 256x256 tile, which corresponds to level log2256 = 8 in Deep Zoom.
Therefore, all the code we need is this:
[OpenStreetMapTileSource.cs]:
public class OpenStreetMapTileSource : MultiScaleTileSource
{
public OpenStreetMapTileSource() : base(0x8000000, 0x8000000, 256, 0) { }
protected override object GetTileLayers(int tileLevel,
int tilePositionX, int tilePositionY)
{
var zoom = tileLevel - 8;
if (zoom >= 0)
return new Uri(string.Format(
"http://tah.openstreetmap.org/Tiles/tile/{0}/{1}/{2}.png",
zoom,
tilePositionX,
tilePositionY));
else
return null;
}
}
The code to use it in a XAML file looks like this:
<dz:MultiScaleImage>
<dz:MultiScaleImage.Source>
<osm:OpenStreetMapTileSource />
</dz:MultiScaleImage.Source>
</dz:MultiScaleImage>
Wrapping up
In this article, we developed the elusive Deep Zoom control for WPF using many ideas from different sources. The control is still in its embrionary stage, and that's why the code is also available on CodePlex [^]. Please help us tweak and improve this control to enable the best Deep Zoom experience!
I hope this article has showed some ideas about high performance interfaces in WPF. If you develop something using this control, post the link in the comments section!
For the future
Some interesting next steps for this app could be:
- Adding support for Deep Zoom Collections.
- Improving performance, especially in deeper zoom levels (> 22).
- Using a unified (possibly native) rendering surface instead of multiple
TileHosts.
- Removing the
int.MaxValue limitation for the number of tiles. This would probably require re-implementing the ZoomableCanvas to use a different system for virtualization.
- Implementing a more optimized system for downloading/caching tiles and cancelling downloads.
- Separating the interactivity from
MultiScaleImage, to keep consistency with Silverlight. This new control would wrap the MultiScaleImage, adding interactivity, navigation, etc.
What do you think?
Was this project useful? How would you implement Deep Zoom in WPF? Is there any point that needs additional explanation?
Please leave your comments and suggestions, and please vote up if this article pleases you. Thanks!
Other links and references
History
- v1.0 (22/11/2010) - Initial release.
