Metro Style Lightweight Image Processing

• Download source code - 1.1 MB

Introduction

Sensor and image are two amazing things which interest programmers most in slate and mobile computer. Using the two features, we can make a lot of amazing applications.

There are two main ways to do image processing, one is using direct X HLSL ,the other is using bitmap pixels. To write a casual game or APP with HDLS might be ‘the big topic does on a small scale’ for application developer. So that is the reason why I write this article.

Recently, I tried to port my image processing program written with GDK+ in the past to a Metro style application. The main part of my application is to get the pixels from bitmap file, recalculate the value of each pixel with math algorithm, then restore it in the originate picture. With that, the program can do many interesting effect as if you can figure out an interesting algorithm. In my application, I have done some algorithm such as negative, tint, oil-paint, emboss ,gray-level, sunlight ,etc.

Metro style application is more like WPF. The old program I wrote is using GDK +, so I tried to modify it to a WPF application. It is interesting that you can find it need some effort to port it to different windows platforms. It is because some of the APIs and namespace are different, so it take time to modify the structure of your program. To get a whole picture, I will introduce how to process bitmap image process in GDK+,WPF, winphone7, metro style application.

Bitmap access in different platform

The way GDK+ handle bitmap is straightforward. It uses the Bitmap class to access the bitmap. The most useful method are SetPixel and GetPixel. To do the work, we need firstly declare a Bitmap class as following:

Bitmap original;

After that we declare a pixel ‘Target’ with System.Drawing.Color structure to store the value of R, G, B.

System.Drawing.Color Target;

After that we can use GetPixel to get the value of pixel and store it into target.

Target= original.GetPixel(x, y);

int r = Target.R;
int g = Target.G;
int b = Target.B;

After that, we can process the content of each R, G, B. For example, in the following, we just get the average value of RGB.

int avg = (r + g + b) / 3;

Then we use can use SetPixel to set the new value to the bitmap.

original.SetPixel(x, y, System.Drawing.Color.FromArgb(avg, avg, avg));

After GDI+, let’s talk about WPF. In WPF, the way to handle bitmap is a little different. It uses BitmapImage class to access the bitmap and use CopyPixels to get the pixel of the bitmap picture to a byte array. To use CopyPixels, we need to use a Rect structure to set the range of the bitmap.

BitmapImage iSrc;
var array = new int[iSrc.PixelWidth * iSrc.PixelHeight];
var rect = new Int32Rect(0, 0, iSrc.PixelWidth,iSrc.PixelHeight);
iSrc.CopyPixels(rect, array, iSrc.PixelWidth * 4, 0);

To get the R, G, B value of the pixel, because it is an integer array, we need to do bit shift to get the value of R, G, B. The Blue value of the pixel is:

byte Blue= (byte)((array [index]&  0x000000FF);

The Green of the pixel is:

byte Green= (byte)((array [index]& 0x0000FF00)>>8);

The Red of the pixel is:

 byte  Red =(byte)(( array [index]& 0x00FF0000) >> 16);

The Alpha of the pixel is:

byte Alpha=(byte)(( array [index]& 0xFF000000) >> 24);

After that we can process the R, G, B value of the pixel as well. Then combine value to the integer array.

array [index]=(Blue)|(Green<<8)|(Red<<16)|(Alpha<<24);

Then we use the modified pixel array to create a new bitmap to show the result:

BitmapImage.Create(modifiedImage.PixelWidth,
modifiedImage.PixelHeight, 96, 96, PixelFormats.Bgra32, null, array,
pixelsNewsize);

After WPF , let’s talk about WP7. In WP7, we can’t direct modify the pixels of bitmap, we need to use ‘WriteableBitmap’ to access bitmap. The code to create a ‘WriteableBitmap’ is as followings:

BitmapImage bitmap= new BitmapImage();
bitmap.SetSource(value);WriteableBitmap;
modifiedImage = new WriteableBitmap(bitmap);

WriteableBitmap include a byte array ‘Pixel’s to store the whole R,G,B value of Pixels.

byet Blue = (byte)(rawpixel.Pixels[offset]& 0x000000FF); 
byet Green =(byte)((rawpixel.Pixels[offset] & 0x0000FF00) >> 8);
byet Red    =  (byte)((rawpixel.Pixels[offset]& 0x00FF0000) >> 16);
byet Alpha= (byte)((rawpixel.Pixels[offset] & 0xFF000000) >> 24);

After calculating the new value, we use the following way to put the new value back.

rawimagepixel.Pixels[offset] =
  (pixels.Blue) | (pixels.Green << 8) | (pixels.Red << 16) | (pixels.Alpha<<24);

Metro style APP use WinRT. One of the major changes between Silverlight and WinRT are namespace names. Rather than System.Windows.----- , we now use Windows.UI.Xaml.----. There is a document mention of interest: Migrating a Windows Phone 7 app to XAML.

In Metro APP, we use the following way to access bitmap. You can find thought there is also a WriteableBitmap as well, but there is not pixels property. Beause of that, we need to figure out a way to access pixels. I can use ‘Stream’ to do the job. The code is as following:

WriteableBitmap bitmap;

byte[] pixels;
Stream pixelStream;
bitmap = new WriteableBitmap(width, height);
pixels = new byte[4 * bitmap.PixelWidth *
bitmap.PixelHeight];
pixelStream = bitmap.PixelBuffer.AsStream();

The following method sets a pixel to a particular color:

int index = 4*(y * bitmap.PixelWidth + x);

pixels[index + 0] = Pcolor.B;
pixels[index + 1] = Pcolor.G;
pixels[index + 2] = Pcolor.R;
pixels[index + 3] = Pcolor.A;

When you need to update the WriteableBitmap from the pixel array, you need to use seek to set the position of the pixel and use write method to write it:

pixelStream.Seek(0,SeekOrigin.Begin);pixelStream.Write(pixels,0, pixels.Length);

Below is the outlook of my sample Metro application for image processing:

Image processing algorithm

All right, now we know how to treat with the pixels in Metro style APP, the next step I would like to introduce is to talk about how to use cool algorithm to modify the pixels. First , let us try to process a negative photo effect . The algorithm is use 255 to minus the R,G,B value of each pixel. The result should between 0 ~255 , the algorithm is as following:

b= (byte)(255 - PC.Blue);
g= (byte)(255 - PC.Green);
r= (byte)(255 - PC.Red);

StreamBuffer[offset+2] = (byte)(r > 255 ? 255 : (r < 0 ? 0 : r));
StreamBuffer[offset+1] =(byte)(g > 255 ? 255 : (g< 0 ? 0 : g));
StreamBuffer[offset+0]=(byte)(b> 255 ? 255 : (b< 0 ? 0 : b));

After processing negative algorithm, the result looks like the following one.

The second algorithm I would like to talk about is color filter. In following code, R,G,B is the rational factors to control the R,G,B value of the pixel. We can use the R,G,B as a gain to control the new R,G,B value. The same, the value must located between 255 and 0 .The algorithm is as following:

b= (byte)(PC.Blue * R);
g= (byte)(PC.Green * G);
r= (byte)(PC.Red * B); 

StreamBuffer[offset+2]= (byte)(r > 255 ? 255 : (r < 0 ? 0 : r));
StreamBuffer[offset+1] =(byte)(g > 255 ? 255 : (g< 0 ? 0 : g));
StreamBuffer[offset+0]=(byte)(b> 255 ? 255 : (b< 0 ? 0 : b));

Here we put G as 1, R as 0, G as 0. The output is as followings:

The third image algorithm is called emboss, we will do the effect with getting two neighbor pixels and get the R,G,B distance of the two pixels then add an offset to be the new value of R,G,B . The algorithm is as following:

PC1 = GetPixel(i, j, streambuffer, w, h);
PC2 = GetPixel(i+1, j+1, streambuffer,w, h);

r= Math.Abs(PC1.Red - PC2.Red + 128);
g= Math.Abs(PC1.Green - PC2.Green + 128);
b= Math.Abs(PC1.Blue - PC2.Blue + 128);

StreamBuffer[offset+2]= (byte)(r > 255 ? 255 : (r < 0 ? 0 : r));
StreamBuffer[offset+1] =(byte)(g > 255 ? 255 : (g< 0 ? 0 : g));
StreamBuffer[offset+0]=(byte)(b> 255 ? 255 : (b< 0 ? 0 : b));

The output of the photo is as following:

The fourth one is sunlight, we define a radius R, if the distance of current point is less then R then use 200 * (1 - MyLength / R) as the new value. The algorithm is as following:

float MyLength = (float)Math.Sqrt(Math.Pow((i - MyCenter.X), 2) + 

Math.Pow((j - MyCenter.Y),2));    

if (MyLength < R)
{
      PC = GetPixel(i, j, streambuffer, w, h);
      float MyPixel = 200 * (1 - MyLength / R);
      int  r = PC.Red + (int)MyPixel;
      PC.Red = (byte)Math.Max(0, Math.Min(r, 255));
      Int g = PC.Green + (int)MyPixel;
      PC.Green = (byte)Math.Max(0, Math.Min(g,255));
      Int b = PC.Blue + (int)MyPixel;
      PC.Blue = (byte)Math.Max(0, Math.Min(b, 255));
      PutPixel(streambuffer, w, h, PC, i, j);
}

The output of the photo is as following:

The fifth algorithm I will like to talk is set photo to gray level style, we set the same value to new r,g,b. The algorithm is as following:

r = (byte)((0.311 * r1) + (0.486 * g1)+(0.213 * b1));
StreamBuffer[offset+2]= (byte)(r > 255 ? 255 : (r < 0 ? 0 : r));
StreamBuffer[offset+1] =r;
StreamBuffer[offset+0]=r;

The output of the photo is as following:

The sixth is Brightness, it is quite easy. We add a constant value to the new pixel. The algorithm is as following:

r = r_original + bright;
g = g_original+ bright;
b = b_original+ bright;

The output of the photo is as following:

The seventh is oil paint, the main concept is to get random points of the bitmap and replace its new position to the random data get in software. The algorithm is as follows:

Random rnd = new Random();
int iModel= 10;
int i =w - iModel;

while (i> 1)
{
    int j= h - iModel;
  
    while(j > 1)
    {
       int iPos = rnd.Next(100000) % iModel;
         
       PC = GetPixel((int)(i + iPos), (int)(j + iPos), modifiedstreambuffer, w,
         h);
                 
       PutPixel(bitmapstreambuffer, w, h, PC, i, j);
       j= j - 1;
     }
     i = i - 1;
}

The output is as follows:

All the effect is quite interesting, right? Of course, you can create more amazing image effect algorithm created by yourself. By the way, we can also add camera function in the image processing program, so after take a picture, we can put special effect for that immediately. Part of the code is as following.

var ui = new CameraCaptureUI();
ui.PhotoSettings.CroppedAspectRatio= new Size(4, 3);
var file = await
ui.CaptureFileAsync(CameraCaptureUIMode.Photo);
             
stream = await file.OpenAsync(FileAccessMode.Read);
var bitmap = new BitmapImage();
             
bitmap.SetSource(stream);
             
Image1.Source = bitmap;

At the end of the article, let us talk about some performance issues of bitmap processing. We use negative algorithm as an example to discuss it. To improve the processing speed, we can:

• Change two ‘for’ loop into one for loop, that’s means process one strip instead of one pixel.
• Read all pixels to a memory buffer but read one pixel directly.

In the sample program, in the beginning, I process pixels with two ‘for’ loops , it take s long time to process the value of pixels. The performance is not good.

for (i = 0;i < modifiedImage.PixelWidth; i++)
{
    for(int j = 0; j < modifiedImage.PixelHeight; j++)
    {
        PC = GetPixel(i, j, streambuffer,w,h);
        PC.Blue = (byte)(255 - PC.Blue);
                 
        PC.Green = (byte)(255 - PC.Green);
        PC.Red = (byte)(255 - PC.Red);
                 
        PutPixel(streambuffer,w,h, PC, i, j);
    }
}

After I shrink two for loop to one as the following:

for (i = 0;i < streambuffer.Length-4; i=i+4)
{
    streambuffer[i + 3] = (byte)( 0xff - streambuffer[i + 3]);
    streambuffer[i + 2] = (byte)( 0xff - streambuffer[i + 2]);
    streambuffer[i + 1] = (byte)( 0xff - streambuffer[i + 1]);
    streambuffer[i + 0] = (byte)( 0xff - streambuffer[i + 0]);
}

It did improve performance a lot.

Hope the article and sample program can help you know more about image effect in Metro style application.