Pixel graphics programming on Windows Mobile

• Download source files - 28.7 Kb
• Download demo project - 11 Kb

Introduction

This article explains the basics of pixel graphics programming on Pocket PC/SmartPhone devices. I've collected the information you may need when starting graphics programming on Pocket PC. I've written an example Caleidoscope application, that demonstrates how to make your window full-screen, and how to access the video memory of your handheld device. I'm not going to discuss how to program graphics using the Windows GDI functions. I think it is a lot faster, yet relatively simple, to put your pixels right into the video memory. I'm aware that there are some free and commercial graphics libraries for Windows Mobile. However, they might use device-dependent tricks, and they might or might not be free for personal use. I think once you get an understanding of how Windows Mobile graphics works, you won't want to pay for any libraries. I admit some libraries might be faster than the approach explained here, but I really had no speed problems with any of my PDAs. And, the code still could be optimized for speed in many ways.

I've put all graphics handling routines into the ZGfx class, which can easily be used in other pixel-graphics applications, both on Pocket PC and SmartPhone. This class is not intended to be a complete graphics library; it is merely a set of handy routines to speed up graphics development.

I assume you are familiar with Visual Studio and the C language, I also assume you know what a pointer and a window handle is.

Pixels, pitches, and frame buffers

Pocket PC/SmartPhone does not support selectable video modes; instead, there are fixed, device-dependent video modes. As of today (October 2006), most  Pocket PC devices that you can get your hands on have one of these resolutions:

• QVGA: 240x320, 16 bits per pixel
• VGA: 480x640, 16 bpp

And SmartPhones can be:

• Standard: 176x220, 16 bpp
• QVGA: 240x320, 16 bpp

There are also square screen devs (QGVA: 240x240, VGA: 480x480). A couple of things about pixel formats. I'll discuss the 16 bpp format called RGB-565. There are (were?) other, older pixel formats and I'm sure there will be soon 24 bpp devices with RGB-888 colors. So be careful with assumptions and always check the hardware your program runs on, and be prepared that new devices might support different pixel formats.

So, these days the majority of devices support 16 bits per pixel: 5 bits red, 6 bits green and 5 bits blue. 16 bits make 2 bytes, and here is how are they stored in video memory:

A value of 0xffff means a white pixel, a value of 0x0000 means black, and 0xf800 means red. Simple, isn't it? Here is a macro that converts a 24-bit RGB color to RGB-565:

#define RGB_TO_565(r,g,b) (WORD) ((r & 0xf8 )<<8) | ((g&0xfc)<<3) | ((b&0xf8)>>3)

Now, what do we need to display a pixel? The answer is, write a 16-bit value into the graphics memory. The graphics memory is called the frame buffer. Once you have the frame buffer pointer, you can write pixel data directly to video memory. What is the frame buffer address? This depends on the device. There are three documented ways to get the desired frame buffer address. I recommend combining them, so in case one is not supported, there is still another to try. These ways are:

1. Using the ExtEscape() GDI function. This Windows function returns the frame buffer address along with other parameters. It's simple, but it's not the best approach and it's likely that future devices will not support this. 
2. Using GAPI. GAPI is a graphics helper library (GX.dll) provided by Microsoft. Works fine with QVGA devices, but does not support the capabilities of VGA devices (only pixel doubling, which scales the display up), and it seems Microsoft will not update it any more.
3. Using DirectDraw. This is the best option, but it is only supported on Windows Mobile 5.0 devices.

I recommend using DirectDraw, and if it's not available on the device, using the others as a fallback. I'll show an example of these methods later.

There is one more thing to be aware of: pitches. I would assume, if I have a frame buffer pointer, and a certain fixed resolution, I can fill the screen, line by line using code like this:

int x, y;
WORD *p;

p=GetFramebuffer();

for(x=0; x<WIDTH; x++)
 for(y=0; y<HEIGHT; y++)
  *p++=0xffff;

This assumes that if you increment the buffer pointer by 2 (the size of a WORD), the pointer moves right one pixel. Also, this assumes that if you add WIDTH * 2, the pointer moves one pixel down. These assumptions are wrong, because of different frame buffer orientations. Without going into very much detail, what you have to keep in mind is:

1. The number of bytes to add to the pointer to move to the next pixel in a row is called XPitch.
2. The number of bytes to add to move to the next row (that is, one pixel down) is called YPitch.

So when calculating the pointer for pixel position (x, y), you have to include the pitch values:

pxy = pbuffer + x * xpitch + y * ypitch;

Let's see an example. The framebuffer pointer could be 0x10000000. For a VGA device, X and Y pitch values could be 2 and 960 (0x3c0). So the address of pixel (1,1) is:

0x10000000 + 1 * 2 + 1 * 960 = 0x100003c2.

On most systems, XPitch is 2 (the size of a pixel, in bytes). But in all cases, you have to query the pitch values. Pitch values also can be negative, so use a signed variable! How to get the pitch values? I'll explain in the next section.

Initializing graphics

I'll summarize the steps necessary to display some graphics:

• Create an application window
• Make it full-screen
• Get device parameters: frame buffer address and pitches.
• Write pixel data into frame buffer 

I think the first step needs no explanation; if you need help, use the Visual Studio AppWizard. It will create a fully functional application along with a nice window for you.

To make the window full-screen, let's use the following code:

RECT rc;

//hide task bar and other icons
SHFullScreen(g_hWnd, SHFS_HIDETASKBAR | SHFS_HIDESTARTICON | 
             SHFS_HIDESIPBUTTON);

//resize window to cover screen
SetRect(&rc, 0, 0, GetSystemMetrics(SM_CXSCREEN), 
        GetSystemMetrics(SM_CYSCREEN));
MoveWindow(g_hWnd, rc.left, rc.top, rc.right-rc.left, 
           rc.bottom-rc.top, false);

The code above will hide the Task Bar and the Start icon, by moving them "behind" your window, then, it will resize the window to cover the entire screen. Let's get device-dependent parameters. As said before, this can be done three ways. The first and in my opinion the simplest is, the Windows GDI function, ExtEscape(). Here is how I use it in the graphics class:

bool ZGfx::GfxInitRawFrameBufferAccess()
{
    RawFrameBufferInfo rfbi;
    HDC hdc;
    bool retval;
    retval=false;
    hdc=GetDC(m_hwnd);
    if(hdc)
    {
        if(ExtEscape(hdc, GETRAWFRAMEBUFFER, 0, 0, 
            sizeof(RawFrameBufferInfo), (char *) &rfbi))
        {
            if(rfbi.wFormat==FORMAT_565)
            {
                m_framebufwidth=rfbi.cxPixels; //store width
                m_framebufheight=rfbi.cyPixels; //store height

                m_xpitch=rfbi.cxStride; //store xpitch

                m_ypitch=rfbi.cyStride; //store ypitch

                m_cbpp=rfbi.wBPP; //store bits per pixel value

                m_framebuf=rfbi.pFramePointer; //store pointer

                retval=true;
            }
        }
        ReleaseDC(m_hwnd,hdc);
    }
    return retval;
}

The call returns all data in a structure, which is quite handy. The RawFrameBufferInfo structure is defined in GX.h. Please note that not all devices support this simple method. Also note that for performance reasons, it's recommended to use DirectDraw if it's available.

Let's have a look at DirectDraw. This approach needs a bit of caution. Even 1-2 year old devices running Pocket PC 2003 operating system do not support this feature. So if you want to be compatible with these devices (and you surely do!), you can't link to ddraw.dll (the DirectDraw library). This means, you can't call DirectDraw functions directly, as you usually call any other system function! Here is a typical way of calling a Windows API function:

MessageBox(hWnd, "Text", "Caption", MB_OK);

You simply type the function name, along with the parameters. I bet you've done this a million times before. As a result, the compiler/linker will include an import reference in your EXE file, to the system .dll file that contains the MessageBox() function. If the referenced .dll file can not be found, Windows will not load your EXE. In other words, an EXE containing DirectDraw calls, won't start if a device does not have DirectDraw, because there will be no ddraw.dll on the system. How to call DirectDraw while maintaining the compatibility with pre-WM5 devices? The answer is, you have to load the DirectDraw dll file manually using the LoadLibrary() API, then you have to find the entry point of the required DirectDraw function (namely, DirectDrawCreate()) with GetProcAddress(), then call the function through a pointer. No panic, it's simpler than it sounds:

//declare the DirectDrawCreate() function
typedef LONG (*DIRECTDRAWCREATE)(LPGUID, LPUNKNOWN *, LPUNKNOWN *); 

...
//load the library and look up the function

bool ZGfx::GfxLoadDirectDraw()
{
    m_hDD=LoadLibrary(L"ddraw.dll");
    if(m_hDD)
    {
        m_DirectDrawCreate=(DIRECTDRAWCREATE)
            GetProcAddress(m_hDD, L"DirectDrawCreate");
        return true;
    }
    else return false;
}

(Note: To use DirectDraw, you need the DirectDraw main header file, ddraw.h from the Windows Mobile 5.0 SDK. It's located in the ZGfx folder of the source codes pack. But to use the ZGfx class, there's no need to use the WM 5 SDK for your project. The Pocket PC 2003 SDK will do just fine. The Caleidoscope example uses the Pocket PC 2003 SDK, too.)

Okay, we have loaded DirectDraw, now let's get the display properties:

DIRECTDRAWCREATE m_DirectDrawCreate;    //our function declaration

IDirectDraw *m_pdd;
DDSURFACEDESC m_ddsd;
IDirectDrawSurface *m_psurf; 

...

bool ZGfx::GfxInitDirectDraw()
{
LONG hr;
    
    //create surface
    hr=m_DirectDrawCreate(0, (IUnknown **)&m_pdd, 0);
    if(hr!=DD_OK)
     return false;    //failed
    hr=m_pdd->SetCooperativeLevel(m_hwnd, DDSCL_FULLSCREEN);
    if(hr!=DD_OK)
    {
        m_pdd->Release();
        m_pdd=0;
        return false;
    }
    //create a simple buffer
    memset((void *)&m_ddsd, 0, sizeof(m_ddsd));
    m_ddsd.dwSize = sizeof(m_ddsd);
    m_ddsd.dwFlags = DDSD_CAPS;
    //no back buffering, only use the visible display area
    m_ddsd.ddsCaps.dwCaps = DDSCAPS_PRIMARYSURFACE;
    //create surface (entire screen)

    hr=m_pdd->CreateSurface(&m_ddsd, &m_psurf, NULL);
    if(hr!=DD_OK )
    {
        m_pdd->Release();
        m_pdd=0;
        return false;
    }
    //get parameters with locking
    
    memset((void *)&m_ddsd, 0, sizeof(m_ddsd));
    m_ddsd.dwSize = sizeof(m_ddsd);
    hr=m_psurf->Lock(0, &m_ddsd, DDLOCK_WAITNOTBUSY, 0);
    if(hr!=DD_OK)
    {
        //locking failed!


        m_psurf->Release();
        m_psurf=0;
        m_pdd->Release();
        m_pdd=0;
        return false;
    }
    //store data
    m_cbpp=m_ddsd.ddpfPixelFormat.dwRGBBitCount;
    m_xpitch=m_ddsd.lXPitch;
    m_ypitch=m_ddsd.lPitch;
    m_framebufwidth=m_ddsd.dwWidth;
    m_framebufheight=m_ddsd.dwHeight;
    //finally unlock surface

    m_psurf->Unlock(0);
    return true;
}

The code above creates a simple buffer with size of the entire screen. It's important to note that with DirectDraw, if you want to do anything with the display buffer, first you have to call the Lock() function, and once you're finished you must call Unlock(). In other words, you don't get a buffer pointer unless you lock the buffer! The code above needs locking only to get screen parameters, it does not draw anything yet.

The third way of getting display props is using GAPI. GAPI is Microsoft's graphics helper library, located in GX.dll. MSDN documentation describes all structures and functions provided by GAPI, so I'm not going into details. For initializing graphics, you only need two functions: GXGetDisplayProperties() and GXBeginDraw(). The former function returns a structure of type GXDisplayProperties. This structure contains the screen width, height values as well as the pitches. The second function returns a void *, which points to the frame buffer. When using GAPI, you must call GXBeginDraw() only before updating the screen, and after the drawing work is done, you must call GXEndDraw(). (This is the same idea as with DirectDraw locking.)

This is how I perform GAPI init:

bool ZGfx::GfxInitGAPI()
{
GXDisplayProperties prop;
int sw, sh;
    prop=m_GXGetDisplayProperties();
    m_cbpp=prop.cBPP;
    m_ypitch=prop.cbyPitch;
    m_xpitch=prop.cbxPitch;
    m_framebufheight=prop.cyHeight;
    m_framebufwidth=prop.cxWidth;
    if(!(prop.ffFormat&kfDirect565)) //verify pixel format

     return false; 
    //if it is a vga device, we have to check 
    //if GAPI's not messing it up

    sw=GetSystemMetrics(SM_CXSCREEN);
    sh=GetSystemMetrics(SM_CYSCREEN);
    if(sw!=prop.cxWidth || sh!=prop.cyHeight)
     return false;
    return true;
}
 

Don't use GAPI if it returns a screen resolution other than reported by GetSystemMetrics(). Note that GAPI does not support VGA devices. It supports a technique called pixel doubling. This means that the 240x320 display can be scaled up to 480x640 on a VGA device. But it's not VGA graphics!

Using the ZGfx class

The purpose of this class is to provide a basic level of abstraction and to make it easier to access the video memory. The class creates a software buffer, where all drawing can be performed. It also takes care of GAPI or DirectDraw initialization, whichever appropriate. You can safely assume that the software buffer XPitch will always be 2 (remember, the size of a pixel in bytes). Also you can assume that YPitch will be, two times the buffer width, that is, 960 for a 480-pixel wide buffer. To use the class, first add the zgfx.h and zgfx.cpp files to your project. For the .cpp file, disable using precompiled headers in the file properties section. Then add a global variable of type ZGfx to your application:

ZGfx g_gfx;

After your main application window has been created, you need to create a display buffer:

GfxRetval gr;
GfxSubsys gs;

gr=g_gfx.GfxCreateSurface(g_hWnd, g_screen_w, g_screen_h, &gs);

Parameters are: handle to the main window, requested buffer width and height, and a variable for the returned subsystem type. The return value will indicate success or failure (see the GfxRetval enum), and in case of success, the used subsystem (Raw frame buffer access, GAPI, or DirectDraw) will be returned (GfxSubsys). The routine will use DirectDraw (the best option) if available; if not, it will fall back to GAPI or Raw frame buffer access.

If the buffer is smaller than the screen resolution, it will be centered on the screen. A buffer cannot be larger than the device resolution. You can clear the entire display area, even if it is larger than your buffer, with the GfxClearHWBuffer() or GfxFillHWBuffer() functions. The latter takes a RGB-565 color value, which can be created with the RGB_TO_565() macro. To clear the software buffer, use the GfxClearScreen() function. The function takes a bool parameter that specifies whether the display should be updated. To update the display (that is, copy the software buffer contents to the video memory), use the GfxUpdateScreen() function.

You can access the software buffer with the GfxGetPixelAddress() function:

unsigned short *pbuffer;
GfxRetval gr;
gr=g_gfx.GfxGetPixelAddress(0, 0, &pbuffer);

The call above will return the address of pixel (0,0), that is, the beginning of the buffer (the upper-left corner). The most important functions of the class are:

• GfxGetPixelAddress() returns a pointer to the specified pixel. Use this before block operations.
• GfxDrawPixel() draws a pixel with a specified color.
• GfxGetPixelColor() returns the color of a pixel.
• GfxDrawLine() draws a line between two points.
• GfxFillRect() fills a rectangle on the screen with a color.
• GfxGetBufferYPitch() returns the number of bytes to add to the buffer pointer to move one pixel down.
• GfxGetWidth() returns the width of the software buffer.
• GfxGetHeight() returns the height of the software buffer. (note: to get the width/height of the physical display device, use the GetSystemMetrics() API function).
• GfxUpdateScreen() will copy the contents of the software buffer to the video memory. Use this function when drawing is done.
• GfxSuspend() will suspend the video buffer access. Use if if your application loses input focus.
• GfxResume() will resume the video buffer access.

There is a number of handy functions that should be added, I agree. For example, text output, drawing shapes or displaying bitmaps. I've written separate routines for these tasks, but they should be integrated into the graphics class.

To use this class in your application, you have to apply a slight modification to your window message handlers. Replace the WM_PAINT handler with this piece of code:

 case WM_PAINT: 
 {
    if(g_gfx.GfxIsInitialized())
    {
        ValidateRect(hWnd, 0); //no repaint

    }
    else
    {
        //gfx not yet inited, let windows draw

        return DefWindowProc(hWnd, message, wParam, lParam);
    }
    break;
 } 

I also recommend handling the WM_SETFOCUS and WM_KILLFOCUS messages, so if eg. a low battery warning pops up, you can suspend and resume drawing operations properly:

 case WM_SETFOCUS:
 {
    if(g_gfx.GfxIsInitialized())
    {
        g_gfx.GfxResume();
        g_gfx.GfxClearHWBuffer(); //clear & redraw screen

        g_gfx.GfxUpdateScreen();
    }
    g_focus++;
    break;
 }
 case WM_KILLFOCUS:
 {
    g_focus--;
    g_gfx.GfxSuspend();
    break;
 }

About the Caleidoscope application

I've provided a sample graphics application created with Visual Studio 2005, which draws a symmetric kaleidoscope-like image. This program illustrates the use of the graphics functions. I've commented the code so it should be a quick and easy reading. The application is based on a Windows GUI program created by the Visual Studio AppWizard. It creates an application window, and then periodically calls the DrawFrame() function, which renders the kaleidoscope image. The displayed gems are in fact small, filled rectangles. The gems are moved slowly in the upper-left quarter of the screen, and this quarter is mirrored to the right, pixel-by-pixel. Then the upper half of the screen is mirrored bottom, line by line. The DoMirror() function is a good example of pixel and line manipulations using pointers, and block movement with memcpy(). Upon key press or screen tap, the application will exit. It should work not only on Pocket PC 2003 and Windows Mobile 5.0, but also on SmartPhone too, because it does not use anything device-specific.

TODOs

The graphics routines could be improved in a number of ways:

• Add some drawing functions (circle, rectangle, etc.)
• Add support for displaying images (JPG, BMP)
• Add support for drawing text
• Optimize for speed, possibly by rewriting parts in ARM assembly
• Fix some small bugs

Any comments, suggestions are welcome.