Unzip Through RAPI

• Download source - 165.71 KB

Introduction

This article implements three solutions for expanding a ZIP file over an ActiveSync (RAPI) connection. ZIP files can be expanded from the device to the PC or from the PC to the device.

Background

I was recently asked by a customer to upgrade a specialized mobile application installer. The mobile application runs in a wide variety of devices, from Pocket PCs to specialized Windows CE devices, some of which are not even recognized by ActiveSync as devices (they show up as storage devices). Using a regular cab installer was out of the question and the adopted solution was to have a custom desktop application install the mobile application and resource files via direct copy to the device (or storage). Both the application and optional resource files are distributed in ZIP files that need to be expanded to the device upon installation. This was the first challenge. 

The second challenge was to produce an updater application that would search the device for the installed application and resource files and help to determine what components to update. Incidentally, some of the installed resource files are ZIP files themselves that contain one text file with a very simple description of the ZIP contents. Being very large (over 2 MB), it was out of the question to download them all to the PC just to extract one tiny file - for performance reasons, the PC updater application should extract the file on the device and copy it to the PC.

For quite some time, I have been using Lucian Wischik's Zip Utils published here and I used this code to solve my first challenge. During install, the ZIP files are decompressed on the PC to a memory buffer and then written to the device using RAPI function calls like CeCreateFile and CeWriteFile. The code works (it shipped this way) but I always got the impression that it could work faster if the decompression took place on the device and not on the PC. The first challenge was met, but I was not happy with the result: could it run faster?

Solving the second challenge was quite easy as the code was already in place. All I had to do was to replace the ZIP file open and read functions with RAPI equivalents. In order to reuse Lucian's code, I opted to use file pointers that would be initialized to the regular API calls (ReadFile, SetFilePointer, ...) when the ZIP file resides on the PC and to RAPI's (CeReadFile, CeSetFilePointer, ...) when the ZIP file resides on the device. Independently of where the ZIP file is, you can always decompress it very efficiently using this method (note that when the ZIP file is on the device, RAPI carries compressed data and not decompressed data as when you decompress a ZIP file from the PC to the device).

Finally I had to tackle the PC-to-device decompression performance issue by implementing a RAPI extension DLL. This article presents the results of this work.

Unzip Scenarios

The article's code explores three unzip scenarios:

1. The original PC-to-device decompression through regular RAPI calls as was implemented in the original installer code (fully rewritten here)
2. A device-to-PC decompression using regular RAPI calls
3. A PC-to-device decompression using a specialized RAPI extension DLL

All of these are implemented in the included sample command line application (CeUnzip). To use it, you will need a Windows Mobile device connected to the PC via ActiveSync (or Sync Center in Vista). The application has a very simple command line format:

CeUnzip -i|-e|-E ZipFileName.zip [TargetDirectory]

The three options map to the implemented scenarios like this:

1. -E: The ZIP file resides on the PC and is decompressed to the device using standard RAPI function calls.
2. -i: The ZIP file resides on the device and is decompressed to the PC using standard RAPI calls.
3. -e: The ZIP file resides on the PC and is decompressed to the device using a specialized RAPI extension DLL (see the CeRemZip solution in the included sources).

When CeUnzip is called, it reports the time in seconds it takes to decompress the whole ZIP file, so it really is interesting to compare the results from scenarios 1 and 3.

Before using the -e command line option, you need to be sure that you copy the included CeRemZip.dll to the device's \Windows directory. This DLL was compiled for Windows Mobile 5 devices, both Pocket PC and SmartPhone and should also run unmodified in Windows Mobile 6 devices (you should also be able to run it in Pocket PC 2003 devices, but I would advise you to recompile it for that target). The DLL is not signed, so you will see the dreaded unknown publisher warning upon first use.

Using the Code

I tried to make minimal changes to Lucian's simple interface. Towards this end, I added a few functions to the unzip.h file:

ZRESULT CeUnzipItem(HZIP hz, int index, const TCHAR *fn); 

Implements the scenario 3 decompression. The hz parameter refers to a desktop ZIP file and the fn parameter refers to a device file name.

HZIP CeOpenZip(const TCHAR *fn, const char *password);
HZIP CeOpenZipHandle(HANDLE h, const char *password); 

These functions implement scenario 2 decompression: the ZIP file resides on the device. To decompress the file to the PC, you can use the regular UnzipItem, FindItem and GetItem functions.

Scenario 1 is explicitly implemented in the sample command line application (see the ExportUnzip function). The ZIP file is handled through the original unzip function calls and the decompressed file is created and written through standard RAPI calls as implemented by the CRemoteAPI class. This class sole purpose is to implement dynamic linking to the RAPI.DLL, avoiding static linking through the version-specific RAPI.LIB.

As you can see from the sample application, initializing and shutting down the RAPI subsystem is the consuming application's responsibility.

Scenario 2 is implemented by the ImportUnzip function in the sample application. As you can see, the only change to a "normal" decompression routine is how the ZIP file is opened: CeOpenZip instead of OpenZip.

Finally, scenario 3 is implemented in the ExportUnzipServer function. Again, the only relevant change you see is in the call to CeUnzipItem instead of UnzipItem.

The RAPI Extension

On the device side, scenario 3 is implemented with the help of a RAPI extension DLL (CeRemZip solution in the sample code). The extension is implemented as a streamed RAPI call:

CEREMZIP_API 
int ZipServer(DWORD        cbInput,
              BYTE*        pInput,
              DWORD*       pcbOutput,
              BYTE**       ppOutput,
              IRAPIStream* pStream)

The PC application communicates with the RAPI extension solely via the IRAPIStream interface, sending commands and receiving acknowledgments. You can see this code in the unzip.cpp file inside a big...

#ifdef RAPI_SERVER

... block. The CeRemZip solution must define this precompiler constant, but the desktop application should not.

The meat of the code is in the StreamUnzip function. When an unzip command (ZMSG_UNZIP) is received by the executive loop in ZipServer, the StreamUnzip function is called and reads a RAPIUNZIP structure (defined in CeRemZip.h) from the stream containing all the information about the file or directory to create. The compressed data stream is read next in chunks of RAPI_BUF_MAX bytes and the decompressed stream written to the device file in blocks of FILE_BUF_MAX bytes. These two constants are very important for performance tuning: the larger, the better (but pay attention to the law of diminishing returns).

On the desktop side, the compressed stream is sent right after being read from the ZIP file (see the unzReadCurrentFile function in unzip.cpp). To make the least amount of changes to Lucian's code, the desktop also decompresses the data to a local buffer but discards it.

Performance

So how do scenarios 1 and 3 compare performance-wise? My findings are that scenario 3 is always faster than 1, although it may not be significantly so. The main factor that affects how faster 3 will be compared to 1 seems to be ZIP compression rates. Here are some of my own findings (times in seconds):

High Compression ZIP File

Low Compression ZIP File

As you can see, scenario 3 is always faster but the differences are really relevant when the ZIP file is highly compressed.

Note that the timings will depend heavily on the particular device and storage performance.

History

• 2007-08-22: Updated the RapiUnzip.zip file
• 2007-08-21: Article published