Apply Compression to RAW Images without Sacrificing Quality: Introduction to JPEG XR

An Alternative to RAW

With the increasing availability of Digital SLR cameras in the consumer market, many users are choosing to capture images in their camera manufacturer’s proprietary RAW format in order to retain image information that is normally lost when capturing to the default JPEG format. Retaining as much information as possible is crucial to the workflow of most photographers so that they can perform post-processing with minimal loss of quality. Additionally, the rising popularity of HDR, Tilt-Shift, and other photographic techniques have made retaining greater dynamic range in digital images paramount.

The advantage of the RAW format is that cameras capture RAW images with greater intensity resolution than JPEG is capable of storing. Many cameras now employ CCDs with more than 8 bits per color (or 24 bits per pixel (BPP)) but the use of JPEG discards image intensity data even before the image is compressed, which loses even more information. The RAW format includes all of the intensity information acquired by the camera including typically 10 to as much as 14 bits per color (or 30BPP-42BPP) as well as additional information associated with the image.

The problem is that the RAW formats cannot be viewed by most imaging applications and require much more storage space than the default JPEG format, making them difficult to transmit over networks or email.

JPEG XR (derived from Microsoft’s HDPhoto which Microsoft originally named Windows Media Photo) is a new compression ISO standard which supports lossy and lossless compression of images over a wide range of image types. This new file format gives photographers a great alternative to RAW files by retaining all of the information needed for editing while providing higher image compression efficiency. JPEG XR can also be viewed in a number of image viewers and editors including Adobe Photoshop, Microsoft Windows Photo Viewer, and therefore Windows Explorer, just like standard JPEG images.

Converting RAW to JPEG XR

Using the PICTools imaging SDK from Accusoft Pegasus, conversion from many proprietary camera RAW formats to compressed JPEG XR is simple. The remainder of this document will describe how to create a simple C++ console application which reads a RAW format image file and extracts a 48BPP RGB image from it using the PICTools Camera Raw Extraction (CAMERARAWE) opcode. We’ll then compress the 48BPP image to a JPEG XR image (even including the camera/image metadata) using the PICTools HDPhoto/JPEG XR opcode (HDPHOTOP) , write the resulting file and view it with the Windows Photo Viewer. Let’s take a look at the code.

RAW to JPEG XR Code

The PICTools opcodes extract and compress images through the use of input and output memory buffers, so the first step is to open the RAW file, allocate a chunk of memory, and read the file into the buffer. The buffer is then passed to a function, CameraRawExtract, which extracts the image as a 48BPP RGB image and also extracts the metadata as an ordered data structure. The metadata is converted to Tiff/EXIF tags and passed along with the image to the JPEGXRSave function. That function compresses the image into JPEG XR format which includes the Tiff/EXIF data and returns the compressed image that is then written out to a new file.

    // Open the camera raw file
    ifstream file(szFileName, ios::in|ios::binary|ios::ate);
    if (file.is_open())
    {
        // Read the file into a buffer
        DWORD dwInputLen = static_cast<DWORD>(file.tellg());
        LPBYTE pbInput = new BYTE[dwInputLen];
        file.seekg (0, ios::beg);
        file.read (reinterpret_cast<char*>(pbInput), dwInputLen);
        file.close();
 
        // Extract a raw image from the file
        PIC_PARM    picParmJXR;
        LPBYTE      pRaw = NULL;
        DWORD       dwRawLen = 0;
        if( CameraRawExtract(pbInput, dwInputLen, picParmJXR, pRaw, dwRawLen) )
        {
            // Relocate TIFF Tags
            P2LIST TIFFTags;
            if(CreateTIFFTags(picParmJXR, TIFFTags))
            {
                // Compress the raw image to JXR
                LPBYTE      pbJXR = NULL;
                DWORD       dwJXRLen = 0;
                if(JPEGXRSave(picParmJXR, pRaw, dwRawLen, TIFFTags, pbJXR, dwJXRLen))
                {
                    // Write the JXR to the output file
                    ofstream ofile (szOutputFileName, ios::out|ios::binary|ios::trunc);
                    ofile.write(reinterpret_cast<char*>(pbJXR), dwJXRLen);
                    ofile.close();
 
                    delete[] pbJXR;
                }
                P2LFree(reinterpret_cast<P2LIST*>(&TIFFTags));
            }
            P2LFree(reinterpret_cast<P2LIST*>(&picParmJXR.PIC2List));
            delete[] pRaw;
        }
        delete[] pbInput;
    }

Now that we have the big picture, let’s look at the interesting work that happens in the CameraRawExtract, CreateTiffTags, and JPEGXRSave functions.

Camera RAW Extraction

Once the file is in memory, it is passed to the CAMERARAWE opcode to extract the image. All of the PICTools opcodes are executed through a single function which is configured through the PIC_PARM structure. This structure contains information about the operation to be performed, an input buffer, metadata buffer, and an output buffer.

BOOL CameraRawExtract(
    LPBYTE pbInput,     // RAW file buffer
    DWORD dwInputLen,   // RAW file buffer length
    PIC_PARM& picParm,  // Operation PIC_PARM Parameters
    LPBYTE& pbRaw,      // Output image buffer
    DWORD& dwRawLen)    // Output image buffer length
{

The Camera Raw operation is configured by first initializing the PIC_PARM structure passed to the function. The Camera Raw Extract operation is selected by setting the Op and ParmVerMinor members.

    // Initialize the operation
    memset( &picParm, 0, sizeof(PIC_PARM) );
    picParm.ParmSize     = sizeof(PIC_PARM);
    picParm.ParmVer      = CURRENT_PARMVER;
    picParm.Op           = OP_CAMERARAWE;
    picParm.ParmVerMinor = 2;

Select the output of the Camera Raw operation to be a RAW image as opposed to a bitmap image because the maximum resolution of a bitmap image is 24BPP and we want to exceed that.

    // Raw supports a larger range of image geometries
    picParm.Flags = F_Raw;

Then set the region flags for the Camera Raw member (CRE) to indicate that we want two bytes for each channel and for the image to be an RGB image. The resulting output will be a 48 BPP RGB image.

    // Extract the image as 48 BPP (2 Bytes each of Blue, Green, and Red)
    picParm.u.CRE.Region.Flags = RF_2Byte | RF_SwapRB;

The resulting image will be an array of pixels where each pixel is composed of 6 bytes: 2 bytes each of red, blue, and green data in little endian byte order.

So, for an average DSLR Camera which captures 3000 pixel x 2000 pixel images, this buffer will be about 36 megabytes.

Next, the Get buffer is configured to read the RAW file buffer that was passed into the function. The Get buffer is a circular queue that can be used in several different configurations. The Start and End members point to the start and end of the input buffer and the Front and Rear members point to the location in the buffer to be read by the operation. In this case, it’s going to behave as a single buffer containing the entire input data, but it could be configured to read parts of the file in chunks in order to reduce memory overhead. The Q_EOF flag indicates that all of the image data is in the buffer so the operation will not need to request further data.

    // Set up the Get (input) buffer
    picParm.Get.Start  = pbInput;
    picParm.Get.End    = pbInput + dwInputLen;
    picParm.Get.Front  = picParm.Get.Start;
    picParm.Get.Rear   = picParm.Get.End;
    picParm.Get.QFlags = Q_EOF;

The operation itself is a three part process: Initialization, Execution, and Termination. During the initialization phase, the opcode parses the RAW data to identify the camera, determine the output image size and extract any image metadata.

    // Call REQ_INIT
    if ( !PicReq(picParm, REQ_INIT) )
    {
        P2LFree( reinterpret_cast<P2LIST*>(&picParm.PIC2List) );
        return ( FALSE );
    }

During the initialization phase, the Camera RAW opcode populates the CRE.Region member with the image geometry. Once that is known, we can then allocate space for the output image. The height is the height of the image in pixels. The stride is the width of the image in bytes. Therefore their product is the number of bytes required to contain the output image.

    // After init, the image geometry is known, so allocate the output buffer
    dwRawLen = picParm.u.CRE.Region.Height * picParm.u.CRE.Region.Stride;
    pbRaw = new BYTE[dwRawLen];
    if ( NULL == pbRaw )
    {
        P2LFree( reinterpret_cast<P2LIST*>(&picParm.PIC2List) );
        picParm.Status = ERR_OUT_OF_SPACE;
        return ( FALSE );
    }

Note that if the allocation above fails, the PIC2List member of the PIC_PARM is freed via a call to P2LFree. We’ll cover this in more detail later, but for now, we’ll just say that the PIC2List member contains any metadata extracted from the image by the Camera Raw operation for example the ISO setting used when the picture was taken or the camera manufacturer.

Similar to the Get buffer configuration earlier, the Put buffer is configured to point to the output buffer which will receive the extracted image during the execution phase except the Put buffer is initially configured to be empty (Front == Rear).

    // Set up the Put (output) buffer
    picParm.Put.Start  = pbRaw;
    picParm.Put.End    = pbRaw + dwRawLen;
    picParm.Put.Front  = picParm.Put.Start;
    picParm.Put.Rear   = picParm.Put.Front;

Next, the image is extracted from the RAW file and the operation is completed in the termination phase which releases any internally allocated resources.

    // Extract the image and complete the operation
    if ( !PicReq(picParm, REQ_EXEC) || !PicReq(picParm, REQ_TERM) )
    {
        P2LFree( reinterpret_cast<P2LIST*>(&picParm.PIC2List) );
        delete[] pbRaw;
        return ( FALSE );
    }
 
    return ( TRUE );
}

TIFF/EXIF Tag Creation

In addition to the image’s extraction, the Camera RAW opcode also extracts metadata as TIFF tags. We’ll be including these output tags in the final JPEG XR image, but to conform to the specification, all of the EXIF tags must be located in the EXIF section. In the CreateTIFFTags function we’ll iterate through the tags returned from Camera RAW and create a new properly formed set of tags.

BOOL CreateTIFFTags(PIC_PARM& picParm, P2LIST& newTags)
{
    // Initialize result list
    newTags.list = NULL;
    newTags.len  = 0;
    newTags.size = 0;

The P2LIST data structure and related functions allow provide a simple interface for dealing with Tiff Tags. Using P2LFirstTiffTag allows us to get a pointer to the first Tiff tag. Subsequent calls to P2LNextTiffTag will return the next tag in the list until the end is reached.

    // Iterate the the tiff tags and create a new set of tags 
    // with EXIF tags in the proper location
    P2PktTiffTag* pkt = P2LFirstTiffTag( reinterpret_cast<P2LIST*>(&picParm.PIC2List), 0 );
    while(pkt)
    {

Depending on the actual tag value, we’ll add the tag to the output tags in either the primary image tag section or add it to the EXIF section.

       switch( pkt->TiffTag )
       {
       case TAG_ISOSpeedRatings:
       case TAG_ShutterSpeedValue:
       case TAG_ApertureValue:
       case TAG_FocalLength:
       case TAG_DNGVersion:
           // These EXIF tags need to be relocated to the EXIF location.
           if( !P2LAddTiffTag( &newTags, LOC_EXIFIFD, 
                   pkt->TiffTag, pkt->TiffType, pkt->TiffCount, pkt->TiffData ))
           {
               return FALSE;
           }
           break;
 
       default:
           // Other TIF tags can be added with no location.
           if( !P2LAddTiffTag( &newTags, LOC_PRIMARYIMAGEIFD,
                   pkt->TiffTag, pkt->TiffType, pkt->TiffCount, pkt->TiffData ) )
           {
               return FALSE;
           }
           break;
       };
 
       pkt = P2LNextTiffTag( (P2LIST*)&picParm.PIC2List, pkt );
    }
 

   return TRUE;
}

After the new tags have been created, they are returned for use in the JPEG XR compression step.

JPEG XR (HDPhoto) Compression

The final step is to compress the RAW image and include the new TIFF tags created in the last step.

BOOL JPEGXRSave( 
    PIC_PARM& picParmRaw,   // Raw operation PIC_PARM result
    LPBYTE pbRaw,           // Raw image buffer
    DWORD dwRawLen,         // Raw image buffer length
    P2LIST& p2list,         // Relocated TIFF Tags
    LPBYTE& pbOutput,       // Output JXR buffer
    DWORD& dwOutputLen )    // Output JXR buffer length
{
    pbOutput = NULL;
    dwOutputLen = 0;

The HDPhoto compression operation is configured by first initializing the PIC_PARM structure passed to the function. The HDPhoto compression operation is selected by setting the Op and ParmVerMinor members in the same way we set up the Camera Raw operation earlier.

    // Initialize the operation
    PIC_PARM picParmJXR;
    memset( &picParmJXR, 0, sizeof(PIC_PARM) );
    picParmJXR.ParmSize     = sizeof(PIC_PARM);
    picParmJXR.ParmVer      = CURRENT_PARMVER;
    picParmJXR.Op           = OP_HDPHOTOP;
    picParmJXR.ParmVerMinor = 2;
 
    // Since the input image is 48BPP, we can’t use F_Bmp  
    picParmJXR.Flags = F_Raw;

The Camera Raw operation filled the CRE.Region with information about the image geometry during its initialization phase. We’ll copy that information to the HDPhoto’s Region (HDP.Region) so the HDPhoto operation will know the height, width, pixel depth, etc. when it begins compressing the image.

    // Copy the output region information from the CameraRaw 
    // extraction to the HDPhoto input region
    picParmJXR.u.HDP.Region = picParmRaw.u.CRE.Region;

We’ll also copy the TIFF tags we created earlier to the PIC2List member so that they can be added to the compressed image.

    // Copy the tiff tags to the output
    picParmJXR.PIC2List = reinterpret_cast<char*>(p2list.list);
    picParmJXR.PIC2ListLen = p2list.len;
    picParmJXR.PIC2ListSize = p2list.size;

Since the JXR standard supports 48 BPP RGB (and not 48 BPP BGR), we’ll set the flag to indicate that the red and green channels should be swapped.

    // The output image is going to be 48BPP RGB, so swap the Red and Blue 
    picParmJXR.u.HDP.Region.Flags = RF_SwapRB;

Next we’ll set up the Get buffer in the same way as we did for the Camera Raw operation, but the difference this time is that the input is the 48 BPP RGB image and not the proprietary RAW format data.

    // Set up the Get (input) buffer 
    picParmJXR.Get.Start  = pbRaw; 
    picParmJXR.Get.End    = pbRaw + dwRawLen;
    picParmJXR.Get.Front  = picParmJXR.Get.Start;
    picParmJXR.Get.Rear   = picParmJXR.Get.End;
    picParmJXR.Get.QFlags = Q_EOF;

The output image will require a buffer to be written to, so we’ll allocate one that is the size of the entire image. That size is a good choice because we expect that the compressed image will be smaller than the original image.

    // The compressed image should be smaller than the orignal, so the output 
    // buffer won't exceed the size of the original image.  
    dwOutputLen = picParmJXR.u.CRE.Region.Stride * picParmJXR.u.CRE.Region.Height;
 
    // Allocate the output buffer
    pbOutput = new BYTE[dwOutputLen];
    if ( NULL == pbOutput )
    {
        return ( FALSE );
    }

The Put buffer is configured to point to the newly allocated memory which will contain the compressed image just like we did in the Camera Raw operation.

    // Set up the Put (output) buffer
    picParmJXR.Put.Start  = pbOutput;
    picParmJXR.Put.End    = pbOutput + dwOutputLen;
    picParmJXR.Put.Front  = picParmJXR.Put.Start;
    picParmJXR.Put.Rear   = picParmJXR.Put.Front;

Next we’ll configure the compression parameters. We want to compress to the ISO Standard JPEG XR format instead of the older HDPhoto format. We’ll also select the Quantization used to compress the image. It ranges from 1-255 and defines the resulting image quality. Lower values yield higher quality while higher values result in higher compression. A value of 1 will specify lossless compression. We’ve selected 128 as providing a good trade-off between quality and compression size for 48BPP RGB images.

    // Enable JPEG-XR output
    picParmJXR.Flags2 = PF2_JPEGXR_Format;
    picParmJXR.u.HDP.Quantization = 128;

Finally we’ll do the compression. Unlike the Camera Raw operation, we do all three steps (Initialization, Execution, and Termination) at one time.

    // Compress the image
    if ( !PicReq( picParmJXR, REQ_INIT ) 
        || !PicReq( picParmJXR, REQ_EXEC ) 
        || !PicReq( picParmJXR, REQ_TERM ) )
    {
        free( pbOutput );
        return ( FALSE );
    }

The compressed image will be smaller than the original, so we determine the size from the difference between the Put buffer’s Rear and Front pointers which are set by the opcode during compression. We’ll reallocate the memory to free the unused portion, and set the return values.

    // Determine the size of the resulting compressed 
    // image and return the output.
    dwOutputLen = static_cast<DWORD>( picParmJXR.Put.Rear - picParmJXR.Put.Front );
    
    return TRUE;
}

How Does the Quality compare to RAW?

A Nikon D60 RAW sample image (RAW_NIKON_D60.NEF) was used to compare the compression quality (shown below) and many other RAW camera formats are available at www.rawsamples.ch.

Compressed with a quantization of 128 and then decompressed, the image was compared with the original RAW image by calculating the Peak Signal to Noise Ratio and Mean Square Error. The results, indicating a high correlation with the original image are shown below.

When the image was extracted from the Camera RAW file in the sample as a 48 BPP RGB image, the original image size was:

3900 rows x 2613 columns x 3 channels x 2 bytes/channel = 61,144,200 bytes

The resulting compressed file was only 1,966,666 bytes which is a compression ratio of over 31:1.

The compressed JPEG XR file saved as RAW_NIKON_D60.wdp (Windows Media Photo) is shown in the Windows Photo Viewer. The Windows Explorer file properties shows the Tiff/EXIF tags set in the sample code.

Conclusion

The ability to compress camera RAW images with JPEG XR allows for high compression rates with very low loss in image quality, while retaining all of the RAW data needed for editing. The PICTools SDK from Accusoft Pegasus makes the extraction and compression task straightforward while supporting a large number of RAW formats and ISO standard JPEG XR compression. The growing number of image viewing and editing applications which support the JPEG XR standard make it an excellent choice for archiving images as well as viewing and sharing them without the need for proprietary RAW format support.

JPEG XR truly is the ideal compromise between the accuracy of RAW and the usability of an established compression format.

Download the PICTools imaging SDK at www.accusoft.com. Please contact us at sales@accusoft.com or support@accusoft.com for more information.

About Accusoft Pegasus

Founded in 1991 under the corporate name Pegasus Imaging, and headquartered in Tampa, Florida, Accusoft Pegasus is the largest source for imaging software development kits (SDKs) and image viewers. Imaging technology solutions include barcode, compression, DICOM, editing, forms processing, OCR, PDF, scanning, video, and viewing. Technology is delivered for Microsoft .NET, ActiveX, Silverlight, AJAX, ASP.NET, Windows Workflow, and Java environments. Multiple 32-bit and 64-bit platforms are supported, including Windows, Windows Mobile, Linux, Sun Solaris, Mac OSX, and IBM AIX. Visit www.accusoft.com for more information.

About the Author

Prior to joining Accusoft Pegasus as a Senior Software Engineer in 2007, Steve Brooks developed instrument control, imaging, and similar scientific software in MFC, ATL, and .NET. He has consulted for Microsoft and NASA and his prior employers include the University of North Carolina at Chapel Hill, Stingray Software, and Varian Inc. Steve earned a Bachelor of Science in Physics from Appalachian State University where he began developing in Turbo C.