Image Magic - Image Levels using Custom Controls

• Download application - 19.6 KB
• Download source - 182.46 KB

Introduction

Adjusting the levels of a digital image is one of the most basic image enhancement operations. Photoshop, The Gimp and Paint.Net provide a similar set of controls for image level adjustment. These controls have become accepted and for better or worse are what users expect. Unfortunately, one cannot create this type of UI using standard WPF controls. This article provides lookless custom controls for image level adjustment and the logic associated with them.

Goals

• Provide image level adjustment controls and logic for user experiments.
• Choose showing what is occurring over ease of use. No graph smoothing or value truncation.
• Just the basics. Avoid beautification and the implementation of seldomly used features. No eyedroppers, semi transparent colors, multiple channel adjustments or statistics.
• Explain well enough to avoid common user surprises.
• Provide an alternative to the standard gray point setting.

Background

Image levels adjustment is based on histograms. Histograms and the Photoshop like UI typically employed to modify them is a broad topic. For a quick introduction or refresher, go to cambridgeincolour. A histogram is a 256 byte array recording the number of pixels in an image at each intensity level. In this article, a pixel's total intensity is defined as (R+G+B)/3. The histograms are displayed in 2 histographs. The input histograph is displayed on top and remains constant. The output histograph is displayed on the bottom and is modified as the user changes values on the input and output sliders. The levels adjustment control consists of the following parts:

• Input Histograph - Displays the histographs of the unmodified image that was loaded. The histograph graphs the 3 color components of the image as well as its total intensity.
• Input Slider - A slider with 3 thumbnails used to adjust the image's black, gray and white values. The input slider modifies the distribution of intensity values in the histogram and their ranges.
• Output Histograph - Displays the histogram of the image after it has been modified by the input and output sliders.
• Output Slider - A slider with 2 thumbnails used to compress the intensity range of the image. Intensity values between output 0-black and white-255 are compressed out. In other words, the intensity range between input black and white is remapped to the range between output black and white.

Using the Code

What You Get

Four projects are provided in the solution:

1. JustXAML - No C# code. XAML only to display the customLevels control. Start here to get a feel for how the control works, resizing behavior and confirming it is a lookless control. The Histographs are empty as an image has not been loaded.
2. LevelsLogic - A DLL containing the classes to manipulate and remap histograms. This is the customer logic.
3. TestCustomLevels - An EXE coupling the CustomLevels control with LevelsLogic. Adds buttons to load and save images and another window for displaying an image.
4. WpfCustomRGBHisto - A DLL containing 4 custom controls. The CustomLevels control makes use of 3 other custom Controls. TwoThumb (OutputSlider), ThreeThumb (InputSlider) and Histograph (input and output graphs).

The TestCustomLevels is a standalone utility demonstrating the use of the CustomLevels control. The code requires references and Using statements to the namespaces of the DLLs for LevelsLogic and WpfCustomRgbHisto.

using WpfCustomRGBHisto;
using LevelsLogic;

The CustomLevels control is the interface to all the custom controls required to implement a PhotoShop like UI for image leveling. CustomLevels is more a C# class than a control. Its functionality is minimized to provide only absolute essentials. It's responsible for...

1. Finding the named Parts of the custom controls it contains and hiding them from the user.
2. Defining the LevelValues class which is used to communicate the value of all controls back to the user. The LevelValues class converts Double values in the custom controls to ints as that is all the user requires.
3. Providing a public event the user registers with to be informed of level value changes in any of the controls. The user is only informed of integer changes greater or equal to one.
4. Providing a method to set all controls to their default values when an image is loaded.

The CustomLevels control has no knowledge of how it is being used. To begin supporting image leveling, 2 buttons are required to load and save images. The XAML is extended to place the buttons in a second row in the Grid as follows:

<Window x:Class="TestCustomLevels.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:lib="clr-namespace:WpfCustomRGBHisto;assembly=WpfCustomRGBHisto"
     Title="TestCustomLevels" Height="500" Width="400" Topmost="True">
    <Grid Background="Transparent">
        <Grid.RowDefinitions>
            <RowDefinition Height="*"></RowDefinition>
            <RowDefinition Height="Auto"></RowDefinition>
        </Grid.RowDefinitions>
        <lib:CustomLevels x:Name="customLevels" Grid.Row="0" 
		Background="DimGray" FontWeight="Bold" />
        <StackPanel Grid.Row="1" Orientation="Horizontal">
            <Button Margin="2" Click="LoadImage_Click">Load Image</Button>
            <Button Margin="2" Click="SaveImage_Click">Save Image</Button>
        </StackPanel>
    </Grid>
</Window>

Examination of the code behind for TestCustomLevels shows 4 variables have been added. They are:

byte[] origRgb;         //unmodified rgb data from a file
byte[] modRgb;          //modified levels adjusted rgb data goes here, 
                        //inited when image loaded from file
HistoRemap histoRemap;  //represents image remapping specified by input 
                        //and output sliders
ImageWindow imgwind;    //image display, load and save

The rgb data from an image that is loaded goes into the origRgb array. Once loaded, the data in origRgb remains unchanged. It is the source of rgb data the user desires to levels adjust. The destination for the actual levels adjusted data is the modRgb array. The origRgb and modRgb arrays have identical Lengths. The histoRemap object is the only connection the program has with the LevelsLogic DLL that performs the levels adjustment. It consumes the level values set in the CustomLevels control and uses them to determine how to remap the original histograms and rgb data to the specified levels.

The imgwind object is responsible for loading, displaying and saving the image under adjustment. When loading an image from file, imgwind reads and stores the image data twice, once for the entire full sized image and once for an image reduced to the pixel dimensions of a maximized window. Only the reduced sized image data is level adjusted and displayed. On my camera, this reduces the length of the origRgb and modRgb arrays by a factor of ten. The reduction of data speeds up the levels adjustment while preserving the quality of the displayed image. On image save, the full sized image is levels adjusted and written to file.

Levels adjusting the image data is a reasonably fast and CPU efficient procedure. Surprisingly, testing revealed that if the sliders were repositioned rapidly, the CPU usage rose enough to cause WPF problems. This manifests itself as jerky slider motion. Sometimes the slider repositioning would even momentarily switch directions. The problem was fixed by performing the levels adjustment of image data in a background thread.

Adjusting Histograms

Histogram levels adjustment is the basis for the graphs that are displayed in the CustomLevels control. All histograms reside in a Histo object. This object actually contains 4 histograms, one for each of the colors R, G and B and one for the total intensity (R+G+B/3). A HistoRemap is used to remap each of the histograms to the levels indicated by the CustomLevels control. The remapping is accomplished by passing the current level values to the RemapAll method and then invoking the RemapHistoArray on each of the histograms. The resulting output Histo object is then sent to the CustomLevels control for display. The code below, taken from the LevelsChanged event handler shows how this is done:

histoRemap = new HistoRemap();              //for level mapping of new slider values
//using current slider values compute a level mapping
histoRemap.RemapAll(levels.InputBlack, levels.InputGray, levels.InputWhite,
	levels.OutputBlack, levels.OutputWhite);

Histo iHisto = customLevels.InputHisto;     //fetch input histos, 
                                            //constant for a given image
Histo oHisto = new Histo();                 //need to compute new output histos
foreach (HistoArray ha in Enum.GetValues(typeof(HistoArray)))   //each Histo has 
                                                     //r,g,b and intensity components
{
    int[] ia = iHisto.GetHistoArray(ha);    //original input histo component
    int[] oa = oHisto.GetHistoArray(ha);    //corresponding new output histo component
    histoRemap.RemapHistoArray(ia, oa);     //use level mapping to transform input 
                                            //histo component to output histo component
}
customLevels.OutputHisto = oHisto;          //update output histograph display in 
                                            //custom control

Adjusting the Image

After updating the output Histograph, the image is levels adjusted and redisplayed. This is done in the background by starting a background worker on the BckgrndRemapImageArray method. The method is passed a RemapImageArgs class containing a HistoRemap object and references to the source and destination rgb data arrays. The method is reproduced below:

private void BckgrndRemapImageArray(object sender, DoWorkEventArgs args)
{
    RemapImageArgs remapArgs = (RemapImageArgs)args.Argument;
    HistoRemap hr = remapArgs.Remap;
    hr.RemapImageArray(remapArgs.OrigRgb, remapArgs.ModRgb);                //do work
    args.Result = remapArgs.ModRgb;     //not used, but this is typically how it's done
} //BckgrndRemapImageArray()

What the Sliders Do

For most users, the best way to learn about levels sliders is to play with them. However if you are interested in image processing, you may find this section interesting. It describes what the sliders do and compares them with PhotoShop sliders. The level changes to be applied to the image are computed in the order below. A single pass is made through the image to apply all level changes.

1. Input Black and White Values
2. Input Gray Value
3. Output Black and White Values

InputSlider

The input Black and White values are used to compress the intensity range of the image. All image intensity values between 0 and the Black value are set to the Black value. Similarly intensity values between the White value and 255 are set to the White value. Counter intuitively, moving the Black thumbnail towards white makes the image darker. Moving the White thumbnail towards black makes the image darker. What seems to be mathematically impossible has an explanation. The output sliders were not modified. So all the dark value that were made lighter and the light values that were made darker are being remapped (expanded) to 0 and 255. The output histograph displays large increases in the number of pixels at the endpoints. The reason behind such an unlikely interface is it makes a better image. The intensity compression and subsequent expansion increases the range between input Black and White. More intensity values increases the image's contrast. This is almost always what one wants.

The input Gray value thumbnail is the most complicated control. It is used to vary intensity distribution, not range. Moving the thumbnail towards white/black makes the image lighter/darker. Changes to the Black or White values cause the Gray thumbnail to be repositioned. However, its value remains constant. If the Gray value is set at its default, 127, it will always be repositioned to the center of the range between Black and White values. The Gray value is only modified when the Gray thumbnail is directly moved by the user.

Intensity distributions may be changed by mapping them to a curve. Typically a Gamma curve is used as it describes how a physical device, like a camera, responds to light. The output of a Gamma curve is defined as b**Gamma, where b is input brightness normalized between 0-1.0 (just divide RGB intensity by 255). The actual value of Gamma being used is meaningless to the user. Photoshop like UIs display the Gamma value. A little digging reveals they actually display 1/Gamma rather than Gamma. The reciprocal is easier to program but it reverses the direction of the Gray value slider. Rather than display a Gamma value, the Input slider displays a value between 0-255. This is the intensity value gray (127) is remapped to by a Gamma curve. The actual value of Gamma needed to accomplish the remapping is calculated behind the scenes. The default Gray value is 127, corresponding to a Gamma of 1.0. This is a straight line with a slope of 45 degrees. In other words a Gray value of 127 means Gamma correction is not performed.

OutputSlider

The OutputSlider is used to control the intensity range and contrast of the image. Any movement of the output thumbnails from their default values lowers contrast! Moving the Black thumbnail towards white makes the image lighter. Moving the White thumbnail towards black makes the image darker. This is the reverse behavior of the Input Black and White thumbnails. The user is advised to avoid the OutputSlider. Its default values, 0 and 255, maximize the range and contrast of the image.

Similar to the InputSlider, the Output Black and White values compress the intensity range. All image intensity values between 0 and the Black value are set to the Black value. Intensity values between the White value and 255 are set to the White value. In addition, intensity values between Input Black and White are remapped to the range specified between Output Black and White using a straight line. Use of a straight line for output remapping washes out the image which makes one appreciate Gamma curves.

Histographs

Combing

If you open an image fresh from your camera, you may be surprised at the shape of the histographs. Instead of the well behaved continuous functions you were expecting, you see badly behaving functions that suddenly drop to zero or rise unexpectedly. Relax, both your camera and the software are working properly. I call this behavior combing. It's an artifact of digital manipulation. In the digital world, values are quantized not continuous. When an image adjustment, such as changing intensity level 10 to 20 occurs, the pixels with intensity level 10 are changed to occupy level 20. This leaves a hole or zeroing of level 10 and adds a spike to level 20. What is amazing is this behavior can provide the desired effect. Image data is manipulated by proprietary algorithms when a camera takes a picture. My camera really chops up the rgb data at the low end, but tries its best to produce a continuous total intensity curve especially in the midtones.

Scaling

The histographs are just a few Polygons inside a Grid. The Polygons are scaled so that the largest value fits in the Grid. As the sliders are repositioned, portions of the Polygons may appear to shrink. What is actually happening is the largest value in a Polygon, probably at the end points, is growing. As the Polygon is scaled to display the largest value, other unchanged values appear to shrink. Photoshop avoids this by truncating the largest Polygon value. It's a choice between an easy to see display and what is actually occurring.

History

• 24-May-2010: Initial release