Adaptive Skin Color Detection

• Download source - 6.8 KB
• OpenVisionLibrary : Version 0.0.1

Introduction

In this article we will look at Adaptive Skin Color Detection technique described in the paper "An Adaptive Real-time Skin Detector Based on Hue Thresholding".

Background

Skin detection is an important problem in image analysis. Skin Color Detection is the process of finding skin colored pixels or regions in an image.By detecting regions of skin, one can often isolate the presence of faces, arms, hands, and gestures.

Skin Detection is often used in HCI(Human-Computer Interaction) applications and extensive research has been done in this area of last serveral decades.To date there does not exist a generic real time skin color detector which works under all lighting conditions,skin colors etc.

There are several approaches to build a skin color detector.The most basic one being a static skin color based thresholding.

Global Skin Color Detector

Each pixel of  RBG color image is often represented as combination of Red-Green-Blue channels or color contents.The static skin color based threshold would specify range of RGB values .Let L=(R1,G1,B1) and H=(R2,G2,B2) represent the ranges.

Let color of each pixel of image be represented as C=(R,G,B).Now we check each pixel of the image and if we find that R1<R<R2, and G1<G<G2 and B1<B<B2 ,ie the color of pixels lies in the specied range.The the pixel is labelled as skin colored pixels else it is labelled as non skin colored pixels.

The outut of static skin colored based thresholding stage is a binary image,with pixels assigned values 1 represent skin colored pixels and those assigned value of 0 represent non skin colored pixels.

However RGB representation of a color has many drawbacks.The primary reason being that appearance of color in an image depends on the illumination conditions (illumination geometry and color) where the image was captured.The important challange in color detection is to represent the perceived color in a way that is invariant or least sensitive to changes in illumination conditions.

The RGB color space does not satisfy this criteria.However HSV color spaces provides a good characteristics for color detection.In HSV color space each pixel of image is represented in terms of Hue-Saturation-Value.The H and S components characterize the color (independent of illumination) and V compoenent specifies the illuminations.Thus by specifying the skin color thresholds in terms of H-S thresholds we can achieve a illumination invaritant or atleast a color representation that is robust to illumination changes.

A skin color detector should ideally only detect the skin colored pixels and reject the pixels belonging to other objects or background.However since we are using a simple threshold based approach with no regard for other information the image ,the static skin color detector would also falsely detect some non skin colored pixels belonging to background or objects with similar hue color as skin like wood etc.

The amount of falsely detected pixels may be large in some situation compared to actual skin pixels,if a significant amount of scene contains objects with hue similar to the skin color.

The choice of image acquisition system,lighting conditions,pre-processing etc affect the choice of hue thresholds.

Hence the optimum thresholds needs to be decided adaptively.

The first component of adative skin detector is a static Global skin color detector.This will use a predefined threshold for detecting skin color.

It has been found that skin color lies in a very narrow range in H channel of HSV Colorspace.The lower and upper hue thresholds are selected as (0,33) repsectively.The Hue range provided is a generic thresholds that will cover all the possible skin colors.

Due to the generic nature of the skin threshold,some background objects whose hue is similar to skin or falls within the specified threshold may also be detected.

Also we do not want to select over exposed and underexposed regions in the image( dark,bright regions).Hence we specify an intensity thresholds as (15,250).

The global skin color detector classifier all the pixels in the image that have Hue values in range (0,33) and Intensity values in the range(15,250).

#include "adaptiveskindetector.hpp"

AdaptiveSkinDetector::AdaptiveSkinDetector()
{
    //static skin color thresholds
    _hueLower=3;
    _hueUpper=33;

    //initialising the global skin color thresholds
    lower=Scalar(_hueLower,50,50);
    higher=Scalar(_hueUpper,255,255);

    ......
    .......

}

void AdaptiveSkinDetector::run(Mat image,Mat &mask)
{

    //converting the image to HSV color space
    cv::cvtColor(image,image,CV_BGR2HSV);
    //applying the static skin color thresholds
    cv::inRange(image,lower,higher,mask);


    .....
    .....

We will be using OpenCV Image processing library for developing the code.As seen above the output of the thresholding stage is a image named "mask".

This is a binary image with pixel assigned values 1 are classified as skin colored pixels and those assigned values 0 are classified as non skin colored pixels.

In the below figure we can see that most of the skin colored pixels are detected using the threshold ,however some overexposed (bright) regions of image are ignored and some regions that are not skin colored like lips ,wall  etc are also selected.

Skin Color Histogram

The detected skin colored pixels can be used to construct a skin colored histogram which represents the statistical distribution of the skin colored pixels in the scene.

A histogram of image is simply the histogram of individual of individual H-S-V channels of the image.Histogram is a import tool in image analysis and helps is understand the data irrespective of the contextual content of the image.

We construct histogram considering only pixels labelled as skin color in the earlier stage.

//setting the mask for histogram
 h.setMask(mask);

//build histogram based on global skin color threshols
 Mat hist=h.BuildHistogram(hue,false);

 //normalize the histogram
 cv::normalize(hist,hist,0,1,NORM_MINMAX);
 //update the histograms
 h.setHist(hist);

In the above code "h" is a object of class that encapsulates histogram computation.We specify that histogram should only be computed by considering the pixels that have value of (255) in the image represented by "mask".

Next let us assume that somone tells us some pixels of the image that truly are skin colored.This is information that we have obtained by some other means.

Let us again construct a histogram considering these pixels.

Thus we have 2 histograms ,one obtained by considering the result of global skin color detector and other histogram obtained by considering the pixels truly belonging skin regions of the image.

Next to get a average represetation of color content of image,we perform a weighted average of the two histograms.

A good result is obtained by choosing a value in the range 0.02-0.05 for weights.

A typical step in statistical methods is to reject the outliers in the data.

The outliers in the data are rejected by considering only data that contributes to signifcant part of histogram or data corresponding to components whose relative frequency compoenents are higher that a certain thresholds.

The criteria used for selection of lower and upper thresholds is such that area under the histogram covers \(f % \).

Thus we select a lower and upper threshold,so that area under the histogram is \(f% of total area of histogram\). In the paper a criteria of 90-96% was used

    //mmask is the binary image representing true skin colored pixels
    //set the new histogram mask
    h.setMask(mmask);

    //compute the histogram based on updated mask
    Mat shist=h.BuildHistogram(hue,false);
    //normalize the histogram
    cv::normalize(shist,shist,0,1,NORM_MINMAX);

    //merge both the histograms with weight of 0.02
    h.mergeHistogram(shist,0.02);

    //get the final histogram
    hist=h.getHist();

    //get the histogram thresholds so that 90% or 10% does not lie within the thresholds
    h.getThreshHist(hist,0.05,0.05);

    //update the histogram thresholds
    _hueLower=range1[0];
    _hueUpper=range1[1];

The final skin color detector output is based on the newly computed thresholds after removing the outliers from the histogram.This _hueLower and _hueUpper are used as thresholds and skin color thresholding is perfomed again.

Below is the output of adaptive skin color detection technique.It can be seen that some non skin colored regions that were detected earlier are rejected after using modified thresholds.Thus adaptive skin color detection technique helps reject false positives of skin colored pixels in the image and therby represents the true skin colored pixels in the image.

Without any additional information ,the criteria used for selection of lower and upper threholds ,simply is instrumental in removal of outliers.

In the figures shown ,some background pixels,pixels belonging to hair,lips etc are also shown as skin colored pixels.

Now let is consider pixels which belong to face,In the above example the true skin colored pixels are provided manually .This in given manually by specifying a mask.Pixels in ROI (168,63,50,50) is explicitly specified as skin colored pixels.

A histogram is computed by considering only the pixels in ROI.

Obviously the pixels in histogram computed over entire image are large than ones computed in small ROI,to avoid bias due to count of pixels used to build the histogram, the histograms are normalized between 0 to 1,before computing the linear combination.

Then we determine the pixels between which 90\% of pixels lie.

The hue range corresponding to this is obtained as (13,16)

The skin color detected considering the new range in shown in figure 1

Thus incorporating the cue's about skin color, enhances the detection performane of skin colored pixels.

In the above example the cue has been encorporated manually, however if we can incorporate the cur automatically then we have make the process of skin color detection completely adaptive.

Using Motion Cues

Some techniques suggested in the paper were based on using motion based cue like frame differencing and optical flow tracking.These techniques are suitable for HCI application assuming the object of interest is in motion.Frame differencing provides a simple method to determine the region which encountered motion and use these pixels .

In HCI applications hand or face regions are used to communicate with the computer and assuming that dominant motion in the scene belongs to hand and skin pixels.

One of the ways to detect the regions belonging to skin regions is motion tracking.We track the regions in the image that encounted motion and assume that these belong to the hand/face region. Upon detecting these region we can determine the HSV threshold ranges for these pixels.

The Hue thresholds are adaptively changed by observing the motion of skin colored pixels in the image.

Thus the first step is to determine the in motion skin colored pixels.

    //obseve the pixels encountering motion
    Mat mmask=Mat();
    if(!p1.empty())
    {

        Mat motion;
        //computing the different in present and past frames
        cv::absdiff(p1,ch[2],motion);
        //consider the pixels with motion threshold above 8
        cv::inRange(motion,Scalar(8,0,0),Scalar(255,0,0),mmask);
        //filering the mask by performing morphological operations
        cv::erode(mmask,mmask,Mat());
        cv::dilate(mmask,mmask,Mat());
    }

Using the code

#include "ImgProc/adaptiveskindetector.hpp"
//...
AdaptiveSkinDetector ss1;
...
Mat hmask;
ss1.run(image,hmask);
//...

Code

The code for the same can be found at OpenVision Repository https://github.com/pi19404/OpenVision The class AdaptiveSkinDetector encapsulates the methods for implementing the skin detector. The code for the same can be found in files ImgProc/adaptiveskindetector.cpp IgmProc/adaptiveskindetector.hpp. For histogram computation the class Histogram is used which can be found in the files ImgProc/Histogram.cpp,ImgProc/Histogram.hpp

References

1. Farhad Dadgostar and Abdolhossein Sarrafzadeh. An Adaptive Real-time Skin Detector Based on Hue Thresholding: A Comparison on Two Motion Tracking Methods . In: Pattern Recogn. Lett. 27.12 (Sept. 2006), pp. 1342 1352.  issn:0167-8655. doi: 10.1016/j.patrec.2006.01.007. url: http://dx.doi. org/10.1016/j.patrec.2006.01.007.

Article Download