Contents
This article demonstrates an application to create an oil painting effect of an image. Basically this effect is achieved by examining
the nearest pixels for all pixels.
For every pixel, it finds the maximum repeated color and that color will be considered as the output. In effect, we will get
a blocky image with less information,
and it will be similar to the painting effect of the image.
Screenshot of the application with painting effect.
Screenshot of the application without oil painting effect.
Details of the oil painting algorithm are explained below.
Oil Painting Effect Details
Analysis of nearest pixels is explained with a real example. A pixel from the above image is considered to analyze the
oil painting algorithm. This example considers
Radius as 2 and Intensity as 10.
For every pixel, the surrounding pixels are analysed. Find
the maximum repeated pixels within the area, and place it as output.
Processing of a pixel [X,Y] analyzes pixels from [X-Radius,Y-Radius] to [X+Radius,Y+Radius].
void PaintEffect::Process( const BYTE* pbyDataIn_i,
const int nRadius_i,
const float fIntensityLevels_i,
const int nWidth_i,
const int nHeight_i,
BYTE* pbyDataOut_o )
{
for( int nY = nRadius_i; nY < nHeight_i - nRadius_i; nY++)
{
for( int nX = nRadius_i; nX < nWidth_i - nRadius_i; nX++)
{
for( int nY_O = -nRadius_i; nY_O <= nRadius_i; nY_O++ )
{
for( int nX_O = -nRadius_i; nX_O <= nRadius_i; nX_O++ )
{
int nR = pbyDataIn_i[( nX+nX_O) * 3 + ( nY + nY_O ) * nBytesInARow ];
int nG = pbyDataIn_i[( nX+nX_O) * 3 + ( nY + nY_O ) * nBytesInARow + 1];
int nB = pbyDataIn_i[( nX+nX_O) * 3 + ( nY + nY_O ) * nBytesInARow + 2];
int nCurIntensity = ( ( ( nR + nG + nB ) / 3.0 ) * fIntensityLevels_i ) / 255;
if( nCurIntensity > 255 )
nCurIntensity = 255;
int i = nCurIntensity;
nIntensityCount[i]++;
nSumR[i] = nSumR[i] + nR;
nSumG[i] = nSumG[i] + nG;
nSumB[i] = nSumB[i] + nB;
}
}
int nCurMax = 0;
int nMaxIndex = 0;
for( int nI = 0; nI < 256; nI++ )
{
if( nIntensityCount[nI] > nCurMax )
{
nCurMax = nIntensityCount[nI];
nMaxIndex = nI;
}
}
pbyDataOut_o[( nX) * 3 + ( nY ) * nBytesInARow ] = nSumR[nMaxIndex] / nCurMax;
pbyDataOut_o[( nX) * 3 + ( nY ) * nBytesInARow + 1] = nSumG[nMaxIndex] / nCurMax;
pbyDataOut_o[( nX) * 3 + ( nY ) * nBytesInARow + 2] = nSumB[nMaxIndex] / nCurMax;
}
}
}
Here analyzing the nearest pixels of a pixel (X,Y), for this example, X,Y is at the center of
the blue rectangle. The nearest pixels of the pixel X,Y is displayed in the right top corner. We will analyse the intensity of these nearest pixels and find out the final R, G, B for pixel (X,Y) which will help create
an oil painting effect.
The RGB values of the nearest pixels are shown in the below image.
Intensity of the above pixels are calculated with the following logic. The average of R ,G, and B is multiplied with Intensity and the final intensity value
is prepared.
int nCurIntensity = ( ( ( nR + nG + nB ) / 3.0 ) * fIntensityLevels_i ) / 255;
The calculated intensity of the nearest pixels are shown in the below image.
The intensity values of the nearest pixels range from 6 to 10. The final pixel value depends on the occurrence of the intensity values.
The most repeating intensity
value is considered for the output. From the above image, the most repeating intensity is 8.
Seven nearest pixels have the intensity value 8. On calculating the intensity of each pixel,
the sum of R, G, B corresponding to each intensity is also calculated.
nSumR[nCurIntensity] = nSumR[nCurIntensity] + nR;
nSumG[nCurIntensity] = nSumG[nCurIntensity] + nG;
nSumB[nCurIntensity] = nSumB[nCurIntensity] + nB;
The sum of R, G, B component for each intensity value is as shown below.
From the above table, the intensity value 8 is the most repeating intensity.
Seven nearest pixels have the intensity value 8, and therefore the output pixel is prepared from the sum
of R, G, B components corresponding to intensity 8.
Final R = 949 / 7 = 135
Final G = 902 / 7 = 128
Final B = 458 / 7 = 65
And at last, RGB (131,151,82) at X,Y is changed to RGB (135,128,65) after analyzing RGB values of
the nearest pixels.
The output pixel is changed according to the intensity of the nearest pixels.
The output image is created by applying the same algorithm for all pixels. The maximum repeated pixels
are considered for output, and it removes small (smooth) changes in the image. In effect we will get a rough/blocky image. It will be similar to that of an oil painting effect.
Parameters of Painting Effect
Radius
This parameter decides the nearest pixels to analyze. For every pixel (X,Y)
the nearest pixels from (X-Radius, Y-Radius) to ( X+Radius, Y+Radius)
are analyzed to prepare the output image.
If the Radius is high, then the processing cost will be high. The iterations inside PaintEffectImpl is as follows: Width * Height * (2 * Radius + 1 ) * (2 * Radius + 1 ).
If Radius is increased, the computation cost of the algorithm will also increase and we will get a much blocky image. Suitable values of Radius
are from 3 to 7, which will produce
a good oil painting effect.
Intensity
This parameter is used to find the intensity of a pixel. The final intensity of a pixel is calculated by the following logic:
int nActualIntensity = ( ( ( nR + nG + nB ) / 3.0 ) * IntensityParameter ) / 255;
This parameter helps to change the intensity, in-effect the output image becomes blocky.
Details of PaintEffect Application
On loading a new image, the effect will be applied with the help of the PaintEffectImpl class. PaintEffectImpl::Process is used to prepare
the oil paint effect applied image. The prototype of PaintEffectImpl::Process is as follows.
void PaintEffect::Process( const BYTE* pbyDataIn_i,
const int nRadius_i,
const float fIntensityLevels_i,
const int nWidth_i,
const int nHeight_i,
BYTE* pbyDataOut_o );
The output image (painting effect applied image) will be available in m_pbyEffectAppliedData. This image is copied to a bitmap object. This bitmap will
be blitted into the static window on each repainting of the dialog.
The following code is used to copy the effect applied image to
the CBitmap object.
HBITMAP hbmDest = ::CreateCompatibleBitmap(hdc, m_nImageWidth, m_nImageHeight);
if (hbmDest)
{
if (SetDIBits(hdc, hbmDest, 0, m_nImageHeight, m_pbyEffectAppliedData, &stBitmapInfo, DIB_RGB_COLORS))
{
m_BitmapObject.DeleteObject();
m_BitmapObject.Attach( hbmDest );
}
}
Output bitmap is displayed to screen with GDI. StretchBlit is used to display the re-sized bitmap to
the output window.
void CPaintEffectDlg::DrawWithGDI()
{
CDC* pDC = GetDlgItem( IDC_STATIC_IMAGE )->GetDC();
if (m_BitmapObject.GetSafeHandle())
{
CDC dcMem;
if (dcMem.CreateCompatibleDC(pDC))
{
CBitmap* pOldBM = dcMem.SelectObject(&m_BitmapObject);
BITMAP BM;
m_BitmapObject.GetBitmap(&BM);
RECT ClientRegion;
GetDlgItem( IDC_STATIC_IMAGE )->GetClientRect( &ClientRegion );
pDC->StretchBlt( 0, 0, ClientRegion.right, ClientRegion.bottom,
&dcMem, 0, 0, m_nImageWidth, m_nImageHeight, SRCCOPY );
dcMem.SelectObject(pOldBM);
}
GetDlgItem( IDC_STATIC_IMAGE )->ReleaseDC( pDC );
}
}
Why OpenGL Display?
Initially this application created without OpenGL support. GDI was used to display the output image into static window. But loading image in different size creates image with improper aspect ratio, and it create bad quality image. The static window[which displays the output image] in the application can display an image of dimension [400,300]. When user loads a new image with different size, a new Bitmap object is created in the actual size of the image, and algorithm is applied on the actual size image. But we have to display the new image in to a static window of dimension 400X300. Therefore CDC::StretchBlit API is used to display the Bitmap of actual size to 400X300.
pDC->StretchBlt( 0, 0, ClientRegion.right, ClientRegion.bottom,
&dcMem, 0, 0, m_nImageWidth, m_nImageHeight, SRCCOPY ); Sometimes resizing of the image creates bad quality image as seen “OpenGL Display OFF” screen shot in the below Figure. If the resizing is performed with OpenGL texture mapping, the output quality will be good. OpenGL texture is created with GL_LINEAR interpolation type for MIN(minimizing) and MAG(Maximizing) interpolation type. Therefore resizing the texture uses Bi-Linear interpolation to create a good quality image.
The
following image shows the difference between StretchBlit and OpenGL display.
Load Image
A new image can be loaded with this button. The GDI+ library is used to get the image buffer of all types of image files. After loading a new image, Oil Paint Effect
is applied to the image and redrawn to the screen.
void CPaintEffectDlg::OnButtonLoadBitmap()
{
CFileDialog* pFileOpenDlg = new CFileDialog( TRUE,L"image", NULL, OFN_HIDEREADONLY | OFN_OVERWRITEPROMPT,
L"Input Image (*.bmp;*.jpg;*.png;*.tga)|*.bmp;*.jpg;*.png;*.tga||");
if( IDOK == pFileOpenDlg->DoModal())
{
CString csFileName = pFileOpenDlg->GetPathName();
int nWidth = 0;
int nHeight = 0;
BYTE* pbyData = 0;
BMPLoader BMPLoaderObj;
if( !BMPLoaderObj.LoadBMP( csFileName.GetBuffer( 0 ), nWidth, nHeight, pbyData ))
{
AfxMessageBox( L"BMP Loading failed" );
return;
}
ProcessEffect();
Invalidate( false );
}
}
Save Image
Effect applied image can be saved to a file. The size of the input image is used as the size of output image.
The PaintEffect dialog will display the resized image
and it is not good to save the resized image. The file name is created with the
Radius and Intensity parameters.
void CPaintEffectDlg::OnBnClickedButtonSave()
{
CString csFileName;
csFileName.Format( L"OilPaintEffect_radius%d_Intensity%d.bmp",m_nRadius, m_nIntensity );
CFileDialog SaveDlg( false, L"*.bmp", csFileName );
if( IDOK == SaveDlg.DoModal())
{
CString csFileName = SaveDlg.GetPathName();
BMPLoader SaveBmp;
SaveBmp.SaveBMP( csFileName, m_nImageWidth, m_nImageHeight, m_pbyEffectAppliedData );
}
}
Resizing the Dialog
The output image is drawn to screen by resizing the output image to fit into the static window in the dialog. Therefore loading
a new image with a different
size may stretch or skew the image. If the dialog can resize, the user can view the image in the desired size.
The WM_SIZE message is handled
and the static window which displays the image is resized according to the new size of the dialog. All controls except the static window which displays images
are moved in the X direction based on the size of new window.
MoveWindowInXDirection( IDC_EDIT_RADIUS, nParamWindowMoveX );
MoveWindowInXDirection( IDC_STATIC_RADIUS, nParamWindowMoveX );
MoveWindowInXDirection( IDC_EDIT_INTENSITY, nParamWindowMoveX );
MoveWindowInXDirection( IDC_STATIC_INTENSITY, nParamWindowMoveX );
MoveWindowInXDirection( IDC_STATIC_PARAM, nParamWindowMoveX );
MoveWindowInXDirection( IDC_BUTTON_LOAD_BITMAP, nParamWindowMoveX );
MoveWindowInXDirection( IDC_BUTTON_SAVE, nParamWindowMoveX );
MoveWindowInXDirection( IDC_BUTTON_ABOUT, nParamWindowMoveX );
MoveWindowInXDirection( IDC_CHECK_PAINTING_EFFECT, nParamWindowMoveX );
MoveWindowInXDirection( IDC_STATIC_STATUS, nParamWindowMoveX );
MoveWindowInXDirection( IDC_CHECK_OPENGL_DISPLAY, nParamWindowMoveX );
Issues and Limitations
- Time for processing a big image is too high. Currently the painting effect is applied in
a GUI thread, therefore processing a big sized image may take time for displaying the image.
- If the Radius is a high value, then the processing time is too high. No parallel processing method
was tried. When I tried to prepare a shader, cg does not support array access
without a loop variable. Trying to prepare a GPU implementation of this effect.
Points of Interest
- Preparing a real example was a little complex. I prepared another application to draw the nearest pixels in large size
and prepared the images used in the Details of Algorithm section.
- I can easily prepare the output of the algorithm. But copying the output properly to screen took
a long time. Preparing a
BITMAP object and copying this image
to a memory DC. In the WM_PAINT message, this memory DC is copied to
the output window.
- Resizing of the dialog is handled in a special way. The
MoveWindowInXDirection function is created to move a control in
the X direction. On each resize,
the required movement in X direction is identified and MoveWindowInXDirection is called for all controls. MoveWindowInXDirection gets
the previous position
of the control and calculates the new position and calls SetWindowPos to apply the new position.
- On changing a parameter related to Oil Paint Effect or loading a new image, the dialog
stops responding, and the user won't get any notification.
Therefore a static button is added to indicate the algorithm processing is going on.
The "Processing Effect..." message indicates the processing of the algorithm is going on,
and we need to wait a few seconds to display the new image.
- In first version, there was some painting issues when switching from Desktop and PaintEffect application. After long time of analysis, I observed that the WM_PAINT is called, and the entire scene was drawn. But sometimes it was not proper. I used the DC of static window by calling GetDlgItem() for drawing, and it caused the painting issue. When I prepared a CPaintDC from static window, the paining issue was solved.
References
Revision History
- 08-October-2012: Initial version.
- 13-October-2012: Solved painting issue, Added slider control to change Radius and Intensity.
