Zoom An Image With Different Interpolation Types

• Download application - 2.15 MB
• Download source - 2.01 MB

Table Of Contents

Introduction

In OpenGL, we can display a texture to small or large area. Displaying a texture in the same size requires copying all texels (texture units) to output pixels. If we want to create a large image from a small texture, OpenGL has to create many intermediate pixels. This process of creating intermediate pixels is called as Interpolation.

OpenGL provides two interpolation methods to create a large texture image. GL_NEAREST and GL_LINEAR. We can provide GL_NEAREST or GL_LINEAR as GL_TEXTURE_MAG_FILTER parameter for glTexParameterf() function. GL_NEAREST is a low quality interpolation and GL_LINEAR provides Bi-Linear interpolation. In addition to OpenGL interpolation types, here different versions of Bi-Cubic interpolation are implemented in pixel shaders. Bi-Cubic interpolation can provide good quality image on creating large image from small textures.

Screenshot of ZoomInterpolation Application. Small area of flower is zoomed with different interpolations. Different versions of interpolation have quality difference. GL_NEAREST type of interpolation creates a blocky image, and CatMull-Rom or Bi-Cubic Spline creates a high quality zoomed image. Details of different interpolations are explained in the below sections.

Background

Interpolation is a "Process used to estimate an unknown value between two known values by utilizing a common mathematical relation".

ZoomInterpolation creates zoomed image with different interpolation types. OpenGL provides a Bi-Linear interpolation at its maximum. When zooming a very small texture area to large area, Bi-Linear provides a less quality.

Bi-Linear interpolation creates an intermediate pixel with the help of nearest 4 pixels. Bi-Cubic interpolation, a high quality version, creates an intermediate pixel with the help of nearest 16 pixels.

OpenGL GL_NEAREST Interpolation

GL_NEAREST just copies data available in the nearest pixel and it creates a blocky effect when a very small region is zoomed to a large screen region. In the nearest interpolation, intermediate pixels are created with the nearest valid pixel. This kind of interpolated images are blocky since same data is copied to intermediate pixels.

Below image is created through OpenGL GL_NEAREST interpolation and its quality is very bad. Edge of the flower seems to be a "stair case". We can remove this stair case through better interpolation methods.

Interpolation with GL_NEAREST interpolation type.

We can use following code to achieve nearest interpolation in OpenGL fixed function pipeline.

glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_NEAREST); 
glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_NEAREST);   

OpenGL GL_LINEAR Interpolation

OpenGL provides a good interpolation method, GL_LINEAR. It interpolates nearest 4 pixels[2 pixel in X direction and 2 Pixel in Y direction] and create a smooth image comparing to the GL_NEAREST method. GL_LINEAR method interpolates both in X and Y direction. Therefore it is called Bi-Linear interpolation. Details of Bi-Linear interpolation is explained in GLSL Bi-Linear section.

The following image is created with GL_LINEAR interpolation and its edges are smooth when comparing to the image created through GL_NEAREST interpolation. Here "stair case" effect is not visible. But we can see a shade of yellow and green through the edge of flower, and a small "stair case" is visible at some edges of the flower.

Interpolation with GL_LINEAR interpolation type.

The following code can be used to achieve Bi-Linear interpolation in OpenGL fixed function pipeline.

glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_LINEAR ); 
glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_LINEAR );    

GLSL Bi-Linear

OpenGL provides only two interpolations methods to create zoomed display of an image(texture). When zooming a very small region of a texture, we can see some blocky edges in the output image.

The first step to create a better interpolation method is to create Bi-Linear interpolation (which is the same as that of GL_LINEAR interpolation. But we prepare a pixel shader with Bi-Linear interpolation). We can improve the quality of Bi-Linear and get a better quality image.

A pixel shader is a program which will execute for each pixels of a rendering. More details of Pixel Shaders are available here or here.

Bi-Linear Shader Implementation

In Bi-Linear interpolation, the nearest four pixels are considered to create an intermediate pixel.

From the above figure, an intermediate pixel F(p’,q’) is created by interpolating nearest four texels F(p,q), F(p,q+1), F(p+1,q), F(p+1,q+1).

Texel is the term used to indicate an element in a texture (similar to a pixel in screen) [A texel, or texture element (also texture pixel) is the fundamental unit of texture space].

Initially two texels in top row and bottom row are interpolated with linear interpolation[horizontal]. Then interpolated pixels of top row and bottom row are also linearly interpolated[vertical]. Explanation with code will help to understand the above description.

An interpolation factor a, is used in X direction to interpolate F(p,q) and F(p,q+1) as follows:

F(p’’) = (1.0 – a ) * F(p,q) + a * F(p+1, q) // Linear interpolation in X direction[Top].
F(p+1’’) = (1.0 – a ) * F(p,q+1) + a * F(p+1, q+1) 	// Linear interpolation in 
						// X direction[Bottom].     

An interpolation factor b, is used in Y direction to interpolate F(p’’) and F(p+1’’) as follows:

F(p’,q’) = (1.0 - b) * F(p’’) +  b *  F(p+1’’) // Linear interpolation in Y direction.

Corresponding GLSL shader code.

// Function to get interpolated texel data from a texture with GL_NEAREST property.
// Bi-Linear interpolation is implemented in this function with the 
// help of nearest four data.
vec4 tex2DBiLinear( sampler2D textureSampler_i, vec2 texCoord_i )
{
    vec4 p0q0 = texture2D(textureSampler_i, texCoord_i);
    vec4 p1q0 = texture2D(textureSampler_i, texCoord_i + vec2(texelSizeX, 0));

    vec4 p0q1 = texture2D(textureSampler_i, texCoord_i + vec2(0, texelSizeY));
    vec4 p1q1 = texture2D(textureSampler_i, texCoord_i + vec2(texelSizeX , texelSizeY));

    float a = fract( texCoord_i.x * fWidth ); // Get Interpolation factor for X direction.
					// Fraction near to valid data.

    vec4 pInterp_q0 = mix( p0q0, p1q0, a ); // Interpolates top row in X direction.
    vec4 pInterp_q1 = mix( p0q1, p1q1, a ); // Interpolates bottom row in X direction.

    float b = fract( texCoord_i.y * fHeight );// Get Interpolation factor for Y direction.
    return mix( pInterp_q0, pInterp_q1, b ); // Interpolate in Y direction.
}

GLSL Bi-Cubic

In the previous method, the nearest four pixels are used to create an intermediate pixel. In this method, the nearest 16 pixels are used to create an intermediate pixel F(p’q’). Therefore the output image quality will increase. In the below figure, an intermediate pixel F(p'q')[Near to F(p,q)] is created by interpolating nearest 4*4 pixels from F(p-1, q-1) to F(p+2,q+2). Details of this interpolation is explained with the help of code and necessary equations.

The following equation [from Digital Image Processing: PIKS Inside, Third Edition] is used to interpolate nearest 16 pixles. First two sigma(S) are appeared as 2 for loops in the shader code.

Where F(p+m,q+n) indicates texel data at location (p+m, q+n). Rc() denotes a Bi-Cubic interpolation function such as a BSpline, Traingular, Bell cubic interpolation function. In this sample, I just use a Triangular, Bell, and B-Spline, and CatMull-Rom interpolation function.

Following is a GLSL shader code for Bi-Cubic Interpolation:

vec4 BiCubic( sampler2D textureSampler, vec2 TexCoord )
{
    float texelSizeX = 1.0 / fWidth; //size of one texel 
    float texelSizeY = 1.0 / fHeight; //size of one texel 
    vec4 nSum = vec4( 0.0, 0.0, 0.0, 0.0 );
    vec4 nDenom = vec4( 0.0, 0.0, 0.0, 0.0 );
    float a = fract( TexCoord.x * fWidth ); // get the decimal part
    float b = fract( TexCoord.y * fHeight ); // get the decimal part
    for( int m = -1; m <=2; m++ )
    {
        for( int n =-1; n<= 2; n++)
        {
			vec4 vecData = texture2D(textureSampler, 
                               TexCoord + vec2(texelSizeX * float( m ), 
					texelSizeY * float( n )));
			float f  = Triangular( float( m ) - a );
			vec4 vecCooef1 = vec4( f,f,f,f );
			float f1 = Triangular ( -( float( n ) - b ) );
			vec4 vecCoeef2 = vec4( f1, f1, f1, f1 );
            nSum = nSum + ( vecData * vecCoeef2 * vecCooef1  );
            nDenom = nDenom + (( vecCoeef2 * vecCooef1 ));
        }
    }
    return nSum / nDenom;
}

BiCubic() function gets a texture coordinate (x,y) and returns the interpolated value of nearest 16 texels.

Nearest 16 texels are iterated through 2 for loops from [x-1,y-1] to [x+2, y+2].

for( int m = -1; m <=2; m++ )
{
    for( int n =-1; n<= 2; n++)

For each nearest element, an interpolation factor is calculated with the corresponding Bi-Cubic interpolation function. In the above BiCubic() function Triangular() is used to get an interpolation factor. Different versions of Bi-Cubic interpolation[BSpline, Traingular, Bell, CatMullRom] can be created by changing Triangular() and its logic. When plotting values from Triangular(), we will get the below image in a triangle form:

From plotting of triangular function, left most value of X axis is -2 and right most value of X axis is +2. Triangular(x) is plotted in Y direction. It indicates Triangular function returns lower value for a high input and high value for a low input.

For each iteration in BiCubic(), return value of Triangular() is multiplied with current data. In effect, an intermediate pixel is created by interpolating nearest 16 data. The weightage will be high for nearest pixels, and low for far pixels. The output image of Triangular Bi-Cubic is smoother than Bi-Linear.

Implementation of Bi-Cubic Interpolation[ Triangular ]

Triangular function is a simple Bi-Cubic function, as defined in the following equation:

Here, the return value of R(x) is x+1 when x is between -1 and 0, and return value is x+1 when x is between 0 and 1. The following pseudo code gives more idea of triangular function. This function provides a low value for high input and a high value for a low input.

if( -1 < x && x <= 0 )
{ 
    return x + 1
}
else if( 0 < x && x <= 1 )
{
    return x - 1
}   

The above diagram is the output of Triangular() function. Triangular(x) is plotted in Y direction.

The following code is used in GLSL shader for Triangular Bi-Cubic implementation.

 float Triangular( float f )
{
	f = f / 2.0;
	if( f < 0.0 )
	{
		return ( f + 1.0 );
	}
	else
	{
		return ( 1.0 - f );
	}
	return 0.0;
}

Interpolation with GLSL Bi-Cubic Triangular interpolation type.

Implementation of Bi-Cubic Interpolation[ Bell ]

When comparing Bi-Cubic Triangular image with Bi-Linear image, the quality of image is increased. But Triangular creates a blurred effect and all edges are smoothen. The reason of this blurred edge is weight returned from Triangular() for near and far texel. If weight of near texel is high and far texel is very low, then Bi-Cubic interpolation can achieve smooth edges. Another Bi-Cubic interpolation function which creates a bell shaped (nearest values[Center] are high) and far values are very low[left and right ends].

The above equation has three conditions to create the following curve. The corresponding code is implemented in BellFunc(). If we plot the input values from -2 to +2 of BellFunc() in a graph, we will get the following curve. Where X direction contains x provided to BellFunc and Y direction is corresponding return value[BellFunc(x)].

The above diagram indicates when x is high, BellFunc(x) is very low, therefore far data of an intermediate texel gets very low weight. For nearest texel BellFunc(x) is high and gets a high weight.

The following code is used in GLSL shader for Bell shaped Bi-Cubic implementation.

float BellFunc( float x )
{
	float f = ( x / 2.0 ) * 1.5; // Converting -2 to +2 to -1.5 to +1.5
	if( f > -1.5 && f < -0.5 )
	{
		return( 0.5 * pow(f + 1.5, 2.0));
	}
	else if( f > -0.5 && f < 0.5 )
	{
		return 3.0 / 4.0 - ( f * f );
	}
	else if( ( f > 0.5 && f < 1.5 ) )
	{
		return( 0.5 * pow(f - 1.5, 2.0));
	}
	return 0.0;
}

Interpolation with Bi-Cubic Bell interpolation type.

Implementation of Bi-Cubic Interpolation[ B-Spline ]

When comparing Bi-Cubic Bell shaped with B spline, interpolated image is smooth and edges are more clear. The below equation is used to create BSpline() function.

The code of this equation is implemented in BSpline() function.

The above diagram is the output of BSpline() function. BSpline(x) is plotted in Y direction. x starts from -2.0 and ends at +2.0. It indicates when x is high, BSpline(x) is very low and therefore far data of an intermediate texel gets very lower weight. When comparing to Bell shaped wave, values in far range (near -2 and +2) are very low. Therefore, the final output image is also smoother than Bell Bi-Cubic interpolated image. The following code is used in GLSL shader for BSpline implementation.

float BSpline( float x )
{
	float f = x;
	if( f < 0.0 )
	{
		f = -f;
	}

	if( f >= 0.0 && f <= 1.0 )
	{
		return ( 2.0 / 3.0 ) + ( 0.5 ) * ( f* f * f ) - (f*f);
	}
	else if( f > 1.0 && f <= 2.0 )
	{
		return 1.0 / 6.0 * pow( ( 2.0 - f  ), 3.0 );
	}
	return 1.0;
}  

Interpolation with Bi-Cubic BSpline interpolation type.

Implementation of Bi-Cubic Interpolation[CatMull-Rom]

float CatMullRom( float x )
{
    const float B = 0.0;
    const float C = 0.5;
    float f = x;
    if( f < 0.0 )
    {
        f = -f;
    }
    if( f < 1.0 )
    {
        return ( ( 12 - 9 * B - 6 * C ) * ( f * f * f ) +
            ( -18 + 12 * B + 6 *C ) * ( f * f ) +
            ( 6 - 2 * B ) ) / 6.0;
    }
    else if( f >= 1.0 && f < 2.0 )
    {
        return ( ( -B - 6 * C ) * ( f * f * f )
            + ( 6 * B + 30 * C ) * ( f *f ) +
            ( - ( 12 * B ) - 48 * C  ) * f +
            8 * B + 24 * C)/ 6.0;
    }
    else
    {
        return 0.0;
    }
} 

BMPLoader BMPLoadObj; // To load an RGB of a bmp file 
BYTE* pbyData = 0;
BMPLoadObj.LoadBMP( IDB_BITMAP_FLOWER, m_nImageWidth, m_nImageHeight, pbyData );
m_glTexture.Create( m_nImageWidth, m_nImageHeight, pbyData );// Texture creating with bmp  

/*
 This function draws miniature of actual image with a Red region
 indicating the zoomed area.
 */
void CZoomInterpolationDlg::DrawActualImage()
{
    // Set Rendering area of Actual image.
    glViewport( 805, 10, 200, 150 );

    // Image is attached.
    m_glTexture.Enable();

    // Entire image is mapped to screen.
    m_glVertexBuffer.DrawVertexBuffer( GL_QUADS );
    m_glTexture.Disable();

    // Set Red color for Zoom area indication.
    glColor3f( 1.0, 0.0, 0.0 );

    float fXStart = m_fXOffset * 2;
    float fYStart = m_fYOffset * 2;
    float fWidth = m_fZoomWidth * 2;
    float fHeight = m_fZoomHeight * 2;

    // Draw a rectangle indicate zoom area.
    glBegin( GL_LINE_LOOP );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart );
    glVertex2d( -1.0 + fXStart + fWidth,  -1.0 + fYStart );
    glVertex2d( -1.0 + fXStart + fWidth,  -1.0 + fYStart + fHeight );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart + fHeight );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart );
    glColor3f( 1.0, 1.0, 1.0 );
    glEnd();
}

/*
 Creating zoomed image.
 */
void CZoomInterpolationDlg::DrawZoomedImage()
{
    // Displays all views. Draw()_ method of GLImageView prepares the 
    // Zoomed image of a view. m_ImageView list holds single object in normal case.
    // When All In View is selected in Interpolation type, m_ImageView will hold 7 View
    // objects.
    for( int nViewIndex = 0; nViewIndex < m_ImageView.size(); nViewIndex++ )
    {
        (m_ImageView[nViewIndex])->Draw();
    }
} 

void CZoomInterpolationDlg::OnBnClickedButtonSave()
{
    CString csFileName;
    csFileName.Format( L"ZoomInterpolation.bmp" );
    CFileDialog SaveDlg( false, L"*.bmp", csFileName );
    if( IDOK == SaveDlg.DoModal())
    {
        RECT stImageArea;
        stImageArea.left =0;
        stImageArea.top = 0;
        stImageArea.right = 800;
        stImageArea.bottom = 600;
        CString csFileName = SaveDlg.GetPathName();
        BYTE* pbyData = new BYTE[stImageArea.bottom * stImageArea.right  * 3];
        if( 0 == pbyData )
        {
            AfxMessageBox( L"Memory Allocation failed" );
            return;
        }
        glReadPixels( 0, 0, stImageArea.right, stImageArea.bottom, 
			GL_BGR_EXT, GL_UNSIGNED_BYTE, pbyData );
        BMPLoader SaveBmp;
        SaveBmp.SaveBMP( csFileName, stImageArea.right, stImageArea.bottom, pbyData );
        delete[] pbyData;
    }
} 

 void CZoomInterpolationDlg::PrepareAllInOneView()
{
    ...............

        int nViewPortXDiv = 800 / 3;
    int nViewPortYDiv = 600 / 3;
    int nResourceId = IDR_GLSL_SHADER_BILINEAR;

    for( int nI = 0; nI < 7; nI++ )
    {
        int nX = nI % 3;
        int nY = nI / 3;

        GLText* pText = new GLText( &m_FontEngine );
        pText->SetText( INTERPOLATION_NAMES[nI] );

        GLImageView* pImage = new GLImageView();
        pImage->SetViewport( nX * nViewPortXDiv, nY * nViewPortYDiv, 
				nViewPortXDiv, nViewPortYDiv );
        pImage->SetText( pText );
        GLenum eTextureFilter;
        if( nI >= 2 )
        {
            GLSLShader* pTempShader = new GLSLShader();

            pTempShader->CreateProgram( nResourceId++, GL_FRAGMENT_PROGRAM_ARB );
            pImage->SetShader( pTempShader, true );
            eTextureFilter = GL_NEAREST;
        }
        else
        {
            pImage->SetShader( 0 ); // Shader is not required for openGL interpolation
            if( 0 == nI )
            {
                eTextureFilter = GL_NEAREST; // Special Handling for OpenGL Interpolation
            }
            else
            {
                eTextureFilter = GL_LINEAR;  // Special Handling for OpenGL Interpolation
            }
        }
        pImage->SetTextureParam( &m_glTexture, m_nImageWidth, 
				m_nImageHeight, eTextureFilter );
        pImage->SetVertexBuffer( &m_glVertexBufferZoom );
        m_ImageView.push_back( pImage );
    }
}