Abstract of the text rendering with OpenGL/OpenTK in MONO/.NET

Download TextPrinter-Test.zip complete MonoDevelop solution to demonstrate TextPrinter
Download QuickFont-Test.zip complete MonoDevelop solution to demonstrate QuickFont
 Download TextureLib-Test.zip complete MonoDevelop solution to demonstrate TexLib
 Download HandCrafted.zip complete MonoDevelop solution to demonstrate TextRenderer Ver. 1
Download TextRenderer-Test.zip complete MonoDevelop solution to demonstrate TextRenderer Ver. 2
 Download FreeType-Test.zip complete MonoDevelop solution to demonstrate FtFont Ver. 1 and Ver. 2
 Download FreeTypeDynamic-Test.zip complete MonoDevelop solution to demonstrate FtFont Ver. 3

Introduction

This article shall help to get a quick overview about the text rendering options for OpenGL/OpenTK, espeially for the MONO/.NET programming languages. I want to share my findings and help programmers, which are looking for a solution, that fits their needs.

Background

There are two basic approaches:

1. CONVENTIONAL: Render the text, using the glyphs of a font, to a bitmap utilizing CPU, transfer the bitmap as an texture to the GPU and blend it into the scene.
2. INNOVATIVE: Render the glyphs of a font to a fast calculatable outline (arc and line segments instead of spline and polynominal segments) at GPU, render a texture based on a pixel-to-outline-distance calculation at GPU and blend it to the scene.

Let's start with a short discussion about the INNOVATIVE approach: This method requires a powerful GPU and a considerable amout of texture buffer memory on the one hand but it relieves the CPU remarkable on the other hand. Currently there is exactly one known implementation: GLyphy. IMO this will definitively be the approach of the future. Future because GLyphy detected various implementation errors in Mesa, almost all video drivers and pixel shaders it has been tested for. And they have to be fixed before it can be used widely.

Now let's go back to the CONVENTIONAL approach: There are thee practices:

1.a.	Render the text-to-display to a (final texture) bitmap. Go on with the bitmap as a texture to blend into the GL window's scene.
1.b.	Render the glyphs of a font to an (intermediate texture) bitmap, pick the glyphs as an excerpt of this bitmap and compose them to a new (final texture) bitmap representing the text-to-display. Go on with the bitmap as a texture to blend into the GL window's scene.
1.c.	Load a bitmap font into an (intermediate texture) bitmap, pick the glyphs as an excerpt of this bitmap and compose them to a new (final texture) bitmap representing the text-to-display. Go on with the bitmap as a texture to blend into the GL window's scene.

The practice 1.a. (text to final texture bitmap) requires DrawString() and MeasureString() methods of the graphics context, used to render the text-to-display.

Pros: All that's needed is provided by the Windows GDI or the X11 font server. Any system font can be used.

Cons: The quality depends on the providerd DrawString() and MeasureString() method implementation.

Description: On Windows the System.Drawing.Graphics class implementation produces excellent output quality. On X11 the Mono implementation of the System.Drawing.Graphics class produces output quality in a wide range:

• Any text color on a uniform background produces excellent results.
• Any black / gray / white text on gradient background produces good results.
• Colored text, especially with multiple text colors, needs a lot of adjustment do produce acceptable results.

On X11 the Mono wrapper for Pango or Cairo's Pango calls could be a good alternative to Mono's System.Drawing.Graphics namespace (Windows GDI replica). Cairo offers Cairo.Context.ShowText() and Cairo.Context.TextExtents() as an equivalent to System.Drawing.Graphics.DrawString() and System.Drawing.Graphics.MeasureString(). But i didn't find code "ready to use" that implements the required functionality for application in the context of OpenTK.

The practice 1.b. (glyphs to intermediate texture bitmap, excerpts to final texture bitmap) is a little bit "reinventing the wheel". Because to render the glyphs of a font to an (intermediate) bitmap is the same thing that Windows GDI or X11 font server already do.

Pros: Absolute control over the the whole text rendering chain (glyphs, texture, blending). Any quality and anny effect can be achieved. Any system font can be used.

Cons: A lot of effort for creation and management of the (intermediade) font bitmaps. As well as for extraction of glyph texture's excerpts and combination to a string. Program initialization requires font bitmap initialization and consumes runtime.

Description: This practice requires DrawString() and MeasureString() method as well, but the produced font bitmaps can be post-processes to achieve a specific quality or effect. Drawbacks of the MONO implementation of the System.Drawing.Graphics class can be counterbalanced.

On X11 a FreeType based text drawing should be used instead of Mono's System.Drawing.Graphics namespace (Windows GDI replica).

The practice 1.c. (bitmap-font to intermediate texture bitmap, excerpts to final texture bitmap) is similar to practice 1.b, but it doesn't provide the same fonts as the Windows GDI or X11 font server already do - it provides fonts from specific bitmap font files. This practice is typically used by games.

Pros: Absolute control over the whole text rendering chain (glyphs, texture, blending). Any quality and anny effect can be achieved.

Cons: A lot of effort for creation and management of the font bitmaps. As well as for extraction of glyph texture's excerpts and combination to a string. Specific font files are required. Program initialization requires font bitmap initialization but is much faster than 1.b.

Description: The creation of font bitmaps can be completely separated (by time, by resources, by location) from their usage. Artificial or texture fonts are easy to achieve. Most of the fonts are monospaced, but proportional fonts are possible - they need the glyph widths in addition to the bitmap provided by the font file.

Using the code

I prepared four sample solutions, that cover the practices 1.a., 1.b. and 1.c., all with MonoDevelop 5.0.1 on Mono 3.8.0 and .Net 4.0. The OpenTK library is assembly version 1.1.0.0, the Mesa library is 10.3.7:

• Practice 1.a. (text to final texture bitmap) TextPrinter-Test (utilizing OpenTK.Graphics.TextPrinter)
• Practice 1.b. (glyphs to intermediate texture bitmap, excerpts to final texture bitmap) QuickFont-Test (utilizing QuickFont class for OpenTK)
• Practice 1.c. (bitmap-font to intermediate texture bitmap, excerpts to final texture bitmap) TextureLib-Test (utilizing TextureLib's TextureFont class for OpenTK)
• Quality comparison HandCrafted-Test (practice 1.a. and practice 1.b.)

Update with article version 2.0

I've updated MonoDevelop 5.0.1 to MonoDevelop 5.10 to overcome the frequent debugger crashes.

I've found the 'missing piece' to use FreeType instead of Mono's GDI implementation (System.Drawing.Graphics class) within the articles Rendering FreeType/2 with OpenGL and Rendering AltNETType (= .NET FreeType port) with OpenGL. They led me to the FtFont class that joins ideas from OpenGL 3.3+ text..., freetype-gl and FreeType Fonts.

I added three further sample applications, that cover the practices 1.a and 1.b:

• Practice 1.a. (text to final texture bitmap) TextRenderer-Test (utilizing System.Drawing.Graphics class)
• Practice 1.b. (glyphs to intermediate texture bitmap, excerpts to final texture bitmap) FreeTypeGlyphWise-Test (utilizing FreeType font class FtFont drawing a bitmap and mapping the bitmap as a texture glyph by glyph)
• Practice 1.b. (glyphs to intermediate texture bitmap, excerpts to final texture bitmap) FreeTypeLineWise-Test (utilizing FreeType font class FtFont drawing a bitmap and mapping the bitmap as a texture for a complete string)

Update with article version 5.0

I've found the 'missing piece' to fix the 'unicode charachter' problems like '¬' instead of '€' within the SFML code. This library adds glyphs dynamically to the intermediate texture bitmap. I can recommend to read SFML-2.5.0\src\SFML\Graphics\Font.cpp, SFML-2.5.0\src\SFML\Graphics\Texture.cpp and SFML-2.5.0\src\SFML\Graphics\Text.cpp. For this solution i had to implement

1. a texture copy algorithm that preserves the existing glyph bitmaps in the case the intermediate texture bitmap is to enlarge for a new glyph, which is required to append dynamically (see tip Copy a GL Texture to Another GL Texture or to a GL Pixel Buffer and from a GL Pixel Buffer and sample application's FtTexture class),
2. a new 'character code to glyph' mapping, that provides dynamic glyph appending (see sample application's FtFontPage and FtGlyphTable classes) and
3. a GL command pipeline, that divides string drawing into the two pices

• preparation of GL commands
• execution of GL commands (see sample application's GlCommandSequence class).

because dynamically glyph appending doesn't work during string drawing, if the intermediate texture is to enlarge in the case a new glyph bitmap to add. Moreover, the command pipeline offers the advantage to buffer the GL commands and avoid recalculation of the glyph texture bitmap excerpts in future.

The new sample application FreeTypeDynamic-Test implements all these techniques. This sample application is based on FreeTypeLineWise-Test. The new sample application

• solves the 'unicode charachter' problems like '¬' instead of '€',
• implements kerning (optionally, because kerning cuts down the speed to the half approx.) and
• produces the same excellent quality. No off-color pixel, no residues.

The practice 1.a. sample programs - TextPrinter and TextRenderer

The TextPrinter-Test sample is based on the obsolete TextPrinter from the OpenTK.Compatibility.dll. Obsolete doesn't mean the code is completely outdated. Instead the code currently lacks of a maintainer to keep it aligned with the OpenTK development progress. The output quality using TextPrinter is excellent.

The next image shows a sample output with TextPrinter and seven coloured different fonts. The quality is excellent. No off-color pixel, no residues.

Instead using TextPrinter, the whole OpenTK community recommends to write an own text printer. But TextPrinter is still present, produces very good output quality and is open source (can be copied and used, even if it would be removed from OpenTK.Compatibility.dll in remote future).

The sample program HandCrafted-Test will take a closer look to the aspect 'write an own text printer'.

Update with article version 2.0

The TextRenderer-Test sample is based on the TextRenderer technology of the HandCrafted-Test sample, but produces the same output as the TextPrinter-Test sample. The output quality is comparable to the obsolete TextPrinter class.

The next image shows a sample output with TextRenderer and seven coloured different fonts. The quality is excellent. No off-color pixel, no residues.

To use TextRenderer as an alternative to the obsolete TextPrinter, a lot of effort is required to speed up the text rendering.

The sample programs FreeTypeGlyphWise-Test and FreeTypeLineWise-Test will also take a closer look to the aspect 'write an own text printer' and are faster than the TextRenderer technology.

The practice 1.b. sample programs - QuickFont, FreeTypeGlyphWise and ~LineWise

The QuickFont-Test sample is based on the OpenTK QuickFont code. The output quality using QuickFont can reach from poor to very good - depending on the font. Some fonts produce residues, it seems QuickFont just uses too little space between the glyphs.

The next image shows a sample output with QuickFont and seven coloured different fonts. The quality is varying. No off-color pixel, but residues with DroidSerif-Bold, DejaVu Serif and luximr.

Update with article version 2.0

During the work on the sample solution FreeTypeLineWise-Test i've been faced with residues as well and solved it with two extra scan-lines within the intermediate bitmap, one obove and one below the glyphs. I think the extraction of glyph texture's excerpt from the intermediate bitmap, that is done with glTexCoord2() on a coordinate range 0.0 ... 1.0, has insufficient precisition and causes the problems. Probably extra scan-lines within the intermediate bitmap would solve the problem for QuickFont too.

The FreeTypeGlyphWise-Test sample is based on the Rendering FreeType/2 with OpenGL article's code. The output quality using FtFont class' first version is excellent.

The next image shows a sample output with FtFont class' first version and seven coloured different fonts. The quality is excellent. No off-color pixel, no residues.

The FreeTypeLineWise-Test sample advances the FtFont class to support string rendering instead character rendering (that is mapping the text glyph texture by glyph texture to the viewport). The output quality using FtFont class' second version is excellent as well.

The next image shows a sample output with FtFont class' second version and seven coloured different fonts. The quality is excellent. No off-color pixel, no residues.

Update with article version 5.0

The FreeTypeDynamic-Test sample advances the FreeTypeLineWise-Test sample and FtFont class to support dynamic glyph appending to the glyph texture. The output quality using FtFont class' third version is excellent as well.

The next image shows a sample output with FtFont class' third version and seven coloured different fonts. The quality is excellent. No off-color pixel, no residues. The glypht positioning has been reworked, now the text output complies with the font metrics.

As you can see, the 'unicode charachter' problems like '¬' instead of '€' is solved.

Kerning is switched off (because kerning cuts down the speed to the half approx.).

Shrink is switched on (which reduces the character spacing by 1/12 character advance).

Kerning and shrink are new parameters of the

public Size DrawString (string text, uint characterSizeInPPEm, bool bold, int startX, int startY,
                        bool applyKerning = false, bool shrink = false)

method.

The practice 1.c. sample program - TextureLib

The TextureLib-Test sample is based in the OpenTK TexLib code. The output quality using TexLib can be very good - depending on the quality of the font bitmap file.

The next image shows a sample output with TexLib and seven different fonts. The quality is varying. The delivered bitmap font big-outline has cut-off ascenders and descenders, the other font bitmaps are created quick and dirty and show only black text. The acquisition of ready-to-use high quality font bitmap files seems to be a problem. The limitation of TexLib to 16 x 16 glyps is a restriction.

Quality comparison sample program - HandCrafted

The HandCrafted-Test sample compares the output quality of the TextRenderer class, the obsolete TextPrinter class and the QuickFont class best output quality.

The next image shows a sample output with TextRenderer (first red text line) compared to TextPrinter (second red line) and QuickFont (third red line). The quality is excellent. No off-color pixel, no residues.

The next image shows a detail with 600% zoom to compare the new TextRenderer class output (upper red string) against the obsolete TextPrinter class output (lower red string).

By the way: The green strings are TextRenderer class output as well.

Conclusion

One option could be to advance the TextRenderer class to be convenient and fast and well documented in the future, because it has much lesser code and produces the same quality as TextPrinter. Nevertheless TextPrinter is a good choice too.

Another option could be to develop a FreeType font class to render without the drawbacks of the Mono implementation of the System.Drawing.Graphics namespace (Windows GDI replica).

Update with article version 2.0

The FreeType FtFont implementation has been successfully and much faster than the TextRenderer implementation. Now i would favor to go on with the FtFont approach and fix 'unicode charachter' problems like '¬' instead of '€'.

Update with article version 4.0

There are alternatives to the FtFont class

• the AltSketch port of the font rendering library Freetype from AltNETType and
• the SFML Font class.

Update with article version 5.0

I've fixed the 'unicode charachter' problems like '¬' instead of '€'. And i've added kerning (optionally, because kerning cuts down the speed to the half approx.). I would definitively recommend to go on with the FreeType FtFont implementation.

Performance issues

Update with article version 2.0^[1] and 5.0^[2]

This are performance figures, measured with a VMware® Player 7.1.2 build-2780323 virtual machine and two cores of a i7-5600U CPU.

sample	practice		qality 1 2 3	performance low CPU load	performance high CPU load
TextPrinter-Test	1.a.	text to final texture bitmap	A A A	190 fps	45 fps
Quick-Font-Test	1.b.	glyphs to intermediate texture bitmap, excerpts to final texture bitmap	A B A	200 fps	50 fps
TextureLib-Test	1.c.	bitmap-font to intermediate texture bitmap, excerpts to final texture bitmap	A A C	380 fps	185 fps
TextRenderer-Test[1]	1.a.	text to final texture bitmap	A A A	40 fps	5 fps
FreeTypeGlyphWise-Test[1]	1.b.	glyphs to intermediate texture bitmap, excerpts to final texture bitmap	A A B	45 fps	9 fps
FreeTypeLineWise-Test[1]	1.b.	glyphs to intermediate texture bitmap, excerpts to final texture bitmap	A A B	400 fps	175 fps
FreeTypeDynamic-Test[2]	1.b.	glyphs to intermediate texture bitmap, excerpts to final texture bitmap	A A A	320 fps	150 fps

The quality levels are

• 1 off-color pixel (on multi-colored background): A = none
• 2 residues (from overlapping texture bitmap): A = none, B = residues on some fonts, C = always residues
• 3 miscellaneous: A = no wrong glyphs, B = some unicode charachter wrong glyphs, C = some unicode charachter wrong glyphs and some cut-off ascenders/descenders

Points of Interest

The HandCrafted-Test sample

It has been tricky to makeTextRenderer class work properly - these are the findings, that led me to the final solution:

• Use the System.Drawing.Imaging.PixelFormat.Format32bppArgb pixel format for the font bitmap.
  • Mind for all bitmap's bit manipulations, that the bitmap bits are stored in BGRA order.
• Use Color.FromArgb (0, 0, 0, 0), not Color.Black (that is 255, 0, 0, 0), for the font bitmap background.
  • Color calcualtions are easy, if RGB component values of foreground colors can be used directly, because they already represent the margin from the background color to the foreground color.
  • Black doesn't falsify the foreground colors on alpha blending, neither absolute (luminance) nor relative (hue).
• Use a custom Clear() method to fill the font bitmap's background.
  • The System.Drawing.Graphics method Clear() is not suitable because it doesn't set the color, if alpha byte is 0.
  • The System.Drawing.Graphics method FillRectangle() is not suitable because it does alpha blending with the existing color.
• Design a PostprocessForeground() method to adjust color and alpha value of font bitmap's pixel to prevent background shading during the blending process.
  • For every RGB color component: Apply the entire target color RGB component to prevent color falsification and the respectively other RGB color component margins (from the background RGB color component to the foreground RGB color component) proportional to brighten up.
  • For alpha byte: Apply the highest of the RGB color component margins.

// Calculate the margin from the background RGB color components
// to the foreground RGB color components.
int deltaB = Math.Abs(bitmapData[index    ] - targetColor.B);
int deltaG = Math.Abs(bitmapData[index + 1] - targetColor.G);
int deltaR = Math.Abs(bitmapData[index + 2] - targetColor.R);
// Determine the highest RGB color component margin.
int deltaM = Math.Max(deltaB, Math.Max (deltaG, deltaR));

// Apply the entire target color RGB component to prevent color falsification
// and the respectively other RGB color component margins proportional to brighten up.
bitmapData[index    ] = (byte)Math.Min(255, targetColor.B + deltaR / 3 + deltaG / 3);
bitmapData[index + 1] = (byte)Math.Min(255, targetColor.G + deltaR / 3 + deltaB / 3);
bitmapData[index + 2] = (byte)Math.Min(255, targetColor.R + deltaG / 3 + deltaB / 3);
// Now we have exactly the target color or the target color  proportional to
// brighten up and can apply the highest RGB color component margin to the alpha byte.
bitmapData[index + 3] = (byte)Math.Min(255, 255 - deltaM);

• Blend the font bitmap and the scene based on alpha values.
  • Set GL.Color4 (1f, 1f, 1f, 1f);
  • Set GL.Enable (EnableCap.Blend );
  • Set GL.BlendFunc (BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha);

Update with article version 2.0 / the FreeTypeLineWise-Test sample

The initial steps to provide a FreeType alternative to System.Drawing.Graphics class have been done successfully. Things, that are currently missing, are:

• A fix for the 'some unicode charachter wrong glyphs' problems like '¬' instead of '€'
• A fast MeasureString() method.
• Scaling, rotating, kerning.
• High sophisticated convenience methods, e. g. to calculate auto line breaks, ...

Update with article version 3.0 / texts rasterization exposures

There is a very interesting article Texts Rasterization Exposures about text rendering detalis written for the Anti-Grain Geometry (AGG) library. The text rendering detalis that are discussed, are namely:

• hinting (snap interpolation points of the glyph outline to the pixel grid for a sharp stroke appearence, but accept non-uniform stroke weight - especially on low screen resolutions)
• sub-pixel positioning (create different pixel set on different positions for the same glyph with the same size and weight to imporove the glyph position accuracy - this sacrifies the aspect of a sharp stroke appearence for the aspect of harmonic spaces between glyphs)
• sub-pixel rendering (use the RGB sub-pixel of one pixel on LCD display to imporove the glyph contour accuracy in x-direction to 1/3 pixel size - this works for all types of displays, that order RGB sub-pixel side by side, and sacrifies the aspect of chrominance/color fidelity for the aspect of luminance/brightness fidelity)
• kerning (decrease/increase the default space between two determined glyphs to an individual space, if one ore both glyph(s) leave/occupy space that can/can't be used to achieve a visually pleasing result)
• gamma correction (prevent the circular and oblique strokes to look heavier than the horizontal/vertical strokes)

Even if the article is pretty old (July 2007) and ClearType is doing a better job since its improvement with DirectWrite (Windows 7), it shows the complexity of text rendering.

The subsequent image shows gray-scale anti-aliased text with color fidelity (on the left) and sub-pixel positioned text with brightness fidelity (on the right), zommed to 400%. Both texts have been rendered with WPF (.NET 4.5) and have different pixel set on different positions.

While gray-scale anti-aliased text might show saw-tooth edges, especially in case of high contrast and small letters.

Sub-pixel positioned text on the other hand might look distorted on a colored background, especially in case of gradient or changing background color.

History

This is the first version from 21. November 2015.
The second version is from 13. January 2016 (improved TextRenderer and new FreeType samples FreeTypeGlyphWise and FreeTypeLineWise; text improvements).
The third version is from 04. July 2016 (text fixes and improvements; additional links).
The fourth version is from 03. October 2018 (text fixes and improvements; additional links).
The fifth version is from 27. October 2018 (new FreeTypeDanamic and FreeType sample; text improvements).