Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / Languages / C++/CLI

Handling non-English characters UTF8 with MS VC++ 2005

1.22/5 (7 votes)
6 Nov 2008LGPL31 min read 19K  
This article describes how to handle streams of non-English characters.

Introduction

Many times, as a beginner in programming with MS Visual Studio and as a Spanish speaker, I have had problems dealing with non-English characters. This simple and short article describes how to read/write text files using the UTF8 characters map and some problems I had with Visual Studio 2005.

Background

During my university life, all the programming that I learned was orientated toward create background code, not end user programs. So I was familiar with common problems associated with the GCC compiler, and everything worked fine. That was until I had to make a end user app. This is a not very big wall, but a one for many. I want to share how I sorted it.

Using the Code

I first tried to use the basic fstream class, but in VS, the kind of char that is used is the 7-bit char, and when trying to use it with wchar_t, no overloading function errors were shown. So, since that, I decided that I had no more choice than using the famous CLR, not knowing that this was just the beginning of my problems.

I started with the FileStream class, with no great results. The string that is used is a 7-bit (again). Then, finally, I found the StreamRead and StreamWrite classes. Shown here is some simple code to read and write a file using this character map:

C++
#include "stdafx.h"
#using <mscorlib.dll>

using namespace System;
using namespace System::IO;
using namespace System::Text;

int main(array<System::String ^> ^args)
{
    String^ filein;
    String^ fileout;
    Console::WriteLine(L"Enter file name: ");
    filein = Console::ReadLine();
    fileout = "out"+filein;

    StreamReader ^srp = gcnew StreamReader(filein, 
                        System::Text::Encoding::GetEncoding(1252));
    StreamWriter ^swp = gcnew StreamWriter(fileout, false, 
                        System::Text::Encoding::GetEncoding(1252));
    
    swp->WriteLine(L"Some simple header text with á é ä Ñe");
        
    String ^cont ;
    while (cont = srp->ReadLine())
    {
        swp->WriteLine(cont); 

       }
    swp->Close();
    srp->Close();
    delete(IDisposable^) swp;
    delete(IDisposable^) srp;

    return 0;
}

Points of Interest

An interesting point is that the text files generated in this program were well read on the VS file editor. I checked this with the context editor. I think may be this is because of the default language setting of VS.

License

This article, along with any associated source code and files, is licensed under The GNU Lesser General Public License (LGPLv3)