What’s Wrong with Persian Culture in .NET?

• Download sample code

This post was republished to Babak Mahmoudi’s Blog at 11:38:10 ق.ظ 08/22/2011

In this post, some mistakes in implementation of Persian culture in .NET are discussed and also get-around methods are proposed.

.NET provides enhanced globalization features mostly based on its implementation of Culture concepts. Programmers may use various aspects of these features to develop software ready for global market. A class called CultureInfo plays a key role in this implementation. It is mainly used to get necessary information about a specific culture. Programmers will create instances of CultureInfo to access required information about a culture. For sure, the framework supports the Persian language too. One may use ”fa-IR” to create a CultureInfo instance for Persian language in Iran. But at it is discussed here, there are a number of problems with this culture instance.

The most critical deficiency of Persian culture is about Persian calendar. While Iranian people use their own calendar, Persian culture assumes they use Arabic Hijri calendar. The following picture shows how CultureInfo assumes HjriCalendar for Persian culture. Also note that PersianCalendar is not even included in OptionalCalendars.

Another problem with Persian culture is about calendar information such as day and month names. They all are Arabic ones:

So in order to have a better Persian CultureInfo, one should:

• Find a way to set PersianCalendar for the culture calendar
• Correct Months and Day names

Correcting Months and Day Names

Months and day names are actually included in DateTimeFormatInfo class property of CultureInfo. They can be easily fixed with code such as:

Culture.DateTimeFormatInfo.MonthNames = new string[] 
{ "فروردین", "ارديبهشت", "خرداد", "تير"
, "مرداد", "شهریور", "مهر", "آبان", "آذر", "دی", "بهمن", "اسفند", "" };

Using Persian Calendar

Using Persian Calendar is not as straightforward as setting months names. Both CultureInfo and DateTimeFormatInfo include a calendar property. To get proper Persian date formatting, one should set these calendars to Persian. One may assume to simply set the Calendar property:

Culture.DateTimeFormatInfo.Calendar = new PersianCalendar();

But the property set method of DateTimeFormatInfo prevents such settings because Persian Calendar is not included in OptionalCalendars of the Persian culture. One may use Reflection to by-pass the property set method to directly access the calendar property:

FieldInfo dateTimeFormatInfoCalendar = typeof(DateTimeFormatInfo).GetField("calendar", 
    BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
dateTimeFormatInfoCalendar.SetValue(info, new PersianCalendar());

Where info is a DateTimeFormatInfo. Note how reflection helps in setting a private field “calendar” in a DateTimeFormatInfo object. This bypasses the set method logic of checking the OptionalCalendars.

Putting it altogether, a candidate method for fixing the DateTimeFormatInfo can be:

public static void FixPersianDateTimeFormat(DateTimeFormatInfo info,bool UsePersianCalendar)
{
    FieldInfo dateTimeFormatInfoReadOnly = typeof(DateTimeFormatInfo).GetField
      ("m_isReadOnly", BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);

    FieldInfo dateTimeFormatInfoCalendar = typeof(DateTimeFormatInfo).GetField
     ("calendar", BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance); ;

    if (info == null)
        return;

    bool readOnly = (bool)dateTimeFormatInfoReadOnly.GetValue(info);

    if (readOnly)
    {
        dateTimeFormatInfoReadOnly.SetValue(info, false);
    }

    if (UsePersianCalendar)
    {
        dateTimeFormatInfoCalendar.SetValue(info, new PersianCalendar());
    }

    info.AbbreviatedDayNames = new string[] { "ی", "I", "س", "چ", "پ", "ج", "O" };

    info.ShortestDayNames = new string[] { "ی", "I", "س", "چ", "پ", "ج", "O" };

    info.DayNames = new string[] 
         { "یکOنEه", "IوOنEه", "ﺳﻪOنEه", "چهCرOنEه", "پنجOنEه", "جمعه", "OنEه" };

    info.AbbreviatedMonthNames = new string[] 
         { "فرورIین", "CرIيEهOE", "IرICI", "Eير", "مرICI", "Oهریور", "مهر", "AECن", "Aذر", 
           "Iی", "Eهمن", "CسفنI", "" };

    info.MonthNames = new string[] { "فرورIین", "CرIيEهOE", "IرICI", "Eير", "مرICI", "Oهریور"
         , "مهر", "AECن", "Aذر", "Iی", "Eهمن", "CسفنI", "" };

    info.AMDesignator = "ق.U";

    info.PMDesignator = "E.U";

    info.FirstDayOfWeek = DayOfWeek.Saturday;

    info.FullDateTimePattern = "yyyy MMMM dddd, dd HH:mm:ss";

    info.LongDatePattern = "yyyy MMMM dddd, dd";

    info.ShortDatePattern = "yyyy/MM/dd";

    if (readOnly)
    {
        dateTimeFormatInfoReadOnly.SetValue(info, true);
    }
}

This will fix the DateFormatInfo for Persian Calendar and also months and day names.

Fixing Optional Calendars

An alternative and also more challenging approach would be adding Persian Calendar as an optional calendar. This requires more detailed information around how locale specific information are managed by CultureInfo. In fact, CultureInfo retrieves culture data from complicated data structures stored in locale files under Windows operating system. Data such as the array of optional calendars are stored in specific data structure and retrieved by special manipulation of pointers. The code given below shows how OptionalCalendars are retrieved from a CultureTableRecord class:

internal int[] OptionalCalendars
{
    get
    {
        if (this.optionalCalendars == null)
        {
            this.optionalCalendars = this.m_cultureTableRecord.IOPTIONALCALENDARS;
        }
        return this.optionalCalendars;
    }
}

CultureTableRecord then returns:

internal int[] IOPTIONALCALENDARS
{
    get
    {
        return this.GetWordArray(this.m_pData.waCalendars);
    }
}

which finally returns optional calendars as:

private unsafe int[] GetWordArray(uint iData)
{
    if (iData == 0)
    {
        return new int[0];
    }
    ushort* numPtr = this.m_pPool + ((ushort*) iData);
    int num = numPtr[0];
    int[] numArray = new int[num];
    numPtr++;
    for (int i = 0; i < num; i++)
    {
        numArray[i] = numPtr[i];
    }
    return numArray;
}

Note how pointer calculations are encountered in this evaluation.

To fix the optional calendars of Persian locale, one should set the Persian calendar identifier in the appropriate place in the locale data structure. This location may be back calculated from source code above. Then using reflection again to get access to private fields, one may get access to the array of optional calendars and fix it on the fly.

But there is still another problem. The array lies in a protected memory area. That is, you have no write access to that part of memory. A workaround is using VirtualProtect to make this memory writeable before attempting to write back the optional calendars back:

public static  CultureInfo FixOptionalCalendars(CultureInfo culture, int CalenadrIndex)
{
    InvokeHelper ivCultureInfo = new InvokeHelper(culture);
    InvokeHelper ivTableRecord = new InvokeHelper(ivCultureInfo.GetField("m_cultureTableRecord"));

    // Get the m_pData pointer as *void
    System.Reflection.Pointer m_pData = (System.Reflection.Pointer)ivTableRecord.GetField("m_pData");

    ConstructorInfo _intPtrCtor = typeof(IntPtr).GetConstructor(
                    new Type[] { Type.GetType("System.Void*") });

    // Construct a new IntPtr
    IntPtr DataIntPtr = (IntPtr)_intPtrCtor.Invoke(new object[1] { m_pData });

    Type TCultureTableData = Type.GetType("System.Globalization.CultureTableData");

    // Convert the Pointer class to object if type CultureTableData to work with
    // reflection API.
    Object oCultureTableData = System.Runtime.InteropServices.Marshal.PtrToStructure
                               (DataIntPtr, TCultureTableData);
    InvokeHelper ivCultureTableData = new InvokeHelper(oCultureTableData);

    // Get waCalendars pointer
    uint waCalendars = (uint)ivCultureTableData.GetField("waCalendars");
    object IOPTIONALCALENDARS = ivTableRecord.GetProperty("IOPTIONALCALENDARS");

    // Get m_Pool pointer
    System.Reflection.Pointer m_pool = (System.Reflection.Pointer)ivTableRecord.GetField("m_pPool");
    IntPtr PoolInPtr = (IntPtr)_intPtrCtor.Invoke(new object[1] { m_pool });

    // Add the waCalendars offset to pool pointer
    IntPtr shortArrayPtr = new IntPtr((PoolInPtr.ToInt64() + waCalendars*sizeof(ushort)));
    short[] shortArray = new short[1];

    // Now shortArray points to an arry of short integers.
    // Go to read the first value which is the number of elements.
    // Marshal array to read elements.
    System.Runtime.InteropServices.Marshal.Copy(shortArrayPtr, shortArray, 0, 1);

    // shortArray[0] is the number of optional calendars.
    short[] calArray = new short[shortArray[0]];

    // Add one element of short type to point to array of calendars
    IntPtr calArrayPtr = new IntPtr(shortArrayPtr.ToInt64() + sizeof(short));

    // Finally read the array
    System.Runtime.InteropServices.Marshal.Copy(calArrayPtr, calArray, 0, shortArray[0]);

    uint old;
    VirtualProtect(calArrayPtr, 100, 0×4, out old);
    calArray[CalenadrIndex] = 0×16;
    System.Runtime.InteropServices.Marshal.Copy(calArray, 0, calArrayPtr, calArray.Length);
    VirtualProtect(calArrayPtr, 100, old, out old);
    return culture; 
}

CultureData in .NET Framework 4.0

The CultureTableRecord class has been replaced by CultureData which holds the Optional Calendars as a private array of integers in waCalendars field. This makes correction of Optional Calendars as easy as correcting a private field:

private static CultureInfo _FixOptionalCalendars4(CultureInfo culture, int CalenadrIndex)
{
    FieldInfo cultureDataField = typeof(CultureInfo).GetField("m_cultureData",
         BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance );
    Object cultureData = cultureDataField.GetValue(culture);
    FieldInfo waCalendarsField = cultureData.GetType().GetField("waCalendars",
        BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);
    int[] waCalendars = (int[])waCalendarsField.GetValue(cultureData);
    if (CalenadrIndex >= 0 && CalenadrIndex < waCalendars.Length)
        waCalendars[CalenadrIndex] = 0x16;
    waCalendarsField.SetValue(cultureData, waCalendars);
    return culture;
}

Conclusion

Problems with Persian culture in .NET are discussed and methods for correcting these problems are proposed. You may download the sample code from here.