Emulating EEPROM using Flash Memory on dsPIC33EP512MC502

One thing I don’t like about the dsPICEE3P (and the PIC32MX) lines is that these devices do not have EEPROM which is very useful to store non-volatile data, e.g., user configuration settings. While you can always use an external EEPROM for this, adding a 24C64 will waste pins and increase production costs:

This post will show you how to self-program the PIC flash memory in order to store non-volatile data, without the need for external EEPROM devices.

The device I am using is a dsPIC33EPMC502, but you can adapt these instructions for other dsPIC or PIC32 devices.

The first thing you should be aware of is that, while you can read and write bytes at any arbitrary location of an EEPROM, flash memory can only be programmed (e.g., erased and written) in pages of 3072 bytes, depending on your device. For this, you should also read part 4.0 (Memory Operation) as well as part 5.0 (Flash Program Memory) of the device datasheet. You should also read section DS70609 (Flash Programming) of the dsPIC33/PIC24 Family Reference Manual.

Next, while most EEPROMs can be written to at least hundreds of thousands of times, flash memory has limited write cycles; exceeding the cycle count and data could be corrupted. To allow for this, a wear leveling algorithm needs to be implemented. Refer to TABLE 30-14: DC CHARACTERISTICS: PROGRAM MEMORY (page 412 of the datasheet), characteristic “Cell Endurance” for the expected life cycle count. A typical value is just 10,000 times. Read cycles are practically unlimited, both on EEPROM and flash memory.

In our implementation, we emulate EEPROM with flash memory by storing multiple copies of the configuration object into a single flash page. Upon reading, locate the highest valid configuration object and use it as the latest configuration. To save user data, write to the next configuration object if the last slot is not used yet. If the last slot is already used, we erase the entire page and write to the first slot again. This way, we can write the configuration for at least 640,000 times (reads are unlimited).

Each configuration object, designed as a C struct, will require some extra values (besides the usual values for user configuration settings). These values include a checksum, a slot index, and optionally an erase count. The checksum, which can just be a CRC16, will help your code decide whether that copy of the configuration is valid. As a further guard, the slot index must match the position of the configuration object in memory. Finally, the erase count will help your application to warn the user when the write cycle has reached its limit. Of course, the erase count has to be incremented after each write.

Using C bitfields, the configuration object (with a 2 bytes CRC16 checkum) can be declared as follows:

typedef struct _APP_CONFIG {
    unsigned int  crc16Checksum;
    unsigned char slotInd;
    unsigned int  nvmEraseCount;

    // Example of user defined data
    unsigned char curPlayMode : 4;
    unsigned char durationMode : 4;
    unsigned char powerType : 4;
    unsigned char idleOffMode : 4;
    .....
} APP_CONFIG;

To allow access to the actual member of the config object (e.g., curPlayMode, durationMode, etc.) as well as the raw data bytes (for flash programming), a C union is used:

union ConfigData {
    // config object, size must be smaller than FLASH_CFG_LEN
    APP_CONFIG config;

    // configuration data bytes
    unsigned char data_bytes[FLASH_CFG_LEN];
};
union ConfigData appConfigObj;

FLASH_CFG_LEN is the maximum length in bytes of the config object and should be a multiple of 6 (see later) and at least sizeof(APP_CONFIG). The flash memory page size should be multiples of FLASH_CFG_LEN. In my design, I used 48 for FLASH_CFG_LEN, so 64 copies of the configuration data can be stored in flash memory. When adding more user-defined member to the APP_CONFIG struct, make sure that sizeof(APP_CONFIG) is always smaller then FLASH_CFG_LEN, else there will be weird issues. In my setup, the configuration can be written for up to 640,000 times before reaching the EEPROM write cycle limit.

To make sure that the flash configuration bytes are not erased after every debugging attempt, in MPLAB, right click your project and choose Set Configuration > Customize > Configuration > PICKit4. Choose Memories to Program > Preserve program memory and enter 54800-54fff (inclusive). Do not enter 54800-55000 which will result in exclamation marks, because the number of words must be even. This will exclude the flash configuration region from being programmed.

Next, we should declare a dummy array to programmatically retrieve the absolute location in flash memory to store app configuration bytes. Depending on your device, this array should be located at top of flash memory, e.g., from 0x557ec downwards while avoiding the last word to avoid linker errors. The area should fit exactly 1 memory page (1024*24-bit words or 3072 bytes) to avoid the need to erase multiple flash memory page when writing configuration. Because the linker is 24-bit word-aligned, 3 bytes is needed to stored 2 bytes. Hence the array should contain exactly 3072 / 1.5 = 2048 (0x800) bytes to cover 3072 bytes of raw flash memory.

#define FLASH_VAR_ADDR               0x54800
#define FLASH_24BIT_WORD_LENGTH      0x800        // 2048
#define FLASH_BYTE_LENGTH            0xC00        // 3072
const unsigned char __attribute__((section("flashVars"), 
    space(prog), address(FLASH_VAR_ADDR))) flashVars[FLASH_24BIT_WORD_LENGTH];

Take note that the above array declaration is solely to retrieve the raw memory address. The code to read/write flash memory has access to all 3072 bytes of memory in a single flash page.

Now implement the following function which will read “len” bytes from “loc” in flash memory into “bytes” array. I am using MPLAB X with the XC16 compiler:

void readFlashBytes(unsigned int loc, unsigned int len, unsigned char bytes[])
{
    unsigned int offset, tbl, d, low_data, high_data;

    tbl = __builtin_tblpage(&flashVars);
    offset = __builtin_tbloffset(&flashVars) + loc;

    TBLPAG = tbl;
    d = 0;
    for (d = 0; d < len; d += 3)
    {
        // Returns bits <15:0> of the specified program memory address.
        low_data = __builtin_tblrdl(offset);
        bytes[d] = low_data & 0xFF;
        bytes[d+1] = low_data >> 8;

        // The lower eight bits of the response
        // are bits <23:16> of the specified program memory address.
        // The upper eight bits returned are 0 (they don't exist).
        high_data = __builtin_tblrdh(offset);
        bytes[d+2] = high_data & 0xFF;

        // debugPrint("%04X %04X %04X %04X", offset, d, low_data, high_data);        

        offset += 2;
    }
}

As multiple copies of the configuration objects are stored in flash memory and each copy is FLASH_CFG_LEN bytes long, the following function will read a particular copy of the configuration object:

// one copy of flash configuration array (length divisible by 6)
// multiple copies of this array exists in memory for wear levelling
// length of config selected to match exactly the flash page 
// (48 bytes * 64 copies = 3072)
#define FLASH_CFG_24BIT_WORD_LEN        (FLASH_CFG_LEN * 2 / 3) // length, where 3 
                                                                // bytes counted as 2.
#define FLASH_CFG_NUM_COPIES            (FLASH_24BIT_WORD_LENGTH / 
 FLASH_CFG_24BIT_WORD_LEN)        // number of copies of config in flash
void readFlashCfg(unsigned char copyInd)
{
    readFlashBytes(copyInd * FLASH_CFG_24BIT_WORD_LEN, FLASH_CFG_LEN, 
                   appConfigObj.data_bytes);
}

This function, adapted from here, will erase an entire flash memory page. It will return TRUE if successful, FALSE otherwise.

// Mask to ensure that flash erase/write starts at 1024-word (2048 bytes) boundary
#define FLASH_BASE_ADDR_MASK    0xF800

BOOL eraseFlashPage()
{
    unsigned int i = 0;
    unsigned int offset = __builtin_tbloffset(&flashVars);
    unsigned int baseAddr = (offset & FLASH_BASE_ADDR_MASK);

    // debugPrint("Erase starts: %04X", baseAddr);

    NVMADRU = __builtin_tblpage(&flashVars);
    NVMADR = baseAddr;              // Base address.
    NVMCONbits.WREN = 1;            // Write enable
    NVMCONbits.NVMOP = 0b0011;      // Page erase
    DISABLE_INTERRUPT;              // Disable interrupt
    __builtin_write_NVM();          // Initiate write sequence

    while (NVMCONbits.WR) {          // Wait for completion
        i++;
        delay_ms(1);

        if (i > 60000)
        {
            SendUARTStr("ErsEr");
            return FALSE;
        }
    }

    debugPrint("EO:%04X", baseAddr);
    return TRUE;
}

The following macro will disable interrupt for the next few instructions, specified as the first parameter. Interrupts level 7 are not affected by this, hence caution should be taken not to use interrupt level 7 while using this function:

#define DISABLE_INTERRUPT       __builtin_disi(6)

Use the following function to verify if the erase has been successful, e.g., all bytes reset to FF.

BOOL verifyEraseOK()
{
    unsigned int c, MyOffset, low_data, high_data;

    TBLPAG   = __builtin_tblpage(&flashVars);
    MyOffset = __builtin_tbloffset(&flashVars);

    for (c = MyOffset; c < MyOffset + FLASH_24BIT_WORD_LENGTH; c+= 2)
    {
        low_data = __builtin_tblrdl(c);                // Low word (bottom 16 bits)
        high_data = __builtin_tblrdh(c);               // High byte (top 8 bits)

        if ( (low_data != 0xFFFF) ||  ( (high_data & 0xFF) != 0xFF) )
        {
            debugPrint("ERV:%04X", c);
            return FALSE;
        }
    }

    // SendUARTStr("Verify Erase OK");

    return TRUE;
}

The following function will write an array of 6 bytes into flash memory at offset “loc”:

BOOL writeFlash6Bytes(unsigned int loc, unsigned char bytes[6])
{
    unsigned int i = 0;

    // dsPIC33EP512MC502 does not support row write
    // Supported operations can be found in NVMCON.NVMOP bits.
    // Only page erase and double-word write are supported for this dsPIC.

    // Bits 11:8 of first write latch, listed in section 
    // "Memory Organization" of datasheet
    // This dsPIC has 2 write latches (0xFA0000 and 0xFA0001). 
    // Each write latch can store 3 bytes
    // When both write latches have been used, 
    // call __builtin_write_NVM to write data to flash
    TBLPAG = 0xFA;                   

    NVMADRU = __builtin_tblpage(&flashVars);
    NVMADR  = __builtin_tbloffset(&flashVars) + loc;;      // Each page is 
    // 1024 * 24-bit words. Page size is listed within first few pages of datasheet

    __builtin_tblwtl(0, (bytes[1] << 8) | bytes[0]);       // Low word 1 (bottom 16 bits)
    __builtin_tblwth(0, bytes[2]);                         // High byte 1 (top 8 bit)
    __builtin_tblwtl(2, (bytes[4] << 8) | bytes[3]);       // Low word 2 (bottom 16 bits)
    __builtin_tblwth(2, bytes[5]);                         // High byte 2 (top 8 bit)

    NVMCONbits.WREN  = 1;           // Enable Write
    NVMCONbits.NVMOP = 0b0001;      // Double-word Write 

    DISABLE_INTERRUPT;              // Disable interrupt
    __builtin_write_NVM();          // Initiate write sequence

    while (NVMCONbits.WR) {         // Wait for completion
        i++;
        delay_ms(1);

        if (i > 60000)
        {
            SendUARTStr("WrEr");
            return FALSE;
        }
    }

    // debugPrint("Write done: #%04X", offset);
    return TRUE;
}

The following function will write an array of bytes into flash memory. The array size should be divisible by 6, which is why FLASH_CFG_LEN declared earlier must be a multiple of 6.

BOOL writeFlashMulti6Bytes(unsigned int loc, unsigned int len, unsigned char bytes[])
{
    unsigned int c1, c2;

    c1 = 0;
    c2 = 0;

    if (len == 0 || len % 6 != 0)
    {
        SendUARTStr("FlWrLenEr");
    }
    else {
        while (c2 < len)
        {
            // debugPrint("wr %04X %04X", c2, loc);

            // each write is on 2*24-bit words
            // flash location pointer (24-bit word) moves by 4
            // data location pointer moves by 6
            if (!writeFlash6Bytes(c1 + loc, bytes + c2))
                return FALSE;

            c1 += 4;
            c2 += 6;
        }
    }

    return TRUE;
}

The following function will write a particular copy of the configuration object into flash memory:

BOOL writeFlashCfg(unsigned char copyInd)
{
    return writeFlashMulti6Bytes(copyInd * FLASH_CFG_24BIT_WORD_LEN, 
                                 FLASH_CFG_LEN, appConfigObj.data_bytes);
}

The following will test to make sure that the functions to read/write config is working as expected by writing random configuration data and reading it back. It should be used sparingly to test code logic as flash write cycles are limited.

BOOL verifyFlashCfgReadWrite()
{
    unsigned int c, copyInd;
    unsigned char expected;

    debugPrint("cSz=%d,cpy=%d", FLASH_CFG_LEN, FLASH_CFG_NUM_COPIES);

    // test data
    for (c = 0; c < FLASH_CFG_LEN; c++)
    {
        expected = ( (~(c & 0xFF)) ^ 0xAF);
        appConfigObj.data_bytes1 = expected;

        // debugPrint("%d: %02X", c, expected & 0xFF);
    }

    for (copyInd = 0; copyInd < FLASH_CFG_NUM_COPIES; copyInd++)
    {
        // debugPrint("test #%d", copyInd);

        // write test data
        writeFlashCfg(copyInd);

        // reset test data so that we can test if we read back the same
        memset(appConfigObj.data_bytes, 0, FLASH_CFG_LEN);

        // read back test data to verify
        readFlashCfg(copyInd);
        for (c = 0; c < FLASH_CFG_LEN; c++)
        {
            expected = ( (~(c & 0xFF)) ^ 0xAF);

            if ( appConfigObj.data_bytes1 != expected)
            {
                debugPrint("V#%d Er%d:%02X/%02X", copyInd, c, 
                            appConfigObj.data_bytes1 & 0xFF, expected & 0xFF);
                return FALSE;
            }
            else {
                // SendUARTStr("VOK");
            }
        }

        // debugPrint("verify ok #%d", copyInd);
    }

    SendUARTStr("VeRwOk");
    return TRUE;
}

Now it’s time to read and time some useful data. For this, we will need a function which creates a CRC16 checksum of a given array. I adapted this function from here.

unsigned int gen_crc16(const unsigned char* data_p, 
                       unsigned char offset, unsigned char length){
    unsigned char x;
    unsigned int crc = 0xFFFF;
    unsigned char ind = 0;

    for (ind = offset; ind < length; ind++){
        // debugPrint("%u - %02X", ind, data_p[ind]);

        x = crc >> 8 ^ data_p[ind];
        x ^= x>>4;
        crc = (crc << 8) ^ ((unsigned short)(x << 12)) ^ 
              ((unsigned short)(x <<5)) ^ ((unsigned short)x);
    }
    return crc;
}

The following function will scan the entire flash memory page for any valid configuration object. If there is one, its slot index will be returned as cfgInd. The erase count will also be returned as nvmCount. The actual configuration data will be copied to the global variable appConfig.

BOOL readAppConfig(unsigned char *cfgInd, unsigned int *nvmCount)
{
#define CFG_IND_INVALID     0xFF

    unsigned char slot;
    *cfgInd = CFG_IND_INVALID;
    BOOL foundCfg = FALSE;

    // debugPrint("Total slots: %d", FLASH_CFG_NUM_COPIES);
    for (slot = 0; slot < FLASH_CFG_NUM_COPIES; slot++)
    {
        readFlashCfg(slot);

        unsigned int crc16csum = gen_crc16(appConfigObj.data_bytes, 
                                           CFG_CHECKSUM_SIZE, FLASH_CFG_LEN);
        if (appConfigObj.config.crc16Checksum == crc16csum && 
                         appConfigObj.config.slotInd == slot)
        {
            // debugPrint("CFGOK#%d CSUM=%04X", slot, crc16csum);

            if (slot > *cfgInd || *cfgInd == CFG_IND_INVALID)
            {
                *cfgInd = slot;
                foundCfg = TRUE;
            }
        }
        else {
            // debugPrint("CFGERR#%d CSUM=%04X", slot, crc16csum);
        }
    }

    if (foundCfg)
    {
        *nvmCount = appConfigObj.config.nvmEraseCount;

        debugPrint("RD#%d CS=%04X NV#%u S0=%d S1=%d", 
                   *cfgInd, appConfigObj.config.crc16Checksum, 
                   appConfigObj.config.nvmEraseCount, 
                   sizeof(APP_CONFIG), sizeof(appConfigObj));
        return TRUE;
    }
    else {
        SendUARTStr("RCEr");
        return FALSE;
    }
}

The following function will write the current appConfig data, stored as global variable, into the next available slot of the flash memory.

BOOL writeAppConfig()
{
    slot = (slot + 1) % FLASH_CFG_NUM_COPIES;
    appConfigObj.config.slotInd = slot;

    BOOL isOK = writeFlashCfg(slot);
    if (isOK)
    {
       debugPrint("SCF#%d NV#%u CSM=%04X", slot, nvmCount, csum);
    }
    else {
       SendUARTStr("CfSvEr");
    }
}

Your code will also need to have some logic to decide when to erase the flash memory page, for example, when you are writing to flash for the first time, or when you have used up 64 slots and need to return to slot 0. In such case, call eraseFlashPage(), reset slot index and also increase the nvmCount:

eraseFlashPage();
appConfigObj.config.slotInd = 0;
appConfigObj.config.nvmEraseCount = appConfigObj.config.nvmEraseCount+1;

With this setup, I can happily use flash memory to store non-volatile data in my projects and I do not need to worry about adding external EEPROM devices.

See Also

• Measuring temperature using the dsPIC Charge Time Measurement Unit (CTMU)