Using Ardunio Flash Memory for variable parameters

• Download demo - 2 KB

Introduction

When you compile and upload a sketch onto an Arduino the programme code is stored in flash memory (PROGMEM) and there is an area of SRAM which is used by the sketch for its variables when it runs.

Every time the board is powered up the programme code in flash runs. After various system initialisations your setup() function runs and then the main programme code in loop() is executed repeatedly until the power is removed.

In this simple model there is no way to save data between sessions. All variables will be re-initialised each time the program runs.

If you have large static constants then you can save precious SRAM space by putting them in PROGMEM at compile time and using the pgmspace library to access them.

There is a third area of memory called EEPROM (electrically erasable programmable read only memory) which you can read and write using the eeprom library. EEPROM and PROGMEM are non-volatile and hold their contents when the power is removed, but can be erased and a new value written. There is a limit to the number of times it can be written - although quite large (approx 100,000 writes) you don't want to be using it for rapidly changing variable data. It can be very useful for holding more or less static variables, or parameters.

PROGMEM can only be written when the programme is first uploaded, so is fine for holding unchanging constant values. Even if you loaded a new programme version every day it would take you 273 years to wear out the flash memory.

EEPROM is accessible to your programme using the eeprom library, but you need to be a little careful about how often you use it. If your main programme loop executes every 10ms and updates a value in EEPROM every time then you will hit the 100,000 writes limit after 16 minutes of execution.

But it is fine to use for data that you need to change occasionally - for example you might like to increment a counter every time the programme is run, or set the value of a trigger threshold that will be remembered next time the board is powered up.

However there is a problem - it is not currently possible to easily initialise EEPROM locations at programme upload time. When the chip is new every location will contain &FF (all bits = 1), but if a location has ever been written to then it will be different.

So the problem is how to know whether the data in EEPROM is what your sketch saved there last time it was run, or some data that some other sketch has left there, or if it has got corrupted

It is possible to hack the Arduino IDE system so that a compiler directive EEMEM is correctly enabled to initialise EEPROM locations. This would allow you to set initial values when your sketch was first compiled and uploaded, but it does require some confidence in hacking the control files for the Arduino IDE on any system you might want to compile your sketch on - and repeating the process potentially every time the IDE is updated.

The workaround is to build in some checking of the EEPROM area you want to use in the setup() function.

Background

The Arduino reference page on types of memory is here http://playground.arduino.cc/Learning/Memory

If you are confident you can follow the instruction to hack the Arduino IDE here http://www.fucik.name/Arduino/eemem.php which will enable you to initialise the EEPROM when the code is uploaded.

Or we can use a workaround that involves saving a unique value for our sketch and version in PROGMEM at compile time and when the sketch runs during setup() comparing this constant with what is in a specific EEPROM location. If it doesn't match then we will initialise our values in EEPROM and write our key to the specific location.

If it does match then probably the rest of the data in the EEPROM area we are using will be valid, but we should run some other checks in case it happens that a different sketch has been loaded since we wrote our key and has changed some of the other locations.

Depending on the type of board we have between 512 and 4096 bytes of EEPROM. This doesn't give us a lot of space to play with so we may need to be a bit parsimonious in how much memory we use for the key. A single byte will not really be enough as there is a fair chance that something else could have written that value.

We also need to consider whether we want to reset the memory every time we upload a new version of the sketch, or whether a minor upgrade will allow the previous values in EEPROM to be retained.

Most people using EEPROM will probably start using it from the first location onwards, so we will save our key at the begining of the memory as that way it is most likely to be corrupted if some other sketch is loaded and uses the EEPROM thus invalidating our saved values.

Using the code

For demo purposes we will use a simple sketch that is designed to run on a standalone device monitoring the status of an analogue input and turning on a digital output (for example to light a warning LED or sound an alarm) if it exceeds a threshold value.

We will also keep track of the number of times the programme has been run since it was installed.

int AnaPin = A0;
int LEDpin = 3;
static unsigned long runCount = 0;   // how many times the sketch has been run
static int threshold = 200;            // the threshold above which the LED will turn on
boolean ledon = false;

void setup(){
    pinMode(LEDpin, OUTPUT);
    pinMode(AnaPin, INPUT);
    runCount += 1;
    delay(200)
}

void loop()
{
    ledon = (analogRead(AnaPin) >= threshold);
    digitalWrite(LEDpin, ledon);
    delay(500);
}

In a real application we might include some de-bounce or hysteresis function on the trigger so that noise in the analogue input didn't cause us problems but for this example we'll just sample every half second

But we need to be able to save the values of runCount and threshold and not reset them every time the programme starts.

#include <EEPROM.h>   //enable use of eeprom
int epromStart = 0;   //the address of the first location in eeprom we will use
// ....
// this in setup()
        int t;
        EEPROM.get(epromStart, runCount);        //get the runCount from EEPROM
        EEPROM.get((epromStart + sizeof(long)), threshold);   //threshold is immedaitely after runCount in EEPROM
        runCount += 1;
        EEPROM.put(keylen, runCount); //write the updated run count back to eeprom
// carry on with setup

Ok, that'll work fine so long as the eeprom has been initialised with valid values. To check this we will define a constant string in PROGMEM which will be set when the programme is uploaded. The first time the programme runs we will write the same string into EEPROM and then we can compare the two and if they match we can assume that the EEPROM contains valid data for us.

Since we often generate a string constant containing the name of the sketch anyway we will use that - it is pretty likely to be unique, and if we want to invalidate the old data in EEPROM when we upload a new version we can slightly change the name when compiling.

#include <avr/pgmspace.h>   //enable use of flash memory

const char sketchName[] PROGMEM = "EepromDemoV1";        // This will be our key to see if EEPROM is valid
static int keylen;    
                                
char strbuffer[50];                                    // variable to copy strings from flash memory as required
int x;
boolean eok = false;
String key = "";
String chk = "";

char* getProgmemStr(const char* str) {
    /** gets a string from PROGMEM into the strbuffer */
    strcpy_P(strbuffer, (char*)str);
    return strbuffer;
}

char* getEeepromStr(int start, int len) {
    /** gets a string from EEPROM into the strbuffer */
    for (x = 0; x < len; x++) {
        strbuffer[x] = EEPROM.read(x);
    }
    strbuffer[x] = 0;
    return strbuffer;
}

void putEepromStr(int start, String str) {
    /** puts a string into the eeprom at a given address */
    int strlen = str.length() + 1;
    char chArray[strlen];
    str.toCharArray(chArray, strlen);
    for (x = start; x < (start + strlen); x++) {
        EEPROM.write(x, chArray[x]);
    }
}

void setup()
{
    key = getProgmemStr(sketchName);
    keylen = key.length() + 1;
    chk = getEeepromStr(0, keylen);
    if (key == chk) {
        //ok we've got our expected key in eeprom, now does the rest of the data look ok?
        //we will be storing the run count as an unsigned long immediately after the key, this could have any any value
        // threshold could have any value from 0 to 1023 which is the max we get from the ADC, so if it is greater it is not valid
        eok = true;
        int t;
        EEPROM.get(keylen, runCount);                //get the runCount from EEPROM immediately after the key
        EEPROM.get((keylen + sizeof(long)), t);        //threshold is immedaitely after runCount in EEPROM
        if (t > 1023) {            //if it is out of range then the EEPROM is invalid and we'll need to reset it
            eok = false;
        }
        else {  //ok we can use the values from eeprom
            threshold = t;
            runCount += 1;
            EEPROM.put(keylen, runCount); //write the updated run count back to eeprom
        }
    }
    if (!eok) { //invalid data in EEPROM so either this is a first run or it has been corrupted
        putEepromStr(0, key); //write the valid key at the begining of eeprom
        chk = String(getEeepromStr(0, keylen));  
        runCount = 0;
        EEPROM.put(keylen, runCount);
        EEPROM.put((keylen + sizeof(long)), threshold);
    }
    // we have left eok as is so we can report it if required

That's quite a chunk. Lets break it down.

First we defined a string constant in PROGMEM, a variable to hold its length as a character array and a buffer to copy character arrays to when reading from PROGMEM or EEPROM.

Personally I prefer to use String objects in Arduino code rather than simple string character arrays as it makes for more readable (and therfore maintainable) code and provides a lot of useful functionaility.

Then we have three short generic functions we can use to get a String from PROGMEM, EEPROM and write a String to EEPROM.

getProgmemStr() we pass a pointer to the character array in PROGMEM and it copies it into the buffer using strcpy_P() and returns the pointer to the buffer.

For getEepromStr() we have to pass it the start address in EEPROM and the length of the character array we are expecting back. Again it copies it (byte by byte this time) into the buffer and null terminates it so we can read it as a string.

For writing a string to the EEPROM putEepromStr() takes the start address and the String object and converts the object into a char array and then iterates through writing the bytes into EEPROM.

Now at the begining of the setup() we can get the key from progmem, see what its length is and get the corresponding number of bytes from EEPROM and compare them.

If they match then we can assume the EEPROM is valid and use the values for threshold and runCount from there, otherwise we will use the default values and write them in to EEPROM.

We are also now adding a check on the value of threshold - we know that the maximum value we can expect from our analogue pin is 1023, so if threshold is greater than that then we will assume it is invalid and write default values into all the locations.

Now we also need some way to set the threshold value and read back the runcount from a master device connected to the serial port.

We will implement a really simple serial protocol so that if we send "t123x" to the board it will interpret this as a command to set the threshold to 123 (or whatever value comes between the 't' and the 'x'

Any other character recieved on the serial port will cause us to report back the current runCount and threshold.

// at the end of setup()
    Serial.begin(115200);
    Serial.println();
    delay(500);
}

void loop()
{
    char serialChar;
    if (Serial.available() > 0) {
        while (Serial.available() > 0) {
            serialChar = Serial.read();
            if (serialChar == 't') {
                doThreshold();
            }
        }
        Serial.print("key="); Serial.println(key);
        Serial.print("chk="); Serial.println(chk);
        Serial.print("eok="); Serial.println(eok);
        Serial.print("runcount="); Serial.println(runCount);
        Serial.print("threshold="); Serial.println(threshold);
    }
    ledon = (analogRead(AnaPin) >= threshold);
    digitalWrite(LEDpin, ledon);
    delay(500);
}

void doThreshold(void) {
/* this will read chars from serial until an 'x' is encountered 
* anything other than digits will be ignored
* the value recieved will be used to set a new threshold
*/
    boolean endOfDigits = false;
    String cmdStr = "";
    int inChr;
    while (!endOfDigits) {
        inChr = Serial.read();
        if (isDigit(inChr)) {
            cmdStr += (char)inChr;
        }
        else if (inChr == 'x') {
            endOfDigits = true;
        }
    }
    int intVal = 0;
    if (cmdStr.length() >= 1) { // if we have a number > 0 
        intVal = cmdStr.toInt();
        Serial.print("intVal="); Serial.println(intVal); //echo the value back for debugging
        if (threshold != intVal) {
            threshold = intVal;
            EEPROM.put((keylen + sizeof(long)), threshold);
        }
    }
}

Each time around the main loop (every half second) if there is anything in the serial buffer we will read it. If we find a 't' then we will go and doThreshold() which will read serial chars until we get an 'x' and a valid number.

If we don't get a t then we will simply empty the serial buffer by reading it and then write out the current values.

Note that we could check in doThreshold() that we've got a valid value (<1024). If you send "t1025x" to the serial port then it will use this and write it to the EEPROM but next time the board is powered up it will find the invalid value in there and reset itself.

Points of Interest

Obviously this is just a simple example and not fully robust, it can be much improved.

For a fully robust solution we should also calculate a checksum for the area of memory we are using every time we update a value and save that at the end of our block. This would also guard against corruption of our data whilst our own programme was loaded.

The reading and writing of Strings to flash memory has been bundled up in functions as I find myself re-using these often.

You can find out how much EEPROM your particular board has using EEPROM.length() which will allow you to ensure that you don't overflow it with unpredictable results. You can of course place your data anywhere you like in the EEPROM, you don't have to start at zero.

In practice if you are logging data in a standalone device you will almost certainly use an external memory like one of the SD card shields. EEPROM is really best used for parameters and status information that you want to keep with the board when it is powered down or when the SD card is changed.

History

First published 4th August 2016