EEPROM emulator for 8-bit data

Post your cool example code here.
Post Reply
arpruss
Posts: 83
Joined: Sat Sep 30, 2017 3:34 am

EEPROM emulator for 8-bit data

Post by arpruss » Wed Nov 15, 2017 2:46 pm

I wanted to store some persistent data from time to time, but my data was just a single configuration byte. The libmaple core's EEPROM emulation code is designed for 16-bit storage and would involve using up four bytes to store that byte, which would mean that there would be one flash erase
per 2.5K data stores. So I wrote my own module that uses two bytes per data value, so there is an erase every 5K store, which is faster and increases flash lifetime. Specifically, you can have up to 255 byte sized configuration variables, numbered 0-254.

I really wanted to be able to use only one byte per data point, which should be possible given that I have only one variable, but it turns out that the STM32F1's flash code only lets you write 16-bits at a time, and doesn't let you do the nice thing where you can turn 1 bits to 0 bits even in an already-written byte.

API:

Code: Select all

void EEPROM8_init(void);
uint8_t EEPROM8_getValue(uint8_t variableNumber); // 255 if missing
boolean EEPROM8_storeValue(uint8_t variableNumber, uint8_t value); // true on success
Code:

Code: Select all

#include <EEPROM.h>
#include <flash_stm32.h>

static boolean invalid = true;
static uint8_t currentPage = 0;

#define EEPROM8_MEMORY_SIZE (2*EEPROM_PAGE_SIZE)
#define GET_BYTE(address) (*(__io uint8_t*)(address))
#define GET_HALF_WORD(address) (*(__io uint16_t*)(address))
#define GET_WORD(address) (*(__io uint32_t*)(address))

#define EEPROM8_MAGIC (uint32_t)0x1b70f1cd

const uint32_t pageBases[2] = { EEPROM_PAGE0_BASE, EEPROM_PAGE1_BASE };

static bool erasePage(uint32_t base) {
  bool success;

  if (base != EEPROM_PAGE0_BASE && base != EEPROM_PAGE1_BASE) {
    return false;
  }

  FLASH_Unlock();
  
  success = ( FLASH_COMPLETE == FLASH_ErasePage(base) );
  
  success = success && 
    FLASH_COMPLETE == FLASH_ProgramHalfWord(base, (uint16_t)EEPROM8_MAGIC) &&
    FLASH_COMPLETE == FLASH_ProgramHalfWord(base+2, (uint16_t)(EEPROM8_MAGIC>>16));
  FLASH_Lock();  
    
  return success && EEPROM8_MAGIC == GET_WORD(base);
}

static bool erasePages() {
  return erasePage(EEPROM_PAGE0_BASE) && erasePage(EEPROM_PAGE1_BASE);
}

uint8_t EEPROM8_getValue(uint8_t variable) {
  if (invalid)
    return -1;

  uint32_t base = pageBases[currentPage];
  for (uint32_t offset = EEPROM_PAGE_SIZE-2 ; offset >= 4 ; offset-=2) {
    if (GET_BYTE(base+offset) == variable) {
      return GET_BYTE(base+offset+1);
    }
  }

  return -1;
}

static bool writeHalfWord(uint32_t address, uint16_t halfWord) {
  if (! ( EEPROM_PAGE0_BASE <= address && address+1 < EEPROM_PAGE0_BASE + EEPROM_PAGE_SIZE ) &&
     ! ( EEPROM_PAGE1_BASE <= address && address+1 < EEPROM_PAGE1_BASE + EEPROM_PAGE_SIZE ) ) {
    return false;
  }
  
  FLASH_Unlock();
  boolean success = FLASH_COMPLETE == FLASH_ProgramHalfWord(address, halfWord);
  FLASH_Lock();  

  return success && GET_HALF_WORD(address) == halfWord;
}

boolean EEPROM8_storeValue(uint8_t variable, uint8_t value) {
  if (invalid)
    return false;

  if ((uint16_t)value == (uint16_t)EEPROM8_getValue(variable))
    return true;
  
  uint32_t base = pageBases[currentPage];
  bool err = false;
  
  for (uint32_t offset = 4 ; offset < EEPROM_PAGE_SIZE ; offset+=2) {
    if (GET_HALF_WORD(base+offset) == 0xFFFF) {
      return writeHalfWord(base+offset, variable | ((uint16_t)value<<8));
    }
  }

  uint32_t otherBase = pageBases[1-currentPage];
  // page is full, need to move to other page
  if (! erasePage(otherBase)) {
    return false;
  }

  // now, copy data
  uint32_t outOffset = 4;

  for (uint32_t offset = EEPROM_PAGE_SIZE; offset >= 4 ; offset-=2) {
    uint16_t data;
    
    if (offset == EEPROM_PAGE_SIZE)
      data = variable | ((uint16_t)value<<8); // give new data value priority
    else
      data = GET_HALF_WORD(base+offset);
      
    if (data != 0xFFFF) {
      uint8_t variable = (uint8_t)data;
      uint32_t j;
      for (j = 4 ; j < outOffset ; j+=2) {
        if (GET_BYTE(otherBase+j) == variable) 
          break;
      }
      if (j == outOffset) {
        // we don't yet have a value for this variable
        if (writeHalfWord(otherBase+outOffset,data))
          outOffset += 2;
        else
          err = true;
      }
    }
  }

  if (!erasePage(pageBases[currentPage]))
    err = true;
  currentPage = 1-currentPage;

  return !err;
}


static void EEPROM8_reset(void) {
  if (erasePage(EEPROM_PAGE0_BASE) && erasePage(EEPROM_PAGE1_BASE)) {
    currentPage = 0;
    invalid = false;
  }
  else {
    invalid = true;
  }
}

void EEPROM8_init(void) {
  if (EEPROM8_MAGIC != GET_WORD(EEPROM_PAGE0_BASE) && ! erasePage(EEPROM_PAGE0_BASE) ) {
    invalid = true;
    return;
  }
  if (EEPROM8_MAGIC != GET_WORD(EEPROM_PAGE1_BASE) && ! erasePage(EEPROM_PAGE1_BASE) ) {
    invalid = true;
    return;
  }
  if (GET_HALF_WORD(EEPROM_PAGE0_BASE+4) != 0xFFFF) {
    currentPage = 0;
  }
  else if (GET_HALF_WORD(EEPROM_PAGE1_BASE+4) != 0xFFFF) {
    currentPage = 1;
  }
  else { // both pages are blank
    currentPage = 1;
  }
  invalid = false;
}
Last edited by arpruss on Thu Nov 16, 2017 12:52 am, edited 1 time in total.

User avatar
RogerClark
Posts: 7418
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: EEPROM emulator for 8-bit data

Post by RogerClark » Wed Nov 15, 2017 7:46 pm

Interesting... Thanks.

Perhaps this should be added to the EEPROM library

victor_pv
Posts: 1734
Joined: Mon Apr 27, 2015 12:12 pm

Re: EEPROM emulator for 8-bit data

Post by victor_pv » Thu Nov 16, 2017 6:11 am

The limitation on not being able to set 1's in a single word after 0s are written is not on the stm32 flash code, but rather flash memory architecture, so no way around that.
About the 16bit writes, is again a silicon limitation in the STM32 flash, the hardware works that way, so again the code can not do anything about it.
To overcome the limit of being able to store only upto 255 variables you could use extra pages of flash, if your program doesn't use it all.
Normally you use 2 pages per set, so one page can hold the data when the other is being cleared.
You could use 2 more pages, to store variables from 255 to 512. You will waste 4 pages of flash, butt if you can spare them, and need to store up to 512 variables, why not?

BTW, thank you for sharing, I was needing to write something to flash and was also thinking the current implementation with 16bit pointer wastes a lot of flash if only a few values need to be stored. With your code the likelyhood of wearing out the flash is halved.

User avatar
RogerClark
Posts: 7418
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: EEPROM emulator for 8-bit data

Post by RogerClark » Thu Nov 16, 2017 6:52 am

Victor,

OK.

I know that some systems can set (or possibly reset ) individual bits.

I thought it was the way that the Flash was erased, so that all cells (bits) got put into a specific state, e.g logic 1, and that write a byte (or 16 or 32 bits), effectively just changed the bit state of the bits that were not the same as the erased state.

I can't remember which way around it is, but thought it was probably logic 1 after it was erased.

So potentially only the bits that were written with a logic 0 had their state changed.

So at a later time, any bits still at logic 1 could be set to logic 0, but not vice versa


However, it sounds like the STM Flash logic does something even to the freshly erased logic 1 bits, that makes them into a different sort of 1 that cant be changed to a 0. ????

victor_pv
Posts: 1734
Joined: Mon Apr 27, 2015 12:12 pm

Re: EEPROM emulator for 8-bit data

Post by victor_pv » Thu Nov 16, 2017 1:45 pm

RogerClark wrote:
Thu Nov 16, 2017 6:52 am
However, it sounds like the STM Flash logic does something even to the freshly erased logic 1 bits, that makes them into a different sort of 1 that cant be changed to a 0. ????
I had to recheck the EEPROM emulation library, but it relies in doing that, writting 0 in some bits in words that were written before but still had some 1s.
I think the STM also let's you write a 0 to individual bits that were still 1. It just doesn't allow the opposite, turning a 0 to a 1.

It uses that to mark when the page is empty (default FFFF), receiving data 0hEEEE or has valid data 0h0000:
https://github.com/rogerclarkmelbourne/ ... .h#L38-L40

In this line writes 0hEEEE to the base address:
https://github.com/rogerclarkmelbourne/ ... M.cpp#L266

In this overwrites that with 0h0000 in the same address:
https://github.com/rogerclarkmelbourne/ ... M.cpp#L192

I do not know if other MCUs use some tricks to turn 0s to 1s, that is something I thought was a limitation of flash memory, and the STM can't do.
But writting 0 to bits that were 1 is a valid operation. If you try to write 1 over a 0, then the operation fails.

arpruss
Posts: 83
Joined: Sat Sep 30, 2017 3:34 am

Re: EEPROM emulator for 8-bit data

Post by arpruss » Thu Nov 16, 2017 2:11 pm

It's a standard trick with flash memory that one can update 1s to 0s without erasing, so when I wrote the initial version of the code, I made use of that, so that variable changes that could be implemented by setting some bits to 0 wouldn't need writing to a new location. But the code wasn't working, and I gave up on that optimization once I read that the STM322F10xxx flash memory manual says:
FPEC preliminarily reads the value at the addressed main Flash memory location and checks that it has been
erased. If not, the program operation is skipped and a warning is issued by the PGERR bit in
FLASH_SR register (the only exception to this is when 0x0000 is programmed. In this case,
the location is correctly programmed to 0x0000 and the PGERR bit is not set).
In other words, 0xFFFE -> 0x0000 is OK, but 0xFFFE -> 0xFFF0 is not.

The core EEPROM.cpp code works because it's only changing 0xEEEE -> 0x0000. If the code tried to change 0xEEEE to any non-zero value, it would presumably fail.

In theory, I could optimize my EEPROM8 code for the special case where variable #0 is set to value 0, but that optimization seems to introduce too much complexity for a rare benefit (unless you're often storing a 0 in variable 0).

I don't know why STM did that. It sounds like it was a deliberate decision rather than forced on them by the way flash hardware works.

victor_pv
Posts: 1734
Joined: Mon Apr 27, 2015 12:12 pm

Re: EEPROM emulator for 8-bit data

Post by victor_pv » Thu Nov 16, 2017 6:07 pm

arpruss wrote:
Thu Nov 16, 2017 2:11 pm


In other words, 0xFFFE -> 0x0000 is OK, but 0xFFFE -> 0xFFF0 is not.

The core EEPROM.cpp code works because it's only changing 0xEEEE -> 0x0000. If the code tried to change 0xEEEE to any non-zero value, it would presumably fail.

In theory, I could optimize my EEPROM8 code for the special case where variable #0 is set to value 0, but that optimization seems to introduce too much complexity for a rare benefit (unless you're often storing a 0 in variable 0).

I don't know why STM did that. It sounds like it was a deliberate decision rather than forced on them by the way flash hardware works.
Ahhh, that's interesting. I knew you could write 0, but I didn't know that was an exception only valid for 0x0000. This introduces the possibility though of completely erasing a later value to leave only the previous one. Not worth the trouble, I can't see a use for it.

I think your code should go to the core with the EEPROM library, so one can choose the 8bit or 16bit implementations as needed. I'm likely going to use yours right away.

User avatar
ahull
Posts: 1646
Joined: Mon Apr 27, 2015 11:04 pm
Location: Sunny Scotland
Contact:

Re: EEPROM emulator for 8-bit data

Post by ahull » Thu Nov 16, 2017 7:33 pm

If you only have a few values to store, you could always use the RTC NVRAM bytes (but you would have to add a battery).

I believe the RTCs 10 x 16 bit "backup registers" have a near infinite number of write cycles, and can also be written bytewise as opposed to sector wise.

Code: Select all

// Define the Base address of the RTC  registers (battery backed up CMOS Ram), so we can use them for config of touch screen and other calibration.
// See http://stm32duino.com/viewtopic.php?f=15&t=132&hilit=rtc&start=40 for a more details about the RTC NVRam
// 10x 16 bit registers are available on the STM32F103CXXX more on the higher density device.

#define BKP_REG_BASE (uint32_t *)(0x40006C00 +0x04)
You could mix and match, with data that has to be changed frequently put in the RTC bytes and other non volatile data stored/updated in flash.
- Andy Hull -

User avatar
RogerClark
Posts: 7418
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: EEPROM emulator for 8-bit data

Post by RogerClark » Thu Nov 16, 2017 7:57 pm

The trick with changing 1 -> 0 on a bitwise basis can be quite handy, as it allows you to keep a count of things without needing to erase so often.

e.g. for a single byte..

11111111= 0
11111110 = 1
11111100 = 2
11111000 = 3


The downside of this, is that to read the counter, you have to read multiple bytes from flash until you find one that is not zero.

But for data logging, where most if the time you are writing and not reading, its a useful trick

Its generally used as part if flash management to determine the "active" bank in flash, that contains the actual data that is being recorded.

Post Reply