Question or suggestion, upload to Flash, then copy to RAM at runtime.

STM32duino bootloader aka Maple bootloader
victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by victor_pv » Mon May 18, 2015 7:34 pm

Roger, the latest commit in my bootloader repo already included the change. Works like this:

-IDE compiled with RAM linker script (so uses RAM addresses), but uploads to ID2 (so normal upload to flash on 8002000).
Bootloader loads the code like any other ID2 to flash.
But, when the bootloader reboots and needs to check what's in flash and RAM to run it checks like this:
1.-First check if there is a valid SP address in 20000c00. That would indicate an upload to RAM, like the original bootloader used to do. If valid SP, then runs from RAM.
2.-Next checks if 8002004 (which is the reset vector position for a sketch uploaded to 8002000), and if the reset vector points to an address that matches 2000xxxx, it means that is a reset vector to a RAM address, not to flash. In that case, it copies 64KB of flash starting on 8002000 to RAM starting at 20000c00, and next boots from there like in step 1 above when the upload was direct to RAM.
3-Next it checks if there is a valid SP in 8002000 (like yours do with ID2), and if so boot to flash in that address.
4-Finally checks valid SP in 8008000 like the old ID1, and if valid jumps to there.

If you reboot the board, Step 1 finds the code in RAM and runs from there directly, only needs to copy from flash to RAM if you completely shutdown.

So step 2 above is the new added feature. I run a sketch with some calculations, and runs faster, but I think it mostly depend on how much jumping around the code needs to do. Flash is fast enough in sequential with the prefetcher, but as we saw with PIN_MAP, it can be slow for other things.

I don't know if this option is needed at all. I just did for the fun of it. It ocurred to me in the way home the other day, and gave it shot, and works fine. Took me may be half an hour to code and test it.
The extra to acomplish step 2 above adds about 200 bytes to the bootloader code.

It's fully functional. The whole idea may come handy for updating the bootloader or a sketch from itself. First copy the "updating" section to a known RAM address, then execute from there, and once running in RAM, it can rewrite the whole flash.
I do not see any immediate use right now though, as I said it was just mostly to test if I could do it.

User avatar
RogerClark
Posts: 7150
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by RogerClark » Mon May 18, 2015 8:36 pm

Thanks victor

I agree its interesting to know its possible.

As long as we don't have false positives e.g bootloader thinks there is a program in ram when there isn't, I can't see any harm in retaining it.

The version where it copies from flash back to ram seems the most fool proof because, I think you are saying that the bootloader is determining the start address of the code by looking at the code in flash and then copying if from flash to ram.

This sounds stable to me if we can guarantee that there can't be false positives.

I think the thing with the bootloader is that it needs to be a lot more stable then the core.
Because is more of a pain to reflash the bootloader than to just recompile the core and upload (using the bootloader)
Which is why I dropped the upload straight to ram.

Perhaps some of the other clever people on the site can let us know if they think there are any ways in which the upload to flash and copy and run from ram could go wrong.

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by victor_pv » Mon May 18, 2015 10:06 pm

Roger, I dont think the can be a false positive in any working code developed without heavily modifying the linker scripts.
In any code linked for flash, the reset vector has to be in an 800xxxx address. If the reset vector stored at 8002000 matches the pattern for 200xxxxxx it can not match the pattern for 800xxxx at the same time.

Regarding the other check, the one that checks for a valid SP address at the 2000C00 address, I made the upload to RAM option check for an SP that matches at least 3 or 4 zeros at the end. That prevent any false possitive in the chips with 20KB, but could allow a false positive in a chip with more than 20KB if the word at ...C00 contains a pointer to another address in the RAM range higher than the initial 20KB, and that pointer address ends in 3 or 4 zeros.

To prevent that we could make that mask a #define line in the config.h file. so the mask matches the amount of RAM in the MCU. so for 20KB, it would check the SP points to the next by after 20KB, but for high density devices it could check that the SP points to the address right after the 64KB of RAM on those.
I still think that the chances of the RAM at C00 containing a pointer that fits the mask I am using are pretty slim.
I can check the code when I get home and tell you how many bits I was using.

User avatar
RogerClark
Posts: 7150
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by RogerClark » Mon May 18, 2015 11:05 pm

Hi Victor,

Can we wait for feedback from Ray and Matthias etc, as I know some people have strong views on whether RAM upload is a safe or good idea

madias
Posts: 813
Joined: Mon Apr 27, 2015 11:26 am
Location: Vienna, Austria

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by madias » Tue May 19, 2015 8:18 am

Ok, call me uncreative, but I was thinking a lot what benefit we get out of this feature. So "auto updating" the bootloader is the only scenario or we can get some speed improvements in general?
But anyway an interesting thing!

User avatar
RogerClark
Posts: 7150
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by RogerClark » Tue May 19, 2015 10:11 am

Hi Matthias,

I'm not even sure the RAM thing helps updating the bootloader
We can already update the bootloader via a sketch (Victor already did this), so I'm not really sure how the ram stuff helps any more with that.

But I guess its the art of the possible.

What concerns me is whether we'd get false positives. For some reason with the old Maple bootloader I seemed to get false positives on my Maple Rev 3.

And I'm generally cautious in terms of this sort of thing, so my choice would be not to include RAM in the normal version, but perhaps have it in a special version

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by victor_pv » Tue May 19, 2015 2:23 pm

Roger, the upload to RAM option was giving false possitives because of the way leaflabs was testing for a valid SP address.
Basically it tested that the SP vector at C00 contained ANY valid RAM address. That means that when any sketch put any pointer in that location, it would give a false positive.
They used this:
if ((sp & 0x2FFE0000) == 0x20000000) {

That means that any address matching 20000000 to 20010000 , gives a positive. That is a the whole 64KB of maximum RAM available in any R or C series chip.
So any pointer whatsoever stored in C00, gives a positive. When we starting using the whole 20KB of RAM for sketches, some pointers ended up in that area.

Now I test it like this:
if ((sp & 0xFFFE0FFF) == 0x20000000)
That means that only addresses that are a multiple of 4KB would match the pattern (4KB, 8KB, 12KB, etc):
20001000
20002000
20003000
20004000

For a 20KB device only 4 addresses could give a false positive, 4, 8, 12, and 16. The real positive would be 20Kb.

For a 64KB device there are a few more false possitve, 15 total:
20005000 (on the 20KB this is the real positive for the SP).
20006000
...

Now, the chances of a pointer being in that location (20000C00) AND that pointer value being one of the 3 possible false positives, I believe are extremely small. If that ever happens, a power down and power up clears the RAM.

BUT, to prevent even more false positives, as we now are using different targets for the Make file, we can define that mask in config.h for each target, such as

if ((sp == SP_MASK) {

Where SP_MASK = 0x20005000 for themaple mini (20KB +1, the only SP vector we use in any of the compiler scripts)
And for the 64KB devices:
SP_MASK = 0x20010000 (the byte after 64KB).

EDIT: With all the above said, on the 20KB devices we should probably take the option out all together. I only included it back in the code I was using for testing the devices with 2KB page size, which have 48 or 64KB at least, but if keeping the RAM upload for all devices is simpler than making it conditional for only some devices, then the method above should solve the issue with false positives.

User avatar
mrburnette
Posts: 1829
Joined: Mon Apr 27, 2015 12:50 pm
Location: Greater Atlanta
Contact:

Re: Question or suggestion, upload to Flash, then copy to RAM at runtime.

Post by mrburnette » Tue May 19, 2015 3:06 pm

EDIT: With all the above said, on the 20KB devices we should probably take the option out all together.
2nd that.

Ray

Post Reply