[SOLVED] STM32F103ZE 64KB Memory depleted by String class array

Generic boards that are not Maple or Maple mini clones, and don't contain the additional USB reset hardware
victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Sat Sep 02, 2017 10:26 pm

Rick, the constructor for the array used in the example the OP posted (String ramtest[xxx]) allocates the buffer memory right when the constructor is called. Not sure if it's just because it's an array, but that's what it does, so each string in the array takes 12bytes in BSS, and 16 more bytes in the heap, while still being empty.

Is that what you expected?

User avatar
Rick Kimball
Posts: 1038
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by Rick Kimball » Sat Sep 02, 2017 10:55 pm

That isn't what I observed in the debugger.
-rick

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Sun Sep 03, 2017 2:17 pm

Can you post the .map file from your code? that will help see where is the memory mostly going, and also to see if it's pulling things from stdlib.
You can also post the sections where you use Strings, as you have seen above String is a memory hog, so perhaps there perhaps there is a different way to achieve what you want to do in a different way that will not take all that mem.
If you can't share those parts, then it's impossible for us to know what's going on. As I proved with your example, the heap can get depleted very easily.

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Sun Sep 03, 2017 2:25 pm

Rick Kimball wrote:
Sat Sep 02, 2017 10:55 pm
That isn't what I observed in the debugger.
Did your test deplete heap like it did in mine? if so when did allocate for heap and for what?
If it did't deplete heap like in my case, did it crash at all?

User avatar
Rick Kimball
Posts: 1038
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by Rick Kimball » Sun Sep 03, 2017 4:50 pm

It does appear that it allocates an empty String.buffer character array when the String array is initialized.

When I was debugging, I noticed the String.buffer variable being NULL. However, by the time you get to the setup() function, String.buffer does have a valid heap allocated value. The minimum allocation seems to be 16 bytes. So yes, it does reserve the .bss section space and heap memory allocation before setup() even runs.

Sorry for the confusion.

I see that the default constructor for String defaults to the String(const char *cstr = "") version:

Code: Select all

(gdb) p my_array 
$3 = {{buffer = 0x20000c60 "", capacity = 0, len = 0}, {buffer = 0x0, capacity = 0, len = 0}}
(gdb) where
#0  String::String (this=0x20000b44 <my_array>, cstr=0x8002e32 "") at /home/kimballr/Arduino/hardware/stm32duino/STM32F1/cores/maple/WString.cpp:34
#1  0x080001ba in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/kimballr/Arduino/sketch_sep03a/sketch_sep03a.ino:1
#2  _GLOBAL__sub_I_my_array () at /home/kimballr/Arduino/sketch_sep03a/sketch_sep03a.ino:10
#3  0x08001216 in __libc_init_array ()
#4  0x08000214 in start_c () at /home/kimballr/Arduino/hardware/stm32duino/STM32F1/variants/generic_stm32f103c/wirish/start_c.c:84
#5  0xfffffffe in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
-rick

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Tue Sep 05, 2017 9:21 pm

gasparobr wrote:
Fri Sep 01, 2017 8:58 pm
The code FREEZE without reason if I add any String or use Serial.print(); or Serial.print(F()), with just 23k of memory ALOCATED. I use a lot of SERIAL PROCESSOR in that code. That is the cause I did that sample code to test the RAM.

I did other test now with other sample and I can just use 23k of the RAM with String, but if I use other arrays of FLOAT and DOUBLE, I can use the full 64k.
As you saw String uses much more ram than the statically allocated at first, and depletes heap. Saying that you can only use 23KB with String is not correct, you are allocating much more, 23KB at compile time and the rest to complete 64KB at run time.
You should check if that is the case with your code, and you are depleting the heap or the stack.
I will update the title of this thread until you can provide some other test piece of code, since the one in the original post does not show any problem with memory allocation in the MCU or the core, but rather shows that creating an array of thousands of String class objects takes a lot of memory at run time.

User avatar
RogerClark
Posts: 7156
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: [SOLVED] STM32F103ZE 64KB Memory depleted by String class array

Post by RogerClark » Tue Sep 05, 2017 9:42 pm

Victor

One thing I have noticed is that malloc() does not seem to fail gracefully.
Its supposed to return null if it cant allocate the memory, but in practice if you keep allocating 8 bytes, eventually the code will crash.

I observed the same behaviour with the Nordic nRF51, which is also ARM based , when I compiled using GCC.

I am not sure if malloc() is the problem, or whether the rest of the core needs a small amount of heap to continue running.

I think I looked though the libmaple core a while ago, to see if malloc was used , and from what I recall, it does not use malloc.
So i presume somehow malloc its self is causing the crash, though I am not sure how that is possible, unless perhaps it allocates memory that is in use by the stack

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: [SOLVED] STM32F103ZE 64KB Memory depleted by String class array

Post by victor_pv » Wed Sep 06, 2017 6:53 pm

I don't think the core use malloc or new either.
In fact I'm sure because when we add that in a sketch, we see the extra code being pulled in.

malloc() it's supposed to call new() if I remember right, and I believe new() does not check where the current stack is, neither it has a hard limit for heap other than the top of RAM (if I remember the code right).
So in that case, new () would continue allocating memory until causing a stack corruption and the core crashing, since no matter how small the stack is, it's definitely going to be lower than the top of RAM at any given time.

I have seen linker scripts that place a minimum reservation for the stack, or a maximum size for the heap. In either case as long as the stack has not gone lower than that address, it's all good, but there is always the chance stack grows larger than our estimate and runs into the heap. We could do that, would be a set of small changes to the linker scripts. We just need to decide what's the minimum space we allocate for the stack, either for all MCUs, or per series.

I dont remember of the top of my head which cores or which linker scripts use that technique but we can look at SAM and Teensy for reference.

Only problem I see is that even if we return an error (or NULL) when the heap reaches a limit, that will happen at run time and most likely just crash the board, exactly as in the example in the first post. Also once something has been running for hours or days, the stack may be going lower, and the heap keeps going up... So it helps identify the problem if the system crashes right at boot up, but I don't think it totally solves it since the stack size is variable.
Even if new() checked where the stack pointer is right when it's called, there is no guarantee that the stack pointer won't go lower once new() returns and a few other functions are called.

This is a good reading on the String class issues:
http://forum.arduino.cc/index.php?topic=115552.0

And there are pointers there to Bill Greyman to how using Heap is completely forbidden in some critical embedded systems:
http://forum.arduino.cc/index.php?topic ... #msg869721

Perhaps Rick has some advice on whether there is any value in reserving some stack space.

Post Reply