[SOLVED] STM32F103ZE 64KB Memory depleted by String class array

Generic boards that are not Maple or Maple mini clones, and don't contain the additional USB reset hardware
gasparobr
Posts: 7
Joined: Tue Jan 17, 2017 4:15 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by gasparobr » Thu Aug 31, 2017 2:26 pm

Thanks Victor for your answer.

My main code is crashing with using more than 20k. So I just did this simple test to check what whas happening with my RAM, without any Stack. My processor have 64k and I really need to use it for local variables and Serial Buffers. There is any way that can I use it all?
victor_pv wrote:
Thu Aug 31, 2017 2:49 am
Not sure why you would need so many strings at the same time, but well that's off topic.

It's hard to say why your sketch is crashing, but besides the memory you assign for variables, there is also the Stack and the Heap in RAM.
They grow towards each other, so you can run in a situation where the Stack overwrites variables in the Heap, or the Heap overwrites data in the stack. Either case causes a corruption of memory and likely a crash.
I can't say how much memory you are using in either, but it you use a lot of "new" or "malloc" to alocate space for variables, that will come from the heap and can end up crashing with the stack.

On the other hand, if you have some code that nests functions calls a lot before unwinding it all, that will grow the stack a lot. Also if you declare objects that take a lot of ram, like those strings, as local, they take a lot of memory from the stack.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Thu Aug 31, 2017 2:39 pm

Does the code in the first post crash? if so, we can use that as a test.
I haven't had time to run it yet. If that doesn't crash, then there may be something else in your code.
In normal conditions you should be able to use all 64KB, but there could be a bug in the core, depending how exactly the memory is going to be used.
If you have a debugger, stlink or any other, it would be great if you can check where exactly is crashing. That's when I plan to do when I have time to run a test.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Fri Sep 01, 2017 12:45 am

I just ran it in a 103RET6 MCU and the problem is in the String class.
The String class constructor allocates 16 bytes of memory per string in the heap, in adition to the other 12bytes it allocates in global space, in BSS.

When the string class constructor is called, with:
String ramtest[nnn]

What it does it copy a string object nnn times (it loops nnn times thru the constructor doing copies) and each of those times it allocates memory from the heap.
Eventually the heap reaches the stack and it crash the MCU.

The ramtest[] object only contains pointers, 12 bytes total, and one of the pointers is to the ram address in the heap where it allocates a buffer, I guess to write the actual string of text you are supposed to use this for, and I can see how each of those String objects contain a pointer 16 bytes ahead of the previous one, so it's allocating 16 bytes of heap for each String object it initializes. The last pointer it writes in the ramtest array before it crashes in my case is in 2000FFA0, and the stack pointer is at that same address, so the heap has crashed the stack.

So if you want to use the String class, you need to understand it will allocate much more than just 12 bytes per string, and that's with empty strings, I can imagine if this did not crash and you wanted to then give a string object a content that's let's say 100 bytes long, it will have to call the copy constructor again and allocate another 100bytes from the heap.
I do not know anything about the string class other than what I saw now in the bdebugger, so unless someone else in the forum can give you some advice, your best bet is to post in the Arduino forums, or the Teensy forum, since it looks like Paul Stoffregen rewrotte the whole String class.

I think this brings us up to my initial question, that now is not off topic any more, but why do you want to create thousands of String objects?
It seems to me that even if you got a different MCU that could fit all those, as soon as you start modifying Strings and they have to be copied again to get more space, you would deplete the memory of any MCU.

Good link about what's happening to your:
https://learn.adafruit.com/memories-of- ... izing-sram

User avatar
RogerClark
Posts: 6675
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by RogerClark » Fri Sep 01, 2017 1:43 am

Victor

Thanks for the info on the String class.

But, I suspect there may still be a problem with LibMaple

Looking at the addresses in RAM which it assigned to the Strings in global scope, (and also things like Serial and Serial1 etc)

The addresses its using are around 5k from the base of RAM, and from the map file I can't see very much using the space below the 5k region

STM32GENERIC seems to allocate global vars much lower to the start of RAM (within a few hundred bytes).
But their memory organisation may be totally different.

Unfortunately I don't know enough about the linker directives to understand why .bss seems to be 5k from the base of RAM
I assume its reserving that space for something else, but I'm not sure what.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Fri Sep 01, 2017 3:58 am

In my case bss starts at dc0 (3520)

Before that all looks normal, but there is a big chunk related to libc impure_data. I believe there was a longer discussion about that a while back in the forum, and it's due to using malloc. Since the string class uses malloc, it's pulling all this in. I dont think there is a problem with libmaple, but there is a problem with String for sure. Also there is some interesting posts from Paul in the Arduino forum explaining how the Arduino team refused to implement most of his fixes for that class and picked only some of them. Some of those threads are old, and more changes may have been brought over, but still may be convenient to look at how is the String class for the Teensy3, since Paul apparently had devoted a lot of time on corrections the Arduino team did not want to merge, so he applied those fixes to the teensy cores.

Code: Select all

 *fill*         0x2000057c        0x4 
 .data.impure_data
                0x20000580      0x428 c:/sloeber/arduinoplugin/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/lib/armv7-m\libc.a(lib_a-impure.o)
 .data._impure_ptr
                0x200009a8        0x4 c:/sloeber/arduinoplugin/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/lib/armv7-m\libc.a(lib_a-impure.o)
                0x200009a8                _impure_ptr
 .data.__malloc_av_
                0x200009ac      0x408 c:/sloeber/arduinoplugin/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/lib/armv7-m\libc.a(lib_a-mallocr.o)
                0x200009ac                __malloc_av_
 .data.__malloc_trim_threshold
                0x20000db4        0x4 c:/sloeber/arduinoplugin/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/lib/armv7-m\libc.a(lib_a-mallocr.o)
                0x20000db4                __malloc_trim_threshold
 .data.__malloc_sbrk_base
                0x20000db8        0x4 c:/sloeber/arduinoplugin/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/lib/armv7-m\libc.a(lib_a-mallocr.o)
                0x20000db8                __malloc_sbrk_base
                0x20000dc0                . = ALIGN (0x8)
 *fill*         0x20000dbc        0x4 
                0x20000dc0                __data_end__ = .

User avatar
RogerClark
Posts: 6675
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by RogerClark » Fri Sep 01, 2017 4:32 am

Victor

Thanks...

On the Maple mini .bss seems to be df0 which is exactly 3k from the start of RAM, which makes me think there is a setting somewhere which controls this

I agree we can't really change this, as the RAM below that area will be in use for other things.

At the moment I think we have bigger fish to fry than optimising Strings to make better use of memory.

IMHO the String class should be avoided for embedded development. I hardly ever use it with the STM32, however I noticed the ESP8266 core makes extensive use of Strings (but they probably have more RAM to play with)

stevestrong
Posts: 1502
Joined: Mon Oct 19, 2015 12:06 am
Location: Munich, Germany

Re: STM32F103ZE 64KB Memory Handle Error

Post by stevestrong » Fri Sep 01, 2017 9:31 am

Can't we just look at Teensy's String class and take over some parts (all?) of it?

User avatar
RogerClark
Posts: 6675
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by RogerClark » Fri Sep 01, 2017 10:10 am

stevestrong wrote:
Fri Sep 01, 2017 9:31 am
Can't we just look at Teensy's String class and take over some parts (all?) of it?
I don't have time to do it.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: STM32F103ZE 64KB Memory Handle Error

Post by victor_pv » Fri Sep 01, 2017 1:50 pm

RogerClark wrote:
Fri Sep 01, 2017 4:32 am
IMHO the String class should be avoided for embedded development. I hardly ever use it with the STM32, however I noticed the ESP8266 core makes extensive use of Strings (but they probably have more RAM to play with)
+1

I dont use String either, I have already spent enough time to find if there was a bug in the core and there is not. If someone is using String and has the time to compare to teensy etc, then submit a PR, we can revisit.

User avatar
Rick Kimball
Posts: 989
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: STM32F103ZE 64KB Memory Handle Error

Post by Rick Kimball » Fri Sep 01, 2017 4:51 pm

Sometimes I wonder if people come in here and post questions just to troll us?

The original poster doesn't seem to be showing us the code that he is really running. So for us to try and figure out what he is doing is just a waste of time. I can't really imagine trying to use an array of Strings the way he has it coded. I'm guessing he meant to allocate one String instance that is 2170 bytes long.
-rick

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest