How can we reduce the excessive RAM use by this core

The official STMicroelectronics Arduino core
danieleff
Posts: 216
Joined: Thu Sep 01, 2016 8:52 pm
Location: Hungary
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby danieleff » Wed Nov 23, 2016 12:38 pm

It depents if one wants to have a warning or an error when there is too low memory left for stack+heap.
Arduino does the warning (at 75% ram), and _user_heap_stack reserves ram making it an error.

So currently (without the modification) both are in effect.

*EDIT: My proposal is to consider stack as essential, all programs use it, so reserve it (Leave ". = . + _Min_Stack_Size;" inside "._user_heap_stack :"), while heap as non essential, a most programs will not use it, and leave it to arduino warning (Remove ". = . + _Min_Heap_Size;" from "._user_heap_stack :")

You can kind of sort of guess stack size for every user, but not heap size.

User avatar
Rick Kimball
Posts: 776
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Rick Kimball » Wed Nov 23, 2016 2:09 pm

Like I mentioned in my earlier post, neither method is actually accurate. By using arbitrary values in the ld script you are just making assumptions about how much heap and stack the arduino might actually use. However, letting the arduino ide warn you with its arbitrary low water mark isn't any better. The arduino seems to use a build engine parameter of 75% of ram (build.warn_data_percentage = 75). So for the 20k stm32f103c8 it will give you a stability warning when you have less than 5120 bytes left. That is even more conservative than the 1536 bytes reserved by the ldscript.

For myself, I'd rather know how much memory is actually used by the .bss and .data sections without having to guess what the .ldscript is holding in reserve.

-rick
Last edited by Rick Kimball on Thu Nov 24, 2016 12:37 pm, edited 1 time in total.
-rick

User avatar
GrumpyOldPizza
Posts: 173
Joined: Fri Apr 15, 2016 4:15 pm
Location: Denver, CO

Re: How can we reduce the excessive RAM use by this core

Postby GrumpyOldPizza » Wed Nov 23, 2016 2:51 pm

Rick Kimball wrote:Like I mentioned in my earlier post, neither method is actually accurate. By using arbitrary values in the ld script you are just making assumptions about how much heap and stack the arduino might actually use. However, letting the arduino ide warn you with its arbitrary low water mark isn't any better. The arduino seems to use a build engine parameter of 75% of ram (build.warn_data_percentage = 75). So for the 20k stm32f103c8 it will give you a stability warning when you have less than 5120 bytes left. That is even more conservative than the 1536 bytes reserved by the ldscript.

For myself, I'd rather now how much memory is actually used by the .bss and .data sections without having to guess what the .ldscript is holding in reserve.

-rick


It might be interesting to use a similar scheme that ARM uses for CMSIS (startup_CM4.S and gcc_arm.ld below). Essentially you can define __HEAP_SIZE as compile time argument (and __STACK_SIZE). If you overflow the minimum requirement (after all .data/.bss data has been allocated), the linker will throw an error.

[code]
.section .heap
.align 3
#ifdef __HEAP_SIZE
.equ Heap_Size, __HEAP_SIZE
#else
.equ Heap_Size, 0x00000C00
#endif
.globl __HeapBase
.globl __HeapLimit
__HeapBase:
.if Heap_Size
.space Heap_Size
.endif
.size __HeapBase, . - __HeapBase
__HeapLimit:
.size __HeapLimit, . - __HeapLimit
[\code]

[code]
.heap (COPY):
{
__HeapBase = .;
__end__ = .;
end = __end__;
KEEP(*(.heap*))
__HeapLimit = .;
} > RAM

/* .stack_dummy section doesn't contains any symbols. It is only
* used for linker to calculate size of stack sections, and assign
* values to stack symbols later */
.stack_dummy (COPY):
{
KEEP(*(.stack*))
} > RAM

/* Set stack top to end of RAM, and stack limit move down by
* size of stack_dummy section */
__StackTop = ORIGIN(RAM) + LENGTH(RAM);
__StackLimit = __StackTop - SIZEOF(.stack_dummy);
PROVIDE(__stack = __StackTop);

/* Check if data + heap + stack exceeds RAM limit */
ASSERT(__StackLimit >= __HeapLimit, "region RAM overflowed with stack")
[\code]

User avatar
Rick Kimball
Posts: 776
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Rick Kimball » Wed Nov 23, 2016 3:40 pm

GrumpyOldPizza wrote:[
It might be interesting to use a similar scheme that ARM uses for CMSIS (startup_CM4.S and gcc_arm.ld below). Essentially you can define __HEAP_SIZE as compile time argument (and __STACK_SIZE). If you overflow the minimum requirement (after all .data/.bss data has been allocated), the linker will throw an error.

That is what the current linker script does, the heap size is at the top of the ld scipt (see links below). Your suggestion of exposing those variables to the command line is great. The problem is the Arduino IDE. It really doesn't provide an easy way to change the command line or provide user specified parameters for each script. You have to edit the boards.txt file or add menu items that have fixed values to affect the command line.

The whole reason this issue popped up is because the WI6labs core uses a lot of memory. In trying to use it with the stm32vldiscovery board, which only has 8k of ram, I was forced to go find out where the memory was used so I could get a simple script to compile without failing. This doesn't really matter for most people.

See:
https://github.com/stm32duino/Arduino_C ... ASH.ld#L51
https://github.com/stm32duino/Arduino_C ... SH.ld#L164

-rick
-rick

User avatar
GrumpyOldPizza
Posts: 173
Joined: Fri Apr 15, 2016 4:15 pm
Location: Denver, CO

Re: How can we reduce the excessive RAM use by this core

Postby GrumpyOldPizza » Wed Nov 23, 2016 6:34 pm

Rick Kimball wrote:
GrumpyOldPizza wrote:[
It might be interesting to use a similar scheme that ARM uses for CMSIS (startup_CM4.S and gcc_arm.ld below). Essentially you can define __HEAP_SIZE as compile time argument (and __STACK_SIZE). If you overflow the minimum requirement (after all .data/.bss data has been allocated), the linker will throw an error.

That is what the current linker script does, the heap size is at the top of the ld scipt (see links below). Your suggestion of exposing those variables to the command line is great. The problem is the Arduino IDE. It really doesn't provide an easy way to change the command line or provide user specified parameters for each script. You have to edit the boards.txt file or add menu items that have fixed values to affect the command line.

The whole reason this issue popped up is because the WI6labs core uses a lot of memory. In trying to use it with the stm32vldiscovery board, which only has 8k of ram, I was forced to go find out where the memory was used so I could get a simple script to compile without failing. This doesn't really matter for most people.

See:
https://github.com/stm32duino/Arduino_C ... ASH.ld#L51
https://github.com/stm32duino/Arduino_C ... SH.ld#L164

-rick


Yup, saw the link-script. The idea was simply to show the command line option way that CMSIS went. Now one could define a __HEAP_SIZE per board rather than a 75% threshold. If you have 8k RAM available, it makes a big difference, for smaller ones.

User avatar
Rick Kimball
Posts: 776
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Rick Kimball » Wed Nov 23, 2016 6:38 pm

You really want to control this at the sketch level, not the board level.

Say I create a big array on the stack

Code: Select all

void somefunction(void) {
   char mybuffer[8192];

  while (1) {
     .. do somethign with the buffer...
  }
}


None of the methods we are talking about are going to detect I just blew out that stack on a board with 8k of ram.

-rick
-rick

User avatar
RogerClark
Posts: 5564
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Thu Nov 24, 2016 7:11 am

umm.

Looks like I need to revert this change until the dust settles

User avatar
Wi6Labs
Posts: 16
Joined: Fri Sep 16, 2016 11:39 am
Location: Rennes, France
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Wi6Labs » Thu Nov 24, 2016 9:14 am

We prefer to keep a warning base on the real RAM usage for the user. We agree that Arduino should give a solution to modify this level (75% is too low).
Reduce stack and heap under the current value is risky (we have made some tests during the development).
Maybe there isn't any solution for the moment.
Wi6Labs team

User avatar
RogerClark
Posts: 5564
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Thu Nov 24, 2016 10:28 am

I've reverted the change to all the linker files, which I changed in the F1 repo

Note.
All recent changes are only to the WIP branch (Work in progress) and should not effect anyone who downloads the master repo zip


Return to “STM Core”

Who is online

Users browsing this forum: No registered users and 1 guest