How can we reduce the excessive RAM use by this core

The official STMicroelectronics Arduino core
User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sat Nov 19, 2016 10:51 am

BTW. I have 2 issues open about various problems with RAM usage

https://github.com/stm32duino/Arduino_C ... /issues/27

https://github.com/stm32duino/Arduino_C ... /issues/23

@danieleff has pointed out some other problems (in another issue) which would be fixed if we made changes to improve RAM usage



Re:dacInstance

It doesnt actually appeared to be used anywhere, probably as the Nucleo F103RB does not have a DAC

We should move it to its own array like I did for timHandle but that array is only needed for F103RC or better

And....

Reuse of pinDescription struct (in RAM) in many places where 90% of the struct should be static or not there at all, also needs to be fixed ;-(

User avatar
GrumpyOldPizza
Posts: 166
Joined: Fri Apr 15, 2016 4:15 pm
Location: Denver, CO

Re: How can we reduce the excessive RAM use by this core

Postby GrumpyOldPizza » Sat Nov 19, 2016 6:33 pm

Just quickly tried out the BareMinimum.ino on the NUCLEO-L432KC with the STM32L4 core (not the STM Core). This is what I am getting:

Code: Select all

   text      data       bss       dec       hex   filename
   5872         8       576      6456      1938   BareMinimum.ino.elf


Guess that could be optimized, as some RTC bss space ended up there, as well as some EXTI bss space. The rest of the space is taken by the SysTick/PendSV queue.

So 9k for the STM core without any SerialUSB seems excessive.

User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sat Nov 19, 2016 6:44 pm

The problem seems to be the statically allocated arrays to handle all possible PWM pins running etc.

Normally I know malloc() is not generally used, due to memory fragmentation, but I think because of the way the HAL needs large data structs, that perhaps the way to handle the PWM, at least, is to only malloc the timHandle structs for each pin as its required.
But don't call free() pin pwm_stop() , because in some circumstances it could result in something resembling a memory leak.

It would be no worse than the current code, and would mean if you are not using PWM it would save over 1k.

I have not looked at the other memory hogs, but hopefully something similar could be done with them.

User avatar
Rick Kimball
Posts: 728
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Rick Kimball » Sat Nov 19, 2016 7:26 pm

The way the arduino IDE calculates RAM usage and the way the linker script attempts to warn you about the low memory usage are in conflict. There are really 1536 bytes of RAM that the linker "allocates" to the heap and the stack. However, the linker's approach doesn't really reflect how much heap or RAM is used. The arduino IDE knows how much RAM a board/chip has and will report low memory issues with a warning about stability. However, neither approach is accurate.

You could probably change the linker script part here from:

Code: Select all

   /* User_heap_stack section, used to check that there is enough RAM left */
    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
      . = . + _Min_Heap_Size;
      . = . + _Min_Stack_Size;
      . = ALIGN(4);
    } >RAM

to:

Code: Select all

    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
    } >RAM


This would stop the reporting of a bogus RAM memory usage and in our case, the arduino IDE will start complaining if you are getting too close to the edge.

-rick
Last edited by Rick Kimball on Sat Nov 19, 2016 8:13 pm, edited 1 time in total.
-rick

User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sat Nov 19, 2016 7:46 pm

Thanks Rick,

The linker stuff is outside my knowledge base ;-)

User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sat Nov 19, 2016 10:02 pm

I've updated the WIP branch to only malloc the timHandle when its actually used.

i.e I have an array of pointers, which should get initialised to 0 (NULL)

Code: Select all

TIM_HandleTypeDef *g_analog_timer_config[NB_ANALOG_CHANNELS]={};


Then in pwm_start() I'm doing

Code: Select all

  if (g_analog_timer_config[id]==NULL)
  {
     g_analog_timer_config[id]=malloc(sizeof(TIM_HandleTypeDef));
  }


I tested on my Nucleo F103RB using the Arduino Fading example, and it seems to work OK

Edit.

I think I spoke too soon. Something odd is happen - don't use the WIP version at the moment until I can figure out whats wrong.

Edit 2

(arrggghh) fixed it.

Its the same issue I had before, where the timHandle memory is not cleared to begin with.

So I've added

Code: Select all

      g_analog_timer_config[id]->State = HAL_TIM_STATE_RESET;


Which has fixed this.

I also moved some code so it doesn't re-init things it doesn't need to (assuming the HAL doesn't mess with the Init part of the struct that you pass it ) - and it does seem to work OK on the Nucleo

User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sun Nov 20, 2016 3:01 am

Rick Kimball wrote:The way the arduino IDE calculates RAM usage and the way the linker script attempts to warn you about the low memory usage are in conflict. There are really 1536 bytes of RAM that the linker "allocates" to the heap and the stack. However, the linker's approach doesn't really reflect how much heap or RAM is used. The arduino IDE knows how much RAM a board/chip has and will report low memory issues with a warning about stability. However, neither approach is accurate.

You could probably change the linker script part here from:

Code: Select all

   /* User_heap_stack section, used to check that there is enough RAM left */
    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
      . = . + _Min_Heap_Size;
      . = . + _Min_Stack_Size;
      . = ALIGN(4);
    } >RAM

to:

Code: Select all

    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
    } >RAM


This would stop the reporting of a bogus RAM memory usage and in our case, the arduino IDE will start complaining if you are getting too close to the edge.

-rick


Tried those settings and it reduced reported RAM usage by about 2k.

danieleff
Posts: 105
Joined: Thu Sep 01, 2016 8:52 pm
Location: Hungary
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby danieleff » Sun Nov 20, 2016 8:21 am

stevestrong wrote:
RogerClark wrote:In which case this doesn't need to be an array of structs, it needs to be an array of pointers to structs

I agree.
In this case it should be an array of pointers:

Code: Select all

PinDescription * g_intPinConfigured[MAX_DIGITAL_IOS];

The array "g_APinDescription[]" should be then stored in FLASH (having removed the member "configured"), the others like "g_intPinConfigured[]" will be placed in RAM.
One could then make the member "#.configured" equivalent to whether the pointer stored in "g_intPinConfigured[]" is NULL or has been written with a non-zero value.

FYI the g_intPinConfigured.configured is written, but never read.

Given that in the WIP branch pin == arduino_id == index in array, you get delete the whole thing:

Code: Select all

diff --git a/cores/arduino/WInterrupts.c b/cores/arduino/WInterrupts.c
index 1547913..effdb42 100644
--- a/cores/arduino/WInterrupts.c
+++ b/cores/arduino/WInterrupts.c
@@ -22,10 +22,6 @@
  extern "C" {
 #endif
 
-//This is the list of the digital IOs configured
-PinDescription g_intPinConfigured[MAX_DIGITAL_IOS];
-
-
 void attachInterrupt(uint32_t pin, void (*callback)(void), uint32_t mode)
 {
   int i;
@@ -36,15 +32,6 @@ void attachInterrupt(uint32_t pin, void (*callback)(void), uint32_t mode)
     return ;
   }
 
-  //find the pin.
-  for(i = 0; i < NB_PIN_DESCRIPTIONS; i++) {
-    if(g_APinDescription[i].arduino_id == pin) {
-      g_intPinConfigured[pin] = g_APinDescription[i];
-      g_intPinConfigured[pin].configured = true;
-      break;
-    }
-  }
-
   switch(mode) {
 
     case CHANGE :
@@ -64,8 +51,8 @@ void attachInterrupt(uint32_t pin, void (*callback)(void), uint32_t mode)
     break;
   }
 
-  stm32_interrupt_enable(g_intPinConfigured[pin].ulPort,
-                          g_intPinConfigured[pin].ulPin, callback, it_mode);
+  stm32_interrupt_enable(g_APinDescription[pin].ulPort,
+                          g_APinDescription[pin].ulPin, callback, it_mode);
 
 }
 
@@ -78,15 +65,6 @@ void detachInterrupt(uint32_t pin)
     return ;
   }
 
-  //find the pin.
-  for(i = 0; i < NB_PIN_DESCRIPTIONS; i++) {
-    if(g_APinDescription[i].arduino_id == pin) {
-      g_intPinConfigured[pin] = g_APinDescription[i];
-      g_intPinConfigured[pin].configured = true;
-      break;
-    }
-  }
-
-  stm32_interrupt_disable(g_intPinConfigured[pin].ulPort,
-                          g_intPinConfigured[pin].ulPin);
+  stm32_interrupt_disable(g_APinDescription[pin].ulPort,
+                          g_APinDescription[pin].ulPin);
 }


User avatar
RogerClark
Posts: 5467
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby RogerClark » Sun Nov 20, 2016 8:56 pm

stevestrong wrote:Looks very nice :)
In the same context, I see huge RAM saving potential in USB stuff, like reduction of data size uin32 to uint8 by features of USB_CfgTypeDef.
Reducing the number of endpoints from 15 down to 7 would be also not bad, I don't think that all endpoints will be used simultaneously in any application.


With the USB. I hadn't realised that was in the HAL code.

I'm trying to limit the changes to the HAL.

I'll add this as an issue to github so we have a record of it

User avatar
Wi6Labs
Posts: 16
Joined: Fri Sep 16, 2016 11:39 am
Location: Rennes, France
Contact:

Re: How can we reduce the excessive RAM use by this core

Postby Wi6Labs » Wed Nov 23, 2016 11:18 am

RogerClark wrote:
Rick Kimball wrote:The way the arduino IDE calculates RAM usage and the way the linker script attempts to warn you about the low memory usage are in conflict. There are really 1536 bytes of RAM that the linker "allocates" to the heap and the stack. However, the linker's approach doesn't really reflect how much heap or RAM is used. The arduino IDE knows how much RAM a board/chip has and will report low memory issues with a warning about stability. However, neither approach is accurate.

You could probably change the linker script part here from:

Code: Select all

   /* User_heap_stack section, used to check that there is enough RAM left */
    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
      . = . + _Min_Heap_Size;
      . = . + _Min_Stack_Size;
      . = ALIGN(4);
    } >RAM

to:

Code: Select all

    ._user_heap_stack :
    {
      . = ALIGN(4);
      PROVIDE ( end = . );
      PROVIDE ( _end = . );
    } >RAM


This would stop the reporting of a bogus RAM memory usage and in our case, the arduino IDE will start complaining if you are getting too close to the edge.

-rick


Tried those settings and it reduced reported RAM usage by about 2k.


We don't agree with this modification. Indeed it seems to reduce the RAM usage return by Arduino but it is wrong. Stack and heap are like memory allocation. The system reserve a part of the RAM for a subsequent usage that you can't control. Maybe we make a mistake but we think Arduino can't say you how much stack and heap will be used.
Stack and heap should be removed from the free RAM.

But if you want finally keep this modification so you should say to Arduino to check the RAM usage not from the 20k but from 20k-(stack+heap).
Wi6Labs team


Return to “STM Core”

Who is online

Users browsing this forum: No registered users and 1 guest