Very lean Serial.printf()

Post your cool example code here.
User avatar
Slammer
Posts: 241
Joined: Tue Mar 01, 2016 10:35 pm
Location: Athens, Greece

Very lean Serial.printf()

Postby Slammer » Wed Apr 13, 2016 11:58 pm

I miss very much the old-school printf() function....
I am tired to write again and again Serial.print() commands to display some values. Unfortunately the typical printf function is very big, the libc version can take almost the half or more memory of a F103C8. It is not only the function itself, is the number of functions that the linker attaches when this library is called (floating point, helpers, stdio stuff, etc)
I remember, years ago there were very optimised versions of printf() for use with 8051s taking no more than 2KB. For this reason I opened my old notebooks and I found some versions of these functions. I tried to adopt them in STM32 environment, inside Print class. I tested over 6-8 functions and I end-up with the version of sdcc compiler. I removed all mcu specific stuff and after some changes the code is running on STM32.

What I have until now:
- A printf function that supports integer (long, short, byte), string, character and pointer variables.
- ' ', -, +, b, l, c, s, p, d, i, o, u, and x format specifiers
- No static variables, no dynamic allocation, no globals, just plain functions running with local variables.
- Print.printf member function
- Very small size : 1080 bytes (for the 3 core functions) and 64 bytes for Print.printf() encapsulation.
- Until now I cant rid off the declaration of a temporary buffer inside Print.printf(), but I am sure that the buffer is not needed (WIP...). For this reason the output of the function is limited by the size of this buffer, now is 64 bytes (no test for overflow!!!).
- No floating point but it is possible to include this functionality, I have to examine the impact of this.

I think that code needs some cleanup, but it is a good start.

The code is included in one file called print_format.c, this includes the main function and some helpers.
Two files of our core files must be changed.
First the Print.h for the declaration of the function, Roger has already the function with header guards, so I wrote a new line

Code: Select all

   int printf(const char * format, ...);


I wrote the required call back function and a custom vsprintf(), together with the member function Print.printf() at the end of Print.cpp:

Code: Select all

#include <stdarg.h>
typedef void (*pfn_outputchar)(char c, void* p);
extern "C"{
int _print_format (pfn_outputchar pfn, void* pvoid, const char *format, va_list ap);
}

static void put_char_to_string (char c, void* p)
{
  char **buf = (char **)p;
  *(*buf)++ = c;
}

static int vsprintf (char *buf, const char *format, va_list ap)
{
  int i;
  i = _print_format (put_char_to_string, &buf, format, ap);
  *buf = 0;
  return i;
}

int Print::printf (const char *format, ...)
{
    va_list arg;
    va_start(arg, format);
    char temp[64];
    char* buffer = temp;
    size_t len = vsprintf(temp, format, arg);
    va_end(arg);
    len = write((const uint8_t*) buffer, len);
    return len;
}


The attached print_format.c file must be included in core directory.

For testing I used the all-time classic program of mrburnette, modified for printf()

Code: Select all

#include<Arduino.h>
#define BOARD_LED_PIN PC13          // Maple Mini pin# 33

int n=0;

void setup()
{
    // initialize the digital pin as an output.
    pinMode(BOARD_LED_PIN, OUTPUT);
    Serial.begin();  // BAUD has no effect on USB serial: placeholder for physical UART
    // wait for serial monitor to be connected.
    while (!(Serial.isConnected() && (Serial.getDTR() || Serial.getRTS()))) {
        digitalWrite(BOARD_LED_PIN,!digitalRead(BOARD_LED_PIN));// Turn the LED from off to on, or on to off
        delay(50);         // fast blink
    }
    Serial.printf("-PROGRAM START-\r\n");
}

void loop()
{
    digitalWrite(BOARD_LED_PIN, HIGH);   // set the LED on
    delay(500);              // wait for a second
    digitalWrite(BOARD_LED_PIN, LOW);    // set the LED off
    Serial.printf("Loop #: % 5d %08X %ld %p %f\r\n",n,n,n,&n,n++);
    delay(500);              // wait
}


The output is here:

Code: Select all

-PROGRAM START-                                                                                                     
Loop #:     0 00000000 0 0x200003dc <%f>                                                                           
Loop #:     1 00000001 1 0x200003dc <%f>                                                                           
Loop #:     2 00000002 2 0x200003dc <%f>                                                                           
Loop #:     3 00000003 3 0x200003dc <%f>                                                                           
Loop #:     4 00000004 4 0x200003dc <%f>                                                                           
Loop #:     5 00000005 5 0x200003dc <%f>                                                                           
Loop #:     6 00000006 6 0x200003dc <%f>
Loop #:     7 00000007 7 0x200003dc <%f>
Loop #:     8 00000008 8 0x200003dc <%f>
Loop #:     9 00000009 9 0x200003dc <%f>

(*) prints <%f> in the place of %f specifier as reminder of missing support
Attachments
print_format.c
(12.15 KiB) Downloaded 82 times

User avatar
ahull
Posts: 1397
Joined: Mon Apr 27, 2015 11:04 pm
Location: Sunny Scotland
Contact:

Re: Very lean Serial.printf()

Postby ahull » Thu Apr 14, 2016 11:52 am

That looks much more compact than any of the other printf() variants I have seen in arduinoland. :D Good work.
- Andy Hull -

User avatar
Slammer
Posts: 241
Joined: Tue Mar 01, 2016 10:35 pm
Location: Athens, Greece

Re: Very lean Serial.printf()

Postby Slammer » Thu Apr 14, 2016 12:22 pm

I am thinking some optimizations to reduce the memory more... eg.
- remove of dedicated pointer printing ( it is possible to print pointers as long unsigned integers with integer specifiers)
- direct call back to Print.write functions without buffering (very important for Serial or LCD printing)

I am studying also some algorithms to convert a float point number to string without float functions. This is essential for floats support without the overhead of floating point libraries. Of course, there are some limitations because these stripped functions do not support scientific notation, have fixed number of precision, etc
By best until now is about 1500 bytes more... but this is WIP....
Last edited by Slammer on Thu Apr 14, 2016 12:27 pm, edited 1 time in total.

User avatar
mrburnette
Posts: 1766
Joined: Mon Apr 27, 2015 12:50 pm
Location: Greater Atlanta
Contact:

Re: Very lean Serial.printf()

Postby mrburnette » Thu Apr 14, 2016 12:27 pm

Looks good! On the STM32F103, 1K flash is not an inordinate quantity of storage to lose to a useful function. I'll have to play around with this later and get the SRAM component, but with bootloader 2.0, 20K of SRAM is usually an ample amount even when large demands are placed on the uC.

But, when SRAM and flash are very tight (maybe a UNO, Nano...) then Mikal Hart's Streaming macro is a no-load way of handling print output formatting. http://arduiniana.org/libraries/streaming/
One can even pull off simple "logic" within the stream:

Example by Rob Tillaart:

Code: Select all

#include <Streaming.h>
// .....
int h = 14;
int m = 6
Serial << ((h<10)?"0":"") << h << ":" << ((m<10)?"0":"") << m << endl;


Ray
Last edited by mrburnette on Thu Apr 14, 2016 6:37 pm, edited 2 times in total.

User avatar
Slammer
Posts: 241
Joined: Tue Mar 01, 2016 10:35 pm
Location: Athens, Greece

Re: Very lean Serial.printf()

Postby Slammer » Thu Apr 14, 2016 12:39 pm

Except the buffer of 64 bytes in Print.printf() member function, the core function use local variables in stack. If I am measuring the bytes correctly, this is about 26 bytes...
Furthermore, if only Serial.printf is used in the program without different versions of Serial.print()/Serial.println() it is possible the total size of program to be smaller.

User avatar
mrburnette
Posts: 1766
Joined: Mon Apr 27, 2015 12:50 pm
Location: Greater Atlanta
Contact:

Re: Very lean Serial.printf()

Postby mrburnette » Thu Apr 14, 2016 12:55 pm

Slammer wrote:Except the buffer of 64 bytes in Print.printf() member function, the core function use local variables in stack. If I am measuring the bytes correctly, this is about 26 bytes...
Furthermore, if only Serial.printf is used in the program without different versions of Serial.print()/Serial.println() it is possible the total size of program to be smaller.



Ummm...
Of course, when a finite buffer is required by the user's code, sprintf() may be a better option. Complex print formatting is just a PITA, but simple stuff can be handled without too much overhead. Your option of printf() is just one more tool in the bag!

Code: Select all

unsigned int i = 0;

void setup() {
  Serial.begin(9600);
}

void loop() {
  char buffer[50];
  sprintf(buffer, "the current value is %d", i++);
  Serial.println(buffer);
}




Ray

User avatar
martinayotte
Posts: 1145
Joined: Mon Apr 27, 2015 1:45 pm

Re: Very lean Serial.printf()

Postby martinayotte » Thu Apr 14, 2016 1:45 pm

Congrat ! you should submit a PR for that !

(On ESP, it is there since awhile, but it was much simpler to implement, since there was enough space to use vsnprintf() ...)

User avatar
Rick Kimball
Posts: 722
Joined: Tue Apr 28, 2015 1:26 am
Location: Eastern NC, US
Contact:

Re: Very lean Serial.printf()

Postby Rick Kimball » Thu Apr 14, 2016 2:17 pm

The down side to printf is that formatting is done using runtime code. If you use the streaming approach or just plain Serial.print all the decisions on how to format are done at compile time. If you are looking for just that little bit extra speed, you might want to stick to the standard stuff.

Over on forum.43oh.com we had long discussions about printf. One user there, Opossum, had a unique approach that resulted in small code that used no buffering. See this post: http://forum.43oh.com/topic/1289-tiny-printf-c-version/
-rick

User avatar
Slammer
Posts: 241
Joined: Tue Mar 01, 2016 10:35 pm
Location: Athens, Greece

Re: Very lean Serial.printf()

Postby Slammer » Thu Apr 14, 2016 2:51 pm

Thanks for the info
I evaluate almost 6-8 different implementations of small printf() functions with additional modifications.

I want something with small footprint (about 1K is OK), no static variables (we need reentrancy), long int support, 0 and (space) specifiers, full support of all integer types, optional float support but without linking of floating point libraries (ok, I can give 1-1.5 more KB for that), integration in Print class.

User avatar
Slammer
Posts: 241
Joined: Tue Mar 01, 2016 10:35 pm
Location: Athens, Greece

Re: Very lean Serial.printf()

Postby Slammer » Thu Apr 14, 2016 11:20 pm

Trying to solve the problem with the buffer in printf(), I realised that I have to move to a different direction.
Instead of trying to encapsulate the C function and the callback inside Print Class (which actually does not have write() function, is only a virtual), it is better to write the printf() as native C++ member function of Print (aka Arduino style). The same approach is used anyway for the other functions of Print.

Now there is no C code, all printf functionality is in a function inside Print class, there is no reason to make callback to write something, it is very easy by calling the virtual function write() of Print. I also need a small internal function to calculate the digits of numerical values.

The total size of these 2 functions is 0x22+0x318 = 0x33A = 826 bytes (No buffers, no static variables)

To use add this code at the end of Print.cpp

Code: Select all

//------------------------------------------------
#ifdef toupper
#undef toupper
#endif
#ifdef tolower
#undef tolower
#endif
#ifdef islower
#undef islower
#endif
#ifdef isdigit
#undef isdigit
#endif

#define toupper(c) ((c)&=0xDF)
#define tolower(c) ((c)|=0x20)
#define islower(c) ((unsigned char)c >= (unsigned char)'a' && (unsigned char)c <= (unsigned char)'z')
#define isdigit(c) ((unsigned char)c >= (unsigned char)'0' && (unsigned char)c <= (unsigned char)'9')

typedef union {
    unsigned char  byte[5];
    long           l;
    unsigned long  ul;
    float          f;
    const char     *ptr;
} value_t;


size_t Print::printDigit(unsigned char n, bool lower_case)
{
    register unsigned char c = n + (unsigned char)'0';

    if (c > (unsigned char)'9') {
        c += (unsigned char)('A' - '0' - 10);
        if (lower_case)
            c += (unsigned char)('a' - 'A');
    }
    return write(c);
}

static void calculateDigit (value_t* value, unsigned char radix)
{
    unsigned long ul = value->ul;
    unsigned char* pb4 = &value->byte[4];
    unsigned char i = 32;

    do {
        *pb4 = (*pb4 << 1) | ((ul >> 31) & 0x01);
        ul <<= 1;

        if (radix <= *pb4 ) {
            *pb4 -= radix;
            ul |= 1;
        }
    } while (--i);
    value->ul = ul;
}

size_t Print::printf(const char *format, ...)
{
    va_list ap;
    bool   left_justify;
    bool   zero_padding;
    bool   prefix_sign;
    bool   prefix_space;
    bool   signed_argument;
    bool   char_argument;
    bool   long_argument;
    bool   lower_case;
    value_t value;
    int charsOutputted;
    bool   lsd;

    unsigned char radix;
    unsigned char  width;
    signed char decimals;
    unsigned char  length;
    char           c;
    // reset output chars
    charsOutputted = 0;

    va_start(ap, format);
    while( c=*format++ ) {
        if ( c=='%' ) {
            left_justify    = 0;
            zero_padding    = 0;
            prefix_sign     = 0;
            prefix_space    = 0;
            signed_argument = 0;
            char_argument   = 0;
            long_argument   = 0;
            radix           = 0;
            width           = 0;
            decimals        = -1;

get_conversion_spec:
            c = *format++;

            if (c=='%') {
                charsOutputted+=write(c);
                continue;
            }

            if (isdigit(c)) {
                if (decimals==-1) {
                    width = 10*width + c - '0';
                    if (width == 0) {
                        zero_padding = 1;
                    }
                } else {
                    decimals = 10*decimals + c - '0';
                }
                goto get_conversion_spec;
            }
            if (c=='.') {
                if (decimals==-1)
                    decimals=0;
                else
                    ; // duplicate, ignore
                goto get_conversion_spec;
            }
            if (islower(c)) {
                c = toupper(c);
                lower_case = 1;
            } else
                lower_case = 0;

            switch( c ) {
            case '-':
                left_justify = 1;
                goto get_conversion_spec;
            case '+':
                prefix_sign = 1;
                goto get_conversion_spec;
            case ' ':
                prefix_space = 1;
                goto get_conversion_spec;
            case 'B': /* byte */
                char_argument = 1;
                goto get_conversion_spec;
//      case '#': /* not supported */
            case 'H': /* short */
            case 'J': /* intmax_t */
            case 'T': /* ptrdiff_t */
            case 'Z': /* size_t */
                goto get_conversion_spec;
            case 'L': /* long */
                long_argument = 1;
                goto get_conversion_spec;

            case 'C':
                if( char_argument )
                    c = va_arg(ap,char);
                else
                    c = va_arg(ap,int);
                charsOutputted+=write(c);
                break;

            case 'S':
                value.ptr = va_arg(ap,const char *);

                length = strlen(value.ptr);
                if ( decimals == -1 ) {
                    decimals = length;
                }
                if ( ( !left_justify ) && (length < width) ) {
                    width -= length;
                    while( width-- != 0 ) {
                        charsOutputted+=write(' ');
                    }
                }

                while ( (c = *value.ptr)  && (decimals-- > 0)) {
                    charsOutputted+=write(c);
                    value.ptr++;
                }

                if ( left_justify && (length < width)) {
                    width -= length;
                    while( width-- != 0 ) {
                        charsOutputted+=write(' ');
                    }
                }
                break;

            case 'D':
            case 'I':
                signed_argument = 1;
                radix = 10;
                break;

            case 'O':
                radix = 8;
                break;

            case 'U':
                radix = 10;
                break;

            case 'X':
                radix = 16;
                break;

            default:
                // nothing special, just output the character
                charsOutputted+=write(c);
                break;
            }

            if (radix != 0) {
                unsigned char store[6];
                unsigned char *pstore = &store[5];

                if (char_argument) {
                    value.l = va_arg(ap, char);
                    if (!signed_argument) {
                        value.l &= 0xFF;
                    }
                } else if (long_argument) {
                    value.l = va_arg(ap, long);
                } else { // must be int
                    value.l = va_arg(ap, int);
                    if (!signed_argument) {
                        value.l &= 0xFFFF;
                    }
                }

                if ( signed_argument ) {
                    if (value.l < 0)
                        value.l = -value.l;
                    else
                        signed_argument = 0;
                }

                length=0;
                lsd = 1;

                do {
                    value.byte[4] = 0;
                    calculateDigit(&value, radix);
                    if (!lsd) {
                        *pstore = (value.byte[4] << 4) | (value.byte[4] >> 4) | *pstore;
                        pstore--;
                    } else {
                        *pstore = value.byte[4];
                    }
                    length++;
                    lsd = !lsd;
                } while( value.ul );
                if (width == 0) {
                    // default width. We set it to 1 to output
                    // at least one character in case the value itself
                    // is zero (i.e. length==0)
                    width = 1;
                }
                /* prepend spaces if needed */
                if (!zero_padding && !left_justify) {
                    while ( width > (unsigned char) (length+1) ) {
                        charsOutputted+=write(' ');
                        width--;
                    }
                }
                if (signed_argument) { // this now means the original value was negative
                    charsOutputted+=write('-');
                    // adjust width to compensate for this character
                    width--;
                } else if (length != 0) {
                    // value > 0
                    if (prefix_sign) {
                        charsOutputted+=write('+');
                        // adjust width to compensate for this character
                        width--;
                    } else if (prefix_space) {
                        charsOutputted+=write(' ');
                        // adjust width to compensate for this character
                        width--;
                    }
                }
                /* prepend zeroes/spaces if needed */
                if (!left_justify) {
                    while ( width-- > length ) {
                        charsOutputted+=write( zero_padding ? '0' : ' ');
                    }
                } else {
                    /* spaces are appended after the digits */
                    if (width > length)
                        width -= length;
                    else
                        width = 0;
                }
                 /* output the digits */
                while( length-- ) {
                    lsd = !lsd;
                    if (!lsd) {
                        pstore++;
                        value.byte[4] = *pstore >> 4;
                    } else {
                        value.byte[4] = *pstore & 0x0F;
                    }
                    charsOutputted+=printDigit(value.byte[4], lower_case);
                }
            }
        } else {
            charsOutputted+=write(c);
        }
    }
    va_end(ap);
    return (size_t)charsOutputted;
}




Add also two definitions in Print.h, include also the file stdarg.h in the beginning of header file (the printDigit can be declared as private)

Code: Select all

   size_t printf(const char * format, ...);
        size_t printDigit(unsigned char n, bool lower_case);



No other implementation is so small, I tried almost everything , there are smaller implementations but they don't support all types of integers or width specifiers or they use static variables....


Return to “Code snipplets”

Who is online

Users browsing this forum: No registered users and 2 guests