[STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Discussions about the STM32generic core
User avatar
Pito
Posts: 1496
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by Pito » Sat Aug 12, 2017 9:50 am

Any thoughts on this?
Would be great if the larger buffer sizes work..
PS: As Steve wrote his buffer is 2kB.
Pukao Hats Cleaning Services Ltd.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by victor_pv » Sat Aug 12, 2017 2:34 pm

Pito wrote:
Sat Aug 12, 2017 9:50 am
Any thoughts on this?
Would be great if the larger buffer sizes work..
PS: As Steve wrote his buffer is 2kB.
With what cores did you experience bytes loses? I was testing the serial speed in the libmaple F4 and noticed from Host -> MCU it loses bytes if the host sends faster than the sketch can receive.

User avatar
Pito
Posts: 1496
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by Pito » Sat Aug 12, 2017 2:45 pm

With STM32GENERIC (this thread) I see loss of data when TX via USB from F4 to TeraTerm with CDC_SERIAL_BUFFER_SIZE (in STM32/cores/arduino/usb/cdc/usbd_cdc_if.h) > 128bytes. The loss is up to 50% of data with larger buffers.
You see nothing wrong unless you start to log the incoming data into a file (in TeraTerm) and count the bytes in that file.
The test used: http://www.stm32duino.com/viewtopic.php ... 354#p31552
Why we need larger buffer? Because the TX is slow - 64kB/sec only with 128bytes.


The libmaple's TX via USB from F4 to TeraTerm at the stock buffer size (2kB as per Steve's info) does not show the loss (speed around 150-220kB/sec).

There is an RX USB speed test written by PaulS I tried with the same results (MapleM) as in the following link

https://www.pjrc.com/teensy/benchmark_u ... ceive.html

BTW - if there was a talented programmer who can write similar benchmark (see PaulS DOS side source) for TX as well, it would be great!
Last edited by Pito on Sun Aug 13, 2017 7:37 am, edited 1 time in total.
Pukao Hats Cleaning Services Ltd.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by victor_pv » Sat Aug 12, 2017 4:09 pm

Pito wrote:
Sat Aug 12, 2017 2:45 pm

There is an RX USB speed test written by PaulS I tried with the same results (MapleM) as in the following link

That's the same test I was doing when I noticed the libmaple F4 would dump incoming bytes if the sketch is not picking them.
If you open the serial port, then don't care to read, and send with Paul's command line utility, it will keep going and going even if the sketch doesn't care to read at all.

A test for TX would be great. Perhaps there is some tool already available somewhere.

On the libmaple F4 I think I have it corrected, at least is not dropping everything, but I need to confirm I'm not missing any byte at all.
I'll see if I can have a look at the generic TX. What buffer sizes did you test that would drop bytes for sure?
In the libmaple core, because of the way the code is written, the buffer needs to be an exact power of 2.

User avatar
Pito
Posts: 1496
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by Pito » Sat Aug 12, 2017 4:59 pm

I'll see if I can have a look at the generic TX. What buffer sizes did you test that would drop bytes for sure?
The CDC_SERIAL_BUFFER_SIZE (in STM32/cores/arduino/usb/cdc/usbd_cdc_if.h) size which works fine is 128.

These I tested with missing bytes: 256, 512, 1024, 2048, 4096, 8k, 16k, 32k on F4, and the same till 16k on F103 (so it is not only about F4).
When talking missing bytes - it is not about a few bytes, but hundreds of kilobytes..

I published the results few weeks back (see my previous posts here in this thread) with speeds up to 1MB/sec for F4 and 450kB/sec for F103, until I started to analyze the amount of data transferred.
It showed the great speeds had been achieved because the transfer did only a fraction of the amount of data and finished smoothly sooner, thus the speeds were such fantastic figures :)

Therefore I deleted the results not to evoke false expectations (until fixed).
I discovered that while messing with TEK emulator, where larger pictures I streamed to TEK via USB started to show defects with larger buffer sizes.
Last edited by Pito on Sun Aug 13, 2017 7:38 am, edited 1 time in total.
Pukao Hats Cleaning Services Ltd.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by victor_pv » Sat Aug 12, 2017 6:07 pm

Pito wrote:
Sat Aug 12, 2017 4:59 pm
I'll see if I can have a look at the generic TX. What buffer sizes did you test that would drop bytes for sure?
The size which works fine is 128.

These I tested with missing bytes: 256, 512, 1024, 2048, 4096, 8k, 16k, 32k on F4, and the same till 16k on F103 (so it is not only about F4).
When talking missing bytes - it is not about a few bytes, but hundreds of kilobytes..

I published the results few weeks back (see my previous posts here in this thread) with speeds up to 1MB/sec for F4 and 450kB/sec for F103, until I started to analyze the amount of data transferred.
It showed the great speeds had been achieved because the transfer did only a fraction of the amount of data and finished smoothly sooner, thus the speeds were such fantastic figures :)

Therefore I deleted the results not to evoke false expectations (until fixed).
I discovered that while messing with TEK emulator, where larger pictures I streamed to TEK via USB started to show defects with larger buffer sizes.
I bet it was nice to see 1MB/s until you found out they were being dropped somewhere :D

I just found this tool:
http://www.serialporttool.com/CommEcho.htm

I'm about to test it, I understand it will echo back all it gets, plus it counts, so if we send let's say 1MB we should receive back 1MB, plus the program should show if Windows received 1MB. Let's see...

User avatar
Pito
Posts: 1496
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by Pito » Sat Aug 12, 2017 6:19 pm

Try the test in the first post here..
Pukao Hats Cleaning Services Ltd.

victor_pv
Posts: 1599
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by victor_pv » Sat Aug 12, 2017 11:45 pm

That program sends down the port the same it receives. Due to that, I had to do some moficications to the sketch so it would read RX bytes too.
Also that allows me to count both the bytes sent up the pipe and received back.

I ran several test, allowing it to wait for longer at the end to see if it would receive any extra bytes, from some misconnunication somewhere, but did not.
Configured like this it waits until the received ammount is the same as the sent amount and displays the total time and speed. It reports 50KB/s, that's each way.
Important to note that either because of Windows, or because or the test program in Windows, the last few bytes take a few seconds to be received back, I guess the program waits to see if it can fill a buffer or something. If that wait wasn't there, the performance would be better. But at least I know at that speed Windows is getting the right amount of bytes, sends them back, and the right amount are received again.
If I raise the TX speed by sending blocks instead of individual bytes (Serial.print (buf, XXXX), at some point I start losing data. But I believe it has more to do with the TX timing out, since there is a max timeout for a transfer and will drop bytes if it takes more than that.

Code: Select all

#define TXCHARS 100000

void loop() {
  char buf[bufsize];

  delay (10000);
  uint32_t n = 0;

  uint32_t i;
  uint8_t x = 85;
  uint32_t elapsed = micros();


  for (i = 0; i < (TXCHARS); i++) {
    Serial.write(x);
    n+= Serial.readBytes (buf, bufsize);
  }

  uint32 endMillis = millis();
  while ((n < TXCHARS)){
  //while ((n < TXCHARS*2) & ((millis() - endMillis) < 10000)){
  n+= Serial.readBytes (buf,bufsize);
  }
  elapsed = micros() - elapsed;

  Serial.println("***");
  Serial.print("USB TX speed = ");
  Serial.print((1000.0 * TXCHARS) / elapsed, 2);
  Serial.println(" KBytes/sec");
  Serial.print ("Elapse (us): ");
  Serial.println (elapsed,DEC);
  Serial.print ("Sent: ");
  Serial.println (TXCHARS, DEC);
  Serial.print ("Received: ");
  Serial.println (n, DEC);

  delay(1000);
  while (1){
      Serial.readBytes (buf,bufsize);
  }
}
This is from the Serial library, and shows the timeout code on USBSerial::write:

Code: Select all

    while (txed < len && (millis() - start < USB_TIMEOUT)) {
        txed += usbSendBytes((const uint8*)buf + txed, len - txed);
        if (old_txed != txed) {
            start = millis();
        }
        old_txed = txed;
    }

vitor_boss
Posts: 61
Joined: Wed Apr 19, 2017 9:50 am

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by vitor_boss » Sun Aug 13, 2017 4:36 am

I have changed the code to this:

Code: Select all

#include "Arduino.h"

#define TXCHARS 100000
#define bufsize 100

void setup() {
  Serial.begin(115200);
  delay(2000);
}

void loop() {
  uint32_t i;
  uint8_t x = 85;
  uint32_t elapsed = micros();
  uint8_t buf[bufsize];

  for (i = 0; i < bufsize; i++) { buf[i] = x; }

  while(1)
  {
    elapsed = micros();
    for (i = 0; i < TXCHARS; i++) {
      Serial.write(x);
    }
    elapsed = micros() - elapsed;

    Serial.println("***");
    Serial.print("USB TX speed = ");
    Serial.print((1000.0 * i) / elapsed, 2);
    Serial.println(" KBytes/sec");
    delay(1000);
  
    elapsed = micros();
    for (i = 0; i < TXCHARS; i+=bufsize) {
      Serial.write(buf, bufsize);
    }
    elapsed = micros() - elapsed;

    Serial.println("***");
    Serial.print("USB buffered TX speed = ");
    Serial.print((1000.0 * i) / elapsed, 2);
    Serial.println(" KBytes/sec");
    delay(1000);
  }  
}
Results on libmaple @Maple mini:
USB TX speed = 427.47 KBytes/sec
USB buffered TX speed = 790.44 KBytes/sec

Results on STM32generic @Maple mini:
USB TX speed = 266.62 KBytes/sec
USB buffered TX speed = 307.24 KBytes/sec

User avatar
Pito
Posts: 1496
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX speed problem

Post by Pito » Sun Aug 13, 2017 7:19 am

Use 1mil chars as in my original test.

Code: Select all

#define TXCHARS 1000000
@vitor_boss: I've got similar speeds (even higher), but the data were lost.. What PC platform do you use??

@victor: to clarify more precisely - the USB TX Buffer Sizes I refer above - those are the buffer sizes here:

CDC_SERIAL_BUFFER_SIZE in STM32/cores/arduino/usb/cdc/usbd_cdc_if.h

Code: Select all

/* Define to prevent recursive inclusion -------------------------------------*/
#ifndef __USBD_CDC_IF_H
#define __USBD_CDC_IF_H

#define CDC_SERIAL_BUFFER_SIZE  128 // USBSerial buffer data length 

#ifdef __cplusplus
Larger CDC_SERIAL_BUFFER_SIZE speeds up the transfer, but in my case the data were lost (ie 500-700kB out of 1MB).
The test finishes ok and you get speeds like 995kB/sec with 16kB CDC_SERIAL_BUFFER_SIZE.
Anything >128 led to loss (in my case - against Win7_64b with TeraTerm, latest hw).

@victor: I cannot get your code working against CommEcho.
CommEcho prints out "U" in 1 sec intervals here (BlackF407ZET)
CommEcho.JPG
CommEcho.JPG (22.5 KiB) Viewed 67 times
Pukao Hats Cleaning Services Ltd.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest