[STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Discussions about the STM32generic core
User avatar
Pito
Posts: 1592
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by Pito » Fri Aug 18, 2017 10:01 pm

http://www.beyondlogic.org/usbnutshell/usb1.shtml
The host is responsible for managing the bandwidth of the bus. This is done at enumeration when configuring Isochronous and Interrupt Endpoints and throughout the operation of the bus.
Pukao Hats Cleaning Services Ltd.

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by victor_pv » Fri Aug 18, 2017 10:44 pm

Yes, but keep reading:
http://www.beyondlogic.org/usbnutshell/ ... sochronous
OUT: When the host wants to send the function a bulk data packet, it issues an OUT token followed by a data packet containing the bulk data. If any part of the OUT token or data packet is corrupt then the function ignores the packet. If the function's endpoint buffer was empty and it has clocked the data into the endpoint buffer it issues an ACK informing the host it has successfully received the data. If the endpoint buffer is not empty due to processing a previous packet, then the function returns an NAK. However if the endpoint has had an error and its halt bit has been set, it returns a STALL.
Also these I think are important to note:
Bulk Transfers
Used to transfer large bursty data.
Error detection via CRC, with guarantee of delivery.
No guarantee of bandwidth or minimum latency.
Stream Pipe - Unidirectional
Full & high speed modes only.
We are supposed to guarantee delivery with the handshake, but there no guarantee of bandwidth, so we should not drop packets just because the application in the host or the MCU is not as fast the other end.

User avatar
Pito
Posts: 1592
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by Pito » Fri Aug 18, 2017 11:18 pm

My naive "user" observation (based on the above results with libmaple F1 and F4) is following:

The libmaple's USB TX "understood the TeraTerm's (the Host) handshaking commands", as it had transferred 1mil chars ok with 102kB/s for both F1 and F4 while TT had logged the data into a file.

It is obvious the libmaple's TX had been orchestrated by the Host, as the total TX speed achieved is the same for F1 and F4 (the packet's speed is the same because the usb clock is the same, but the overhead F4/F1 would have made a difference in total TX speed when the packets were not synced by the Host).

With Daniel's TX we can achieve Nx higher speeds (with larger CDC buffer sizes) but we loose say 70% of data - thus it seems the stm32generic TX ignores the TT Host's handshake commands..

The Arduino's serial monitor is perhaps much faster than TT, therefore it captures 1mil chars without proper handshaking, or, it uses a different handshaking model the stm32generic TX understands better..
Pukao Hats Cleaning Services Ltd.

User avatar
RogerClark
Posts: 7145
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by RogerClark » Sat Aug 19, 2017 1:09 am

Libmaple has code that checks one of the handshaking signals (DTR I think), and if it can't send data to the host, then the code "blocks" in the write() function

It has been noted that this differs from the Due which don't seem to check if the Host is ready for the data

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by victor_pv » Sat Aug 19, 2017 1:09 am

@Pito, I pretty much agree with you on that theory.

victor_pv
Posts: 1681
Joined: Mon Apr 27, 2015 12:12 pm

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by victor_pv » Sat Aug 19, 2017 1:18 am

RogerClark wrote:
Sat Aug 19, 2017 1:09 am
Libmaple has code that checks one of the handshaking signals (DTR I think), and if it can't send data to the host, then the code "blocks" in the write() function

It has been noted that this differs from the Due which don't seem to check if the Host is ready for the data
Roger, I looked a lot at the F1 code when writing readbytes to try to achieve the max speeds. The Libmaple F1 will work with the host with the ACK and NACK in both TX and RX, so when the buffer in one side is full, the other side will hold on further packets. when the buffer gets space, an ACK is sent to the other end, and continues the transmission, until a buffer fills and that end sends NACK again.
As far as I what I saw with DTR, is used only to detect the reset magic word, but with for normal handshaking, but the handshaking with ACK and NACK is what's supposed to happen, and seems to be a good implementation.

The F4 in RX did not have that. It would always send ACK, even if a buffer was full. I changed that and works pretty good now. I can get 500KB/s, similar to the F1 (the MCU is faster, but the code is based on the SPL, mush more overhead than libmaple).
The F4 TX did work fine in my tests and I didn't make any change to it.

I haven't had a chance to dig deep in the one used by the Generic core to see what exactly does, but I suspect the handshaking is not right as Pito suspects.

Personally I think that an implementation that loses bytes in either direction is just pointless. If I care to send something one way is because I want to receive it in the other end.
I understand for sprinkling serial debug prints here and there may not matter much, but for transfering images, tables, sensor data, whatever else is sent, I wouldn't want to lose a single byte unless the link is down or one end is not responding for too long.
I'd rather have lossless 100KB/s that 500KB with loses.

vitor_boss
Posts: 61
Joined: Wed Apr 19, 2017 9:50 am

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by vitor_boss » Sat Aug 19, 2017 4:26 am

I'm not a programmer, just a curious, this is what I could find:

Code: Select all

...
       CDC_Transmit_FS(&tx_buffer.buffer[tx_buffer.iTail], transmitting); //Set buffer and begin tranmission
...
That line is for call HAL USB/serial stuff, now the calls:

Code: Select all

uint8_t CDC_Transmit_FS(uint8_t* Buf, uint16_t Len) //usbd_cdc_if.c
{
  uint8_t result = USBD_OK;
  /* USER CODE BEGIN 7 */
  USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef*)hUsbDeviceFS.pClassData;
  if (hcdc->TxState != 0){ //1 when have something sending
    return USBD_BUSY;
  }
  USBD_CDC_SetTxBuffer(&hUsbDeviceFS, Buf, Len);
  result = USBD_CDC_TransmitPacket(&hUsbDeviceFS);
  /* USER CODE END 7 */
  return result;
}
//usbd_cdc.c
uint8_t  USBD_CDC_SetTxBuffer  (USBD_HandleTypeDef   *pdev,
                                uint8_t  *pbuff,
                                uint16_t length)
{
  USBD_CDC_HandleTypeDef   *hcdc = (USBD_CDC_HandleTypeDef*) pdev->pClassData;
  
  hcdc->TxBuffer = pbuff;
  hcdc->TxLength = length;  
  
  return USBD_OK;  
}
uint8_t  USBD_CDC_TransmitPacket(USBD_HandleTypeDef *pdev)
{      
  USBD_CDC_HandleTypeDef   *hcdc = (USBD_CDC_HandleTypeDef*) pdev->pClassData;
  
  if(pdev->pClassData != NULL)
  {
    if(hcdc->TxState == 0)
    {
      /* Tx Transfer in progress */
      hcdc->TxState = 1;
      
      /* Transmit next packet */
      USBD_LL_Transmit(pdev,
                       CDC_IN_EP,
                       hcdc->TxBuffer,
                       hcdc->TxLength);
      
      return USBD_OK;
    }
...
//usbd_conf_F1.c or usbd_conf_F4.c
USBD_StatusTypeDef  USBD_LL_Transmit (USBD_HandleTypeDef *pdev,
                                      uint8_t  ep_addr,
                                      uint8_t  *pbuf,
                                      uint16_t  size)
{
  HAL_PCD_EP_Transmit((PCD_HandleTypeDef*) pdev->pData, ep_addr, pbuf, size);
  return USBD_OK;
}
//stm32f1xx_hal_pcd.c or stm32f4xx_hal_pcd.c
HAL_StatusTypeDef HAL_PCD_EP_Transmit(PCD_HandleTypeDef *hpcd, uint8_t ep_addr, uint8_t *pBuf, uint32_t len)
{
  PCD_EPTypeDef *ep = NULL;
  
  ep = &hpcd->IN_ep[ep_addr & 0x7FU];
  
  /*setup and start the Xfer */
  ep->xfer_buff = pBuf;  
  ep->xfer_len = len;
  ep->xfer_count = 0U;
  ep->is_in = 1U;
  ep->num = ep_addr & 0x7FU;

  if ((ep_addr & 0x7FU) == 0U)
  {
    USB_EP0StartXfer(hpcd->Instance , ep);
  }
  else
  {
    USB_EPStartXfer(hpcd->Instance , ep);
  }

  return HAL_OK;
}
//stm32f1xx_ll_usb.c
HAL_StatusTypeDef USB_EPStartXfer(USB_OTG_GlobalTypeDef *USBx , USB_OTG_EPTypeDef *ep, uint8_t dma)
//seems to end here
HAL_StatusTypeDef USB_WritePacket(USB_OTG_GlobalTypeDef *USBx, uint8_t *src, uint8_t ch_ep_num, uint16_t len, uint8_t dma)
As far I read, I didn't found any handshaking signals.

danieleff
Posts: 336
Joined: Thu Sep 01, 2016 8:52 pm
Location: Hungary
Contact:

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by danieleff » Sat Aug 19, 2017 6:05 am

Libmaple latest from repo, Maple Mini, TeraTerm log, test code from first post (GCC 6-2017-q1-update), the USB seems to drop data (All ***'s should be on same column):
teraterm3.png
teraterm3.png (48.29 KiB) Viewed 175 times

User avatar
Pito
Posts: 1592
Joined: Sat Mar 26, 2016 3:26 pm
Location: Rapa Nui

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by Pito » Sat Aug 19, 2017 7:12 am

Hmm, what I've done this morning - installed Wireshark - free open network analyzer, which includes "USB Packet capture" analyzer.

https://www.wireshark.org/

Testing via the loop sending "U" (0x55) to TeraTerm (Win7_64b), from Black F407:

Libmaple F4 - the data packets identified as "5555" and Ethernet II - "'Malformed packet", always data packet protocol "5555" with some header data and then containing 2048x "U", and a small "Ethernet packet" protocol type assigned "28 URB Bulk in Malformed", containing 1x U in between, no handshake visible (unless the malformed is the handshake packet)

32Generic - the data packets identified as 5555 or 'Ethernet II", single packet containing 16-100 "U"s, no handshake visible

As a proof it somehow works I've tried for fun:
I saved the file with UUUs to corsair flash drive, and read it into an editor.
It is a different device (mass storage) but - all data packets assigned "Good", up to 65535 large each, with handshake (2 <-> transactions between Host an device) between the data packets. The packet protocols were USB related.

Libmaple:
Libmaple USB capture.JPG
Libmaple USB capture.JPG (194.53 KiB) Viewed 160 times
Libmaple USB capture 2.JPG
Libmaple USB capture 2.JPG (148.42 KiB) Viewed 157 times
Generic capture.JPG
Generic capture.JPG (122.57 KiB) Viewed 154 times
Last edited by Pito on Sat Aug 19, 2017 7:58 am, edited 1 time in total.
Pukao Hats Cleaning Services Ltd.

stevestrong
Posts: 1740
Joined: Mon Oct 19, 2015 12:06 am
Location: Munich, Germany

Re: [STM32GENERIC/HAL] SerialUSB TX/RX speed problem

Post by stevestrong » Sat Aug 19, 2017 7:57 am

Pito, very interesting.
The libmaple F4 USB CDC is St SPL based.
Can you please test libmaple F1?

Post Reply