"Driverless" Bootloader Proof of Concept

STM32duino bootloader aka Maple bootloader
devan
Posts: 50
Joined: Sat May 14, 2016 1:45 am

"Driverless" Bootloader Proof of Concept

Post by devan » Wed Jul 13, 2016 1:11 am

This is a spin-off of the discussion about the Maple bootloader:
viewtopic.php?f=20&t=1234#p15835

I've been playing around with "driverless" devices lately, with the end goal of making my debugger firmware usable on recent versions of Windows without installing drivers, including the firmware upgrade.

My understanding is that LeafLabs had two obstacles to implementing the bootloader with a composite runtime DFU + CDC-ACM device:
  1. Composite USB devices used different custom descriptors on Windows, Mac OSX, and Linux
  2. On Windows, the DFU and CDC-ACM functions had no native Windows driver. Even where there were existing kernel drivers, they had to be bound to the USB device in a separate installation step.
Point 1 isn't an issue anymore, since I think all three major OSes accept the standardized interface association descriptors.

Point 2 remains, but isn't as bad on Windows 10 (and maybe Windows 8, I haven't tried it). On Windows 10, it will load the default CDC-ACM driver as long as your interface class/subclass/protocol codes match, even if it's embedded in a composite device with other functions that it doesn't understand, like DFU.

For DFU, the end goal is to bind the interface to WinUSB or LibUSB, as the Zadig installer does so that dfu-util can access it. It's possible to do this with Microsoft's custom OS descriptors and the compatible ID descriptor.

As a proof of concept, I've built a proof of concept USB-serial adapter for the bluepill:
  • At boot, the DFU bootloader checks if the flash is empty or if a magic flag is set (in an RTC backup register, which survives reset)
  • The rest of the time, the main application runs a composite DFU + CDC-ACM device on PA2/PA3
  • When the runtime DFU interface receives a detach request, it writes to the RTC backup register and resets so that the main bootloader runs.
I've confirmed that I can load firmware with dfu-util in a Windows 10 VM (with manual intervention to connect the device after it switches to the bootloader and back) and that loopback with a jumper works from PuTTY.

The forum doesn't seem to like .bin attachments (virus risk, probably), so here's a DropBox link to the combined bootloader + usb-serial firmware that can be flashed directly with no offset (ie, address 0x08000000).

https://dl.dropboxusercontent.com/u/193 ... loader.bin

If you want to test that dfu-util works, I have an alternative application-only image with a slightly different product string ("TERMLINK103-2" instead of "TERMLINK103"):
https://dl.dropboxusercontent.com/u/193 ... link-2.bin

The source isn't really documented yet, but it's up on GitHub under the winusb branch:
https://github.com/devanlai/termlink/tree/winusb

As Roger mentioned in the original thread, it would still be a bit of work to integrate a DFU runtime into the libmaple core proper - my code is using libopencm3's USB stack - but on the Windows side of things at least, things don't seem too bad.

As for older versions of Windows - well, the LeafLabs bootloader seems to work okay for most people anyways ;) .

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 7:39 am

@devan

Thanks

Thats very interesting. I'll need to flash it onto one of my boards

What USB VID / PID addreses are you using so that you don't need to install drivers on Windows?

Also, rather than using DFU for upload, you could use either USB mass storage or just use the serial USB (and use a different protocol)
Leaflabs did originally start writing a bootloader that accepted serial uploads in the same format that the STM Flash program sends.

https://github.com/leaflabs/maple-bootl ... erial-boot

I did briefly look at it, but I don't know if it even compiles or works at all

However it may be worth flashing to a board just in case it does work.

I will also contact @jcw to find out how he got on with his bootloader

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 7:47 am

Edit.

I have emailed @jcw, but I don't know if I have offended him in some way, because he has not been on the forum for ages.

But I hope he will reply even if I have somehow unintentionally offended him

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 10:34 am

@devan

I got a reply from @jcw, he is fine.

His bootloader code is here https://github.com/jeelabs/embello/tree ... s/usbserup

Cheers

Roger

EDIT.

Fixed the link
Last edited by RogerClark on Wed Jul 13, 2016 11:48 am, edited 1 time in total.

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 11:38 am

@devan

What hardware does the bootloader work with? Is it just the Maple mini, i.e what does it use to reset the USB ?

I just tried it on a generic F103TB board, but it doesn't appear on USB at all. I guess I need to dig out a Maple mini that I'm not using and flash it with this code.

PS. If you want to attach file to the forum, you could just zip them as it accepts zips. I could change the bbs to accept bin, but it may be a security problem to some users, so its probably best to just use zip

Edit.

Something is wrong with my F103TB board.

I'll try another board tomorrow
Last edited by RogerClark on Wed Jul 13, 2016 11:49 am, edited 1 time in total.

madias
Posts: 813
Joined: Mon Apr 27, 2015 11:26 am
Location: Vienna, Austria

Re: "Driverless" Bootloader Proof of Concept

Post by madias » Wed Jul 13, 2016 11:39 am


User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 11:46 am

Hi Matthias ;-)

Thats odd

I pasted it, perhaps I accidentally deleted the last character

Try this link

https://github.com/jeelabs/embello/tree ... s/usbserup

devan
Posts: 50
Joined: Sat May 14, 2016 1:45 am

Re: "Driverless" Bootloader Proof of Concept

Post by devan » Wed Jul 13, 2016 4:50 pm

I've been testing with the generic Blue Pill boards (with no dedicated USB disconnect circuitry).
What USB VID / PID addreses are you using so that you don't need to install drivers on Windows?
It doesn't use any special VID/PID pair - actually, right now it's just using a VID/PID that's meant for testing.
On newer versions of Windows, it looks at the class and subclass codes in the USB interface descriptors to determine if it has a generic class driver for the standard USB class. So a CDC-ACM USB-serial class device should automagically use the USB serial driver, just like on Mac OSX/Linux.

For device classes that aren't standardized, or that Microsoft didn't develop a class driver for (i.e. DFU), Microsoft queries a special "compatible ID descriptor" the first time the device is plugged in, which can be used to instruct Windows to load certain drivers like WinUSB/LibUSB/MTP, etc.

The libwdi wiki has a very helpful page that goes over the ins and outs of making it work:
https://github.com/pbatard/libwdi/wiki/WCID-Devices.
I got a reply from @jcw, he is fine.

His bootloader code is here https://github.com/jeelabs/embello/tree ... s/usbserup
The usbserup bootloader looks pretty neat - I thought about writing a high memory bootloader after reading a forum post by... jcw. I guess that explains why the approach sounded so familiar.

I was actually aiming for the approach LeafLabs described for the original Maple Rev 1:
http://docs.leaflabs.com/static.leaflab ... aple-rev-1

It uses the standard DFU detach/reset sequence that dfu-util understands, instead of using the serial port secret-knock reset sequence. From the IDE, it probably doesn't make much of a difference (though now that I think about it, maybe the reason I always had to use the perpetual bootloader mode on the Maple was because I opened the serial port from outside of the IDE...). It does make it directly usable with dfu-util, though.

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Wed Jul 13, 2016 9:14 pm

thanks

I will try my blue pill board

User avatar
RogerClark
Posts: 7482
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: "Driverless" Bootloader Proof of Concept

Post by RogerClark » Sat Jul 16, 2016 6:51 am

@devan

I tried loading the new bootloader, but W7
bootloader_error.png
bootloader_error.png (15.61 KiB) Viewed 947 times
Looking in the Windows device manager. I see a new "Unknown device"

Post Reply