As in bug 290951, the USB key already contained a live distribution and a GRUB
instance in the MBR of the disk. Despite reformatting the key, I wasn't able to
boot as the MBR wasn't reset/updated.
syslinux has an mbr.bin file that you can dd to the device to solve this problem.
Added --reset-mbr as an option that will copy the syslinux mbr.bin.
Unfortunately, doing it by default is probably not the right approach as there
could be something needed on there. And trying to figure that out. ugh.