As in bug 290951, the USB key already contained a live distribution and a GRUB instance in the MBR of the disk. Despite reformatting the key, I wasn't able to boot as the MBR wasn't reset/updated. syslinux has an mbr.bin file that you can dd to the device to solve this problem.
Added --reset-mbr as an option that will copy the syslinux mbr.bin. Unfortunately, doing it by default is probably not the right approach as there could be something needed on there. And trying to figure that out. ugh.