Description of problem: Sandisk Cruzer 8G thumb drive with U3 hangs Fedora 15 Alpha Desktop media. When pressing ESC, messages about sr0 appear. Version-Release number of selected component (if applicable): Fedora 15 Alpha (gold) and recent nightlies. How reproducible: Use dd to place Fedora image directly on drive, and boot computer. Actual results: Boot hangs Expected results: Happy Fedora experience. Additional info: I don't see this issue with non-U3 devices. Fedora 14 booted perfectly despite the U3 device on the same thumb drive.
Please collect the following logs and attach them individually to this bug report: /tmp/anaconda.log /tmp/storage.log /tmp/program.log /var/log/messages Thanks.
Would love to collect logs, however this occurs before the Live environment allows logins. As a side note, I've now cleared the device via u3-tool (http://u3-tool.sourceforge.net/) with the following: $ sudo ./u3-tool -p 0 /dev/sdc Fedora now boots properly.
There's not much we can do with this bug report without those logfiles. At least we have a workaround...
Created attachment 487117 [details] dmesg photo 1 This is a photo of the kernel output during bootup. At this point in the process, it just sits like this for more than a couple minutes before scrolling error messages in the next photo.
Created attachment 487118 [details] dmesg photo 2 Photo of the scrolling error messages that appear after a few minutes. System does not continue booting.
(In reply to comment #5) > Created attachment 487118 [details] > dmesg photo 2 > > Photo of the scrolling error messages that appear after a few minutes. System > does not continue booting. I don't think those messages come from the kernel. Maybe udev is printing them?
(In reply to comment #5) > Created attachment 487118 [details] > dmesg photo 2 > > Photo of the scrolling error messages that appear after a few minutes. System > does not continue booting. No, I am not familiar with these messages. I see, this is a Live image booting with dracut. dracut activates (if provided by the kernel), the cdrom autopolling. echo 1000 > /sys/block/sr*/events_poll_msecs Maybe these are the debug messages for autopolling?? For the live medium, you could try to add "rd.info rd.shell" or even "rd.debug rd.shell" to the kernel command line, to see what's going on.
Created attachment 487652 [details] photo with dracut debugging 1 Photo of bootup with dracut debugging enabled. The process sits in this state for a long time before scrolling the messages in the next photo...
Created attachment 487653 [details] photo with dracut debugging 2 Photo of bootup with dracut debugging enabled. It keeps doing this part over and over, and my USB stick is blinking the entire time.
Ok, seems like the in kernel cdrom kernel is not working the way I thought it will be. I'll disable it for now.
dracut-009-1.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/dracut-009-1.fc15
Package dracut-009-1.fc15: * should fix your issue, * was pushed to the Fedora 15 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing dracut-009-1.fc15' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/dracut-009-1.fc15 then log in and leave karma (feedback).
Created attachment 488345 [details] dracut-009.1.fc15 USB u3 lockup I'm still encountering this issue with dracut-009.1.fc15. Attached is a photo of where it gets stuck, which contains more debugging output.
Add "rd.shell rd.break=pre-trigger" to the kernel command line and manually run: # /lib/udev/cdrom_id --debug /dev/sr0 # /sbin/blkid -o udev -p -u noraid /dev/sr0 and see, if this hangs
dracut-009-2.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/dracut-009-2.fc15
(In reply to comment #15) > dracut-009-2.fc15 has been submitted as an update for Fedora 15. > https://admin.fedoraproject.org/updates/dracut-009-2.fc15 disregard this... it's not fixed with this version.
(In reply to comment #14) > Add "rd.shell rd.break=pre-trigger" to the kernel command line > > and manually run: > > # /lib/udev/cdrom_id --debug /dev/sr0 > # /sbin/blkid -o udev -p -u noraid /dev/sr0 > > and see, if this hangs /dev/sr0 does not exist that this point in the process for me.
ok, use "rd.shell rd.break=initqueue"
Created attachment 488808 [details] photo from dracut shell (In reply to comment #14) > Add "rd.shell rd.break=pre-trigger" to the kernel command line > > and manually run: > > # /lib/udev/cdrom_id --debug /dev/sr0 > # /sbin/blkid -o udev -p -u noraid /dev/sr0 > > and see, if this hangs Nope, neither of these commands hang. Photo attached.
Proposing this as a F15 nice-to-have bug (https://fedoraproject.org/wiki/QA:SOP_nth_bug_process#Proposing_nice-to-have_bugs).
This was discussed at the 2011-04-15 blocker bug meeting and it was concluded that we need more information before deciding on NTH for this. - Does this affect all USB 3.0 liveusb images - Does this only affect liveusb images created with dd?
(In reply to comment #21) > This was discussed at the 2011-04-15 blocker bug meeting and it was concluded > that we need more information before deciding on NTH for this. > > - Does this affect all USB 3.0 liveusb images This has nothing to do with USB 3.0. This issue affects all USB keys with U3 http://en.wikipedia.org/wiki/U3 These USB drives are extremely popular, and this bug will frustrate many users, who will most likely give up and install another distro before trying a different USB stick. > - Does this only affect liveusb images created with dd? No, this affects *all* LiveUSBs.
(In reply to comment #22) > (In reply to comment #21) > > This was discussed at the 2011-04-15 blocker bug meeting and it was concluded > > that we need more information before deciding on NTH for this. > > > > - Does this affect all USB 3.0 liveusb images > > This has nothing to do with USB 3.0. > This issue affects all USB keys with U3 http://en.wikipedia.org/wiki/U3 > > These USB drives are extremely popular, and this bug will frustrate many users, > who will most likely give up and install another distro before trying a > different USB stick. > > > - Does this only affect liveusb images created with dd? > > No, this affects *all* LiveUSBs. OK, I think that we completely misunderstood this bug. When you get close to the 4 hour mark of blocker meetings, you get tired and I don't think that I had ever heard of a u3 smart drive before. To re-summarize: liveusb images on u3 smart drives fail to boot very early in the boot process. This happens regardless of the method used to create the liveusb image (liveusb-creator or dd). It is possible to workaround this issue by clearing the device using u3-tool but that is far from an ideal solution.
(In reply to comment #23) > To re-summarize: liveusb images on u3 smart drives fail to boot very early in > the boot process. This happens regardless of the method used to create the > liveusb image (liveusb-creator or dd). It is possible to workaround this issue > by clearing the device using u3-tool but that is far from an ideal solution. Correct.
Discussed at 2011-04-21 blocker review meeting (http://meetbot.fedoraproject.org/fedora-bugzappers/2011-04-21/f15-blocker-review.2011-04-21-17.00.html) ... AGREED: 683265 - AcceptedNTH
Created attachment 495389 [details] udev strace Attached is an strace of udev after I plug my in my Cruzer USB stick on Fedora 15 Beta. It never gets properly initialized, and it just sits there and blinks while udev spins.
I ran `udevadm monitor`, plugged in the drive, and these messages scroll over and over. KERNEL[1303965881.981688] change /devices/pci0000:00/0000:00:1d.7/usb2/2-6/2-6.1/2-6.1.2/2-6.1.2:1.0/host73/target73:0:0/73:0:0:1/block/sr1 (block) UDEV [1303965881.983246] change /devices/pci0000:00/0000:00:1d.7/usb2/2-6/2-6.1/2-6.1.2/2-6.1.2:1.0/host73/target73:0:0/73:0:0:1/block/sr1 (block)
I have found that only way reliable way to use a U3 USB is to boot it in windows and click on the u3 application and delete it. It is a hidden partition that has to delete itself. (gparted does not see it in formatting) I have a caution note in sugarlabs wiki wrt using these USB's Note that the packaging does not always show the u3 presence on them.
Re-assigning to udev, since simply plugging in a U3-enabled USB stick makes it very unhappy (see Comment #27).
*** Bug 706989 has been marked as a duplicate of this bug. ***
* i hope you have taken notice of my function-tracing in Bug 706989, showing that the udev-events reported in "Comment 27", finally arrive from the func- tion "sr_get_events()" in source-file "drivers/scsi/sr.c" line#s in that file: 198, 199: "else if (med->media_event_code == 2) return DISK_EVENT_MEDIA_CHANGE;" * and that commenting out these lines stop the udev-events reported in "Com- ment 27": "/* else if (med->media_event_code == 2) return DISK_EVENT_MEDIA_CHANGE; */" * but maybe you already know this? (sorry).
* if it is possible to access the usb-id = (vendor = v, producId = p ) of these 'strange' usb-sticks in the function "drivers/usb/storage/scsiglue.c/queuecommand_lck()", maybe with a function like "struct usb-id scsi_cmnd_to_usb_id(struct scsi_cmnd * cmd) { }" (takes cmd and delivers usb-id), to be defined. * i would add a module-parameter to the module usb_storage, to provide to the module the usb-ids from above. * add to the function ".../queuecommand_lck()" something like this: " [...] uint8_t scsicmd = srb->cmnd[0]; [...] if(GET_EVENT_STATUS_NOTIFICATION == scsicmd) if( ' it is an sr [not a normal disk] ' ) if( ' "scsi_cmnd_to_usb_id(struct scsi_cmnd * srb)" is in the list of usb-ids, provided by the module-parameter from above ' ) { ' do not queue the command but give directly back the result: ' media not changed ' } [...] " * i will try to do the above i you dont want to.
(In reply to comment #32) > * i will try to do the above i you dont want to. Please :)
(In reply to comment #32) > * i will try to do the above i you dont want to. So what you're saying is that this device always reports "media changed" when queried? You may want to take this report upstream to the linux-kernel and/or linux-scsi lists if you can.
Created attachment 500897 [details] kernel oops Attached is a kernel oops when unplugging my USB stick. Looks to be related to the U3 "cdrom". Seems racy, as plugging it in a second time seemed to work.
i have followed your piece of advice (comment #34) and sent the following mail to Mr. M. Dharm, the kernel-maintainer of usb-storage: =============================================================================== From: private_mail_addess("Thomas Bruecker") Subject: u3 device on Sandisk Cruzer 8G USB Drive hangs Fedora 15 Alpha boot Date: Wed, May 25, 2011 20:24 (Swiss-local-time) To: "Matthew Dharm" <mdharm-usb> Dear Mr Dharm, I have investigated the problem above and i propose that the cause of the bug (1*) is either a bug in the module "usb_storage", or these usb-sticks (2*) are responding incorrectly to some scsi-command. * the boot-hang is caused by an incessant stream of udev messages like: "KERNEL[1303965881.981688] change /devices/pci0000:00/0000:00:1d.7/usb2/2-6/2-6.1/2-6.1.2/2-6.1.2:1.0/host73/target73:0:0/73:0:0:1/block/sr1 (block)" (got with "udevadm monitor --kernel"). (3*) * these messages come finally from the function "sr_get_events()" in source-file "drivers/scsi/sr.c" (4*) line#s in that file: 198, 199: "else if (med->media_event_code == 2) return DISK_EVENT_MEDIA_CHANGE;" * in any case booting is normal and these messages cease, when you comment out "/* else if (med->media_event_code == 2) return DISK_EVENT_MEDIA_CHANGE; */" in the file mentioned above. 1*) see "https://bugzilla.redhat.com/show_bug.cgi?id=683265". 2*) usb-ids of such usb-sticks (e.g. Sandisk): * 2GB: (Vendor=0781 ProdID=5406 Rev=2.00), * 16GB: (Vendor=0781 ProdID=5535 Rev=2.00) "/proc/bus/usb"-details and other details see: "https://bugzilla.redhat.com/show_bug.cgi?id=706989": "Attachments". 3*) see also: "https://bugzilla.redhat.com/show_bug.cgi?id=683265#c27", and in "https://bugzilla.redhat.com/show_bug.cgi?id=706989": "Description", "Complete Description" 4*) applies to "kernel-2.6.36" and "kernel-2.6.38.5-24.fc15.x86_64"; the function-tracing from the udev-messages (3*) to the "sr_get_event()..." see in "https://bugzilla.redhat.com/show_bug.cgi?id=706989": "Description", "Complete Description". I hope this will help you to fix the bug, Sincerely Thomas Bruecker
'answer' to comment #36: From: [ "Matthew Dharm" <mdharm-usb> ] Subject: Re: u3 device on Sandisk Cruzer 8G USB Drive hangs Fedora 15 Alpha boot Date: Wed, May 25, 2011 22:33 [shows Swiss-local-time] To: [ private_mail_address("Thomas Bruecker") ] Cc: "Matthew Dharm" <mdharm-usb> [ mail_account( private_mail_address("Thomas Bruecker" ) is not ok. ] U3 devices include an emulated CD-ROM in addition to the regular flash-based storage area. Based on this data, it is pretty clear that there is an error in the CD-ROM emulation on the device which triggers this bug by appearing to be a continual stream of media-changed events. As noted in the bugzilla you cited, removing the emulated CD-ROM makes the problem go away. The only other way to fix this would be to filter the GET_EVENT_STATUS_NOTIFCATION command from being sent to the device. I would recommend filtering it in sr.c based on the INQUIRY data returned by the device, rather than filtering in usb-storage. Filters placed in usb-storage will also affect other unrelated codepaths, such as the SG-IO interface. Matt =============================================================================== On Wed, May 25, 2011 at 11:24 AM, [ != Swiss-local-time ] [ private_mail_address( "Thomas Bruecker" ) ] wrote: Dear Mr Dharm, I have investigated the problem above [...] =============================================================================== --> ok, then i will (Thomas Bruecker continue with my idea of comment #32 .
to comment #32: it IS POSSIBLE to get the 'usb_id' from the (struct scsi_cmnd* srb). --> i think i can now execute my idea.
The source-code according to comments #32, #33, #37 and 38 is now available at "http://www.thomas-r-bruecker.ch/favorites/io/disk_/usb/linux/bug_683265,_706989/.license_dialog.php". After accepting the license (if you want), change to the folder "src". The latest source-files are: * "version_.0/debug0x05bf.h" * "version_.0/scsiglue.c" * "version_.0.0.1/strange_usb_sticks0x05bf.c" * "version_.0/strange_usb_sticks0x05bf.h" * Most probably these files are not very helpful without compiling- /installing- and user-instrtuctions. They will follow soon.
The user instructions, mentioned in "Comment #39" are now available at: "http://www.thomas-r-bruecker.ch/favorites/io/disk_/usb/linux/bug_683265,_706989/.workaround_license_dialog.php". After accepting the lincense (if you want), you may read the text "Workaround.php". It contains also links to the "Compiled Modules" for some kernels.
nothing udev can do, so reassign to component kernel
Created attachment 516377 [details] Makefile for sources according to Comment # 39 and 40
* I have just recognized, that you have a new kernel (...-2.6.40-4.fc15). * I define OLD_SOLUTION := "the solution proposed in Comments # 32, 38 - 40". * For kernels with a version-# >= 2.6.39 maybe there would be a better approch to solve the problem, than using the OLD_SOLUTION (according to Comment #37, Mr. Dharm too does not like changes in the function queuecommand_lck()" ): * In "kernel-2.6.39", in "block/genhd.c" at line # 1597: "if (events & disk->events & (1 << i))", obvioulsy the events sent to the userland are masked out by "disk->events" (struct gendisk::events). I assume that these "disk->events" for a cdrom, will be set in "drivers/scsi/sr.c", in the function ("sr_probe()", line #603) at line# 640: "disk->events = DISK_EVENT_MEDIA_CHANGE | DISK_EVENT_EJECT_REQUEST;" during the scan for disks. * So I think, a good workaround for this bug would be, instead of using the OLD_SOLUTION, to ask the "usb_storage"-driver, during disk-scan for usb-storage media, which events should be sent to the userspace, and not setting "DISK_EVENT_MEDIA_CHANGE" for sticks like "Sandisk Cruzer 8G". * (more detailed:) in "drivers/scsi/sr.c", in the function ("sr_probe()", line #603) the line# 640 should be changed to something like (strict details are most probabely wrong!) : "typdef int (askFunction_t)(); askFunction_t *askFnPtr = ...->askUnderlyingDriverForEventsThatShouldBeSentToTheUserspace; "if (null != askFnPtr) /* the disk-driver has such an ask-function */ disk->events = (askFnPtr)(); else /* the disk driver does not have ... */ disk->events = DISK_EVENT_MEDIA_CHANGE | DISK_EVENT_EJECT_REQUEST;" * the function "int askUnderlyingDriverForEventsThatShouldBeSentToTheUserspace()" for usb-storage devices should then return 0 for a "stange_usb_stick". * to achieve, of course the usb-storage-driver code, "sr.c", etc.(?) must be changed. I will not do this. * I have also proposed this new solution to Mr. Dharm on May 31, 2011. * Remarks to Comment #40 and my Workaround.php: Taking into account, that * (see above), there will be a better solution, * (only) 13 users are in the CC List, * there have been very few accesses to my pulished documents from Comment #40, * no download of my compiled kernel-modules has taken place, * it takes my time to compile my modules for every new kernel published by Fedora, * a description for "normal users", how to change an initrd, takes a lot of time to write. * (RH) my workaround requires changes to "/etc/rc.d/rc.sysinit" (this file may be changed [and so the workaround is reverted] inadvertly by every (automatic) update. * I do not belong to the Fedora- / Redhat-staff. So I have decided to discontinue maintaining my published documents; they will remain, as they are, as a suggestion; for completeness I have added the Makefile, that I have used, as Comment # 42 ATTENTION: THIS MAKEFILE MUST BE PATCHED FOR EVERY OTHER KERNEL-VERSION. Because of (RH) it would be best if ev. Fedora / Redhat would provide a complete solution including an rpm-package with the appropriate "/etc/rs.sysinit" in case of the OLD_SOLUTION or ... . At least I have shown, that the problem may be solved (An FC14 on an Acer Netbook runs perfectly with two such 'strange usb-sticks').
* This Bug (683265) seems to be solved with: kernel-2.6.40.6-0.fc15: * booting is ok, * the messages according to Comment #27 are not displayed. * See also kernel-source: kernel-2.6.40.6-0.fc15.src.rpm, patch-3.0.6.bz2, line# 13401 ff.
Also see: http://wiki.sugarlabs.org/go/Sugar_Creation_Kit#Cautions_with_u3_USB_sticks http://u3-tool.sourceforge.net/ Be sure you have removed the U3 hidden partition first (see comment 37)
(In reply Comment #45): * I claim, that, with kernel-2.6.40.6-0.fc15 (!), the measures according to Comment #45 etc, will not be necessary to avoid the bug. * I have tested this, by having connected to the computer a "Sandisk, Ultra Backup"-stick: as specified in Attachments of "bug 706989": "/proc/bus/usb/devices-7", "/proc/scsi/scsi-7" and "/proc/scsi/usb-storage/7", booting the computer without a problem, an not seeing the messages according to Comment #27 . * On other kernels, I have been able to produce the bug with the above stick, see "bug 706989".
(Addendum to Comment #44): See also "ChangeLog-3.0.1" (actually at "www.kernel.org" but it is not available there at the time of THIS comment; so alternate source: "http://ftp.sunet.se/pub/Linux/kernels/v3.0/ChangeLog-3.0.1" ), (search for) "commit 645b2cf1067d945c662ffeea45b4c0f7036bc1ee", ff. .