Bug 504444

Summary: svirt denials break USB device passthrough
Product: [Fedora] Fedora Reporter: Bob Cochran <cochranb>
Component: libvirtAssignee: Daniel Veillard <veillard>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 12CC: berrange, clalance, clasohm, crobinso, dwalsh, erik, itamar, luca.botti, markmc, rob+redhat, sbruno, veillard, virt-maint
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: selinux-policy-3.6.12-54.fc11 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-17 16:09:02 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 480594    
Attachments:
Description Flags
Requested log of /var/log/libvirt/qemu/Ubuntu9.log
none
Requested output of lsusb -v
none
From /var/log/libvirt/qemu/Ubuntu9.log, log file of device passthrough occuring in SELinux permissive mode
none
Messages from /var/log/messages on Fedora 11 host machine, recorded from just before plugging device in to when device is unplugged from host.
none
Log messages recorded in the Ubuntu 9.04 guest machine for the same FTDI uart device
none
Console messages reflecting successful use of FTDI uart device from the Ubuntu 9.04 guest's "Arduino-0016" application
none
Contains text files showing kernel messages on both host and guest none

Description Bob Cochran 2009-06-06 21:21:25 EDT
Description of problem:

Coding this xml:

<hostdev mode='subsystem' type='usb'>
       <source>
          <vendor id='0x0403'/>
	  <product id='0x6001'/>
       </source>
</hostdev>

should let my Ubuntu 9.04 guest have access to the device, which in this case is a USB-to-Serial UART (based on an FTDI FT232RL chip.) However the guest does not "see" the device or recognize it in any way. It does not show up in the guest's /var/log/messages or in the output of lsusb. I would like the guest to have access to this device.

This is what virsh seems to be seeing:

# virsh dumpxml Ubuntu9
<domain type='kvm'>
  <name>Ubuntu9</name>
  <uuid>cb8c3d15-fe92-1bf2-5676-03370743e815</uuid>
  <memory>1572864</memory>
  <currentMemory>1572864</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <disk type='file' device='disk'>
      <source file='/var/lib/libvirt/images/Ubuntu9.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='network'>
      <mac address='54:52:00:74:32:2f'/>
      <source network='default'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <sound model='es1370'/>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0403'/>
        <product id='0x6001'/>
      </source>
    </hostdev>
  </devices>
</domain>


Version-Release number of selected component (if applicable):

libvirt-0.6.2-11.fc11.x86_64

How reproducible:

Every time

Steps to Reproduce:
1. Start the guest OS running.
2. Plug the USB device into the host machine's USB port. In my case, the host machine is a Dell Latitude E6400 laptop. I'm using the machine's own USB ports and not the USB ports on the docking station, even though the machine is docked.
3. Log into the guest OS.
4. Check /var/log/messages and/or the output of 'lsusb'.
  
Actual results:

The usb device is not seen or recognized in the guest.

Expected results:

I should be able to have normal USB functionality if I specify the device product and vendor IDs in the xml processed by libvirt for that guest domain.

Additional info:

Entire xml file for the guest OS:

# cat /etc/libvirt/qemu/Ubuntu9.xml

<domain type='kvm'>
  <name>Ubuntu9</name>
  <uuid>cb8c3d15-fe92-1bf2-5676-03370743e815</uuid>
  <memory>1572864</memory>
  <currentMemory>1572864</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <disk type='file' device='disk'>
      <source file='/var/lib/libvirt/images/Ubuntu9.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='network'>
      <mac address='54:52:00:74:32:2f'/>
      <source network='default'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <hostdev mode='subsystem' type='usb'>
       <source>
          <vendor id='0x0403'/>
	  <product id='0x6001'/>
       </source>
    </hostdev>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <sound model='es1370'/>
  </devices>
</domain>
Comment 1 Daniel Berrange 2009-06-08 07:01:13 EDT
Can you attach the /var/log/libvirt/qemu/$VMNAME.log file after attempting to start a guest with this USB device attached.

Also can you provide the 'lsusb -v' output from the host OS.
Comment 2 Mark McLoughlin 2009-06-08 13:43:58 EDT
Also, the qemu version number
Comment 3 Bob Cochran 2009-06-08 16:58:04 EDT
Created attachment 346930 [details]
Requested log of /var/log/libvirt/qemu/Ubuntu9.log
Comment 4 Bob Cochran 2009-06-08 16:59:22 EDT
Created attachment 346931 [details]
Requested output of lsusb -v
Comment 5 Bob Cochran 2009-06-08 17:01:16 EDT
Qemu version:

[rlc@deafeng3 ~]$ rpm -q qemu
qemu-0.10.4-4.fc11.x86_64


I noticed the 'permission denied' error in the log file, but don't know how to fix it properly.

Thanks for working this bug for me. I am always happy to help in any way. Usually I get home from my day job in the evenings, Eastern Standard Time.

Bob
Comment 6 Mark McLoughlin 2009-06-09 03:31:36 EDT
Any SELinux errors? Try "ausearch -m AVC -ts recent". Does it work in permissive mode?

(See bug #499259 for how sVirt breaks KVM PCI passthrough)
Comment 7 Daniel Berrange 2009-06-09 05:42:47 EDT
Yeah, I expect it'll be being denied by SELinux. For security reasons we don't allow QEMU to access any host USB or PCI devices with the default policy. So you'll likely have to add a custom policy extension, or put it into permissive.
Comment 8 Bob Cochran 2009-06-09 20:58:42 EDT
I changed my SELinux to run in permissive mode, set it to relabel, rebooted to accomplish the relabel, and tried to pass the FTDI device to my Ubuntu guest again. It worked and I was rewarded with the pleasure of being able to upload an Arduino sketch to one of the Freeduino boards I had built (a "Really Bare Bones Board" purchased as a kit from wulfden.org). The guest did this entirely on it's own! So I am really pleased. I did set the wrong board type at first, but once I corrected it, I was able to send a small sketch to the Atmega chip.

I'm going to attach output from that, including the guest's log messages and the hosts log messages, for your benefit.

I have not given much thought to the perspectives that led to denying qemu access to host USB or PCI devices. However, I've been running in SELinux enforcing mode with very few problems for a long time now, and I like that. Can you please tell me how to write a custom policy extension that will allow USB devices of my choice such access? In fact I would prefer that every hardware device I add to the domain xml description simply be passed through to the guest and enabled as fully functional. I need the device access. 

Mark, I forgot to run your ausearch request before going into permissive mode and relabelling. I apologize. Here is the output after:

# ausearch -m AVC -ts recent
<no matches>

As mentioned, a bunch of attachments will follow.

Thanks a lot for quickly looking into this bug. I especially appreciate your help because I know you have a large number of bugs to deal with and you really put in some extra time and effort to help me.

Thanks, Bob
Comment 9 Bob Cochran 2009-06-09 21:07:01 EDT
Created attachment 347119 [details]
From /var/log/libvirt/qemu/Ubuntu9.log, log file of device passthrough occuring in SELinux permissive mode
Comment 10 Bob Cochran 2009-06-09 21:08:58 EDT
Created attachment 347120 [details]
Messages from /var/log/messages on Fedora 11 host machine, recorded from just before plugging device in to when device is unplugged from host.
Comment 11 Bob Cochran 2009-06-09 21:10:35 EDT
Created attachment 347121 [details]
Log messages recorded in the Ubuntu 9.04 guest machine for the same FTDI uart device
Comment 12 Bob Cochran 2009-06-09 21:12:44 EDT
Created attachment 347122 [details]
Console messages reflecting successful use of FTDI uart device from the Ubuntu 9.04 guest's "Arduino-0016" application
Comment 13 Mark McLoughlin 2009-06-19 06:13:54 EDT
See bug #499259 for a similar issue with PCI passthrough

Error message in enforcing mode was:

  /proc/bus/usb/007/002: Permission denied
  Warning: could not add USB device host:0403:6001

Bob: when you run the guest in permissive mode, you should be seeing selinux AVC messages; could you post them here too?
Comment 14 Daniel Walsh 2009-06-19 07:26:47 EDT
I think this is fixed with selinux-policy-3.6.12-54.fc11

I added a boolean to allow svirt to interact with usbfs

SO if you download this package and install it, and then 
set the boolea virt_use_usb it should work.


# setsebool -P virt_use_usb 1
Comment 15 Bob Cochran 2009-06-19 10:17:57 EDT
I'm at selinux-policy-3.6.12-45.fc11.noarch. 

Mark: I am still running this machine in permissive mode. I'll connect the UART in a moment and look for AVC denials. These usually show up as iconized warning balloons at the top right of my Gnome Desktop and I didn't see any of those the last time I tried running my Ubuntu guest with the uart connected. I will try this again in a moment and check /var/log/messages for AVC denials.

Dan: When I install the updated selinux-policy you mention, I assume this means I can switch back to enforcing mode, relabel my drive, then run `setsebool` as you indicate and the Ubuntu guest should have access to the device. Correct me if my procedure is wrong. 

Thanks a lot, Mark and Dan!

Bob
Comment 16 Bob Cochran 2009-06-19 10:35:56 EDT
In response to Mark's request:

At here are the messages that show in /var/log/messages once I start the graphical Virtual Machine Manager and then start my Ubuntu guest and log into it. Prior to this point, the USB uart device I wish to pass to the Ubuntu guest has already been plugged in to a port on the host itself (as opposed to a port on a powered or unpowered USB hub which is connected to the host.)

I don't see anything that looks like AVC denials here, but I could have bad eyes or a senior moment. The NetWorkManager messages are interesting. At the very bottom of these messages you can see the point where I shut down the Ubuntu guest and then closed Virtual Machine Manager. 

Again, this is done while running in SELinux permissive mode.


-------------------------------------------------------------

Jun 19 10:22:07 deafeng3 libvirtd: 10:22:07.747: warning : Sound cards for VMs are disabled while SELinux security model is active
Jun 19 10:22:07 deafeng3 kernel: tun: Universal TUN/TAP device driver, 1.6
Jun 19 10:22:07 deafeng3 kernel: tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
Jun 19 10:22:07 deafeng3 kernel: device vnet0 entered promiscuous mode
Jun 19 10:22:07 deafeng3 kernel: virbr0: topology change detected, propagating
Jun 19 10:22:07 deafeng3 kernel: virbr0: port 1(vnet0) entering forwarding state
Jun 19 10:22:07 deafeng3 NetworkManager: nm_device_ethernet_new: assertion `driver != NULL' failed
Jun 19 10:22:08 deafeng3 kernel: ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
Jun 19 10:22:08 deafeng3 kernel: ftdi_sio 7-1:1.0: device disconnected
Jun 19 10:22:08 deafeng3 avahi-daemon[1882]: Registering new address record for fe80::d418:54ff:fefe:584e on vnet0.*.
Jun 19 10:22:10 deafeng3 ntpd[2163]: Listening on interface #8 vnet0, fe80::d418:54ff:fefe:584e#123 Enabled
Jun 19 10:22:12 deafeng3 nm-system-settings: Added default wired connection 'Auto vnet0' for /org/freedesktop/Hal/devices/net_d6_18_54_fe_58_4e
Jun 19 10:22:16 deafeng3 kernel: kvm: emulating exchange as write
Jun 19 10:22:16 deafeng3 kernel: usb 7-1: reset full speed USB device using uhci_hcd and address 2
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio 7-1:1.0: FTDI USB Serial Device converter detected
Jun 19 10:22:17 deafeng3 kernel: usb 7-1: Detected FT232RL
Jun 19 10:22:17 deafeng3 kernel: usb 7-1: FTDI USB Serial Device converter now attached to ttyUSB0
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio 7-1:1.0: device disconnected
Jun 19 10:22:17 deafeng3 usb_id[3635]: unable to access '/devices/pci0000:00/0000:00:1d.1/usb7/7-1/7-1:1.0/ttyUSB0/tty/ttyUSB0'
Jun 19 10:22:17 deafeng3 NetworkManager: <WARN>  libudev_get_modem_capabilities(): couldn't inspect device '/sys/devices/pci0000:00/0000:00:1d.1/usb7/7-1/7-1:1.0/ttyUSB0/tty/ttyUSB0' with libudev
Jun 19 10:22:17 deafeng3 NetworkManager: <WARN>  modem_device_creator(): (ttyUSB0): udev probing failed; using only HAL modem capabilities.
Jun 19 10:22:17 deafeng3 NetworkManager: <info>  (ttyUSB0): ignoring due to lack of mobile broadband capabilties
Jun 19 10:22:17 deafeng3 kernel: usb 7-1: reset full speed USB device using uhci_hcd and address 2
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio 7-1:1.0: FTDI USB Serial Device converter detected
Jun 19 10:22:17 deafeng3 kernel: usb 7-1: Detected FT232RL
Jun 19 10:22:17 deafeng3 kernel: usb 7-1: FTDI USB Serial Device converter now attached to ttyUSB0
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
Jun 19 10:22:17 deafeng3 kernel: ftdi_sio 7-1:1.0: device disconnected
Jun 19 10:22:17 deafeng3 usb_id[3645]: unable to access '/devices/pci0000:00/0000:00:1d.1/usb7/7-1/7-1:1.0/ttyUSB0/tty/ttyUSB0'
Jun 19 10:22:17 deafeng3 NetworkManager: <WARN>  libudev_get_modem_capabilities(): couldn't inspect device '/sys/devices/pci0000:00/0000:00:1d.1/usb7/7-1/7-1:1.0/ttyUSB0/tty/ttyUSB0' with libudev
Jun 19 10:22:17 deafeng3 NetworkManager: <WARN>  modem_device_creator(): (ttyUSB0): udev probing failed; using only HAL modem capabilities.
Jun 19 10:22:17 deafeng3 NetworkManager: <info>  (ttyUSB0): ignoring due to lack of mobile broadband capabilties
Jun 19 10:22:54 deafeng3 dnsmasq[2637]: DHCPDISCOVER(virbr0) 54:52:00:74:32:2f 
Jun 19 10:22:54 deafeng3 dnsmasq[2637]: DHCPOFFER(virbr0) 192.168.122.122 54:52:00:74:32:2f 
Jun 19 10:22:54 deafeng3 dnsmasq[2637]: DHCPREQUEST(virbr0) 192.168.122.122 54:52:00:74:32:2f 
Jun 19 10:22:54 deafeng3 dnsmasq[2637]: DHCPACK(virbr0) 192.168.122.122 54:52:00:74:32:2f bobubuntu9
Jun 19 10:25:07 deafeng3 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Jun 19 10:25:07 deafeng3 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Jun 19 10:25:07 deafeng3 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Jun 19 10:25:07 deafeng3 pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Jun 19 10:27:24 deafeng3 avahi-daemon[1882]: Withdrawing address record for fe80::d418:54ff:fefe:584e on vnet0.
Jun 19 10:27:24 deafeng3 kernel: virbr0: port 1(vnet0) entering disabled state
Jun 19 10:27:24 deafeng3 kernel: device vnet0 left promiscuous mode
Jun 19 10:27:24 deafeng3 kernel: virbr0: port 1(vnet0) entering disabled state
Jun 19 10:27:26 deafeng3 ntpd[2163]: Deleting interface #8 vnet0, fe80::d418:54ff:fefe:584e#123, interface stats: received=0, sent=0, dropped=0, active_time=316 secs
Comment 17 Daniel Walsh 2009-06-19 11:08:40 EDT
No relabel required.

If you set the boolean with the package in Koji you should be able to run in enforcing mode.

The SELinux error messages would go in /var/log/audit/audit.log
Comment 18 Mark McLoughlin 2009-06-22 07:10:01 EDT
Bob: could you confirm that USB passthrough now works in enforcing mode with the virt_use_usb boolean enabled?
Comment 19 Bob Cochran 2009-06-22 20:53:14 EDT
Created attachment 349020 [details]
Contains text files showing kernel messages on both host and guest 

The tarball contains text files dated 2009-06-19 and 2009-06-22. The 2009-06-19 files reflect kernel messages and avc denial messages I had prior to installing Dan's selinux-policy and selinux-policy-targeted packages on koji. They also respond to Mark's request for the denial messages. The files dated 2009-06-22 show the kernel messages that result after installing Dan's packages. One of the files shows the versions I installed, but here they are gratis: selinux-policy-targeted-3.6.12-56.fc11.noarch and selinux-policy-3.6.12-56.fc11.noarch. Installing these packages does seem to allow USB device passthrough to a virtual guest when the host is running in enforcing mode. See my further comments below.
Comment 20 Bob Cochran 2009-06-22 21:02:45 EDT
I'm very pleased, installing 

selinux-policy-targeted-3.6.12-56.fc11.noarch
selinux-policy-3.6.12-56.fc11.noarch

from Koji does seem to work, thanks a ton Dan! I have not tried to actually use the uart device yet. But I promise I will this week and report back. I'm pretty excited about this and I'm very grateful for your help. Thanks a lot!
Comment 21 Mark McLoughlin 2009-06-23 10:38:12 EDT
Thanks for confirming Bob
Comment 22 Mark McLoughlin 2009-09-03 08:08:00 EDT
This is long since fixed
Comment 23 Erik Logtenberg 2010-03-30 13:53:28 EDT
My apologies for re-opening this old bug, but I seem to run into the same issue: i can't get USB pass-through to work with SELinux in enforcing mode.

My SELinux packages are way newer than the ones mentioned earlier in the original bugreport, so should contain the fix... but this doesn't seem to be so.

selinux-policy-3.6.32-103.fc12.noarch
selinux-policy-targeted-3.6.32-103.fc12.noarch

On a fully updated Fedora 12 x86_64 install.

I already set virt_manage_sysfs on and also virt_use_usb on. Nevertheless it doesn't work. Qemu doesn't have access.

As soon as I put SELinux in permissive mode, the guest immediately gains access to the device, but the very second that I put SELinux back in enforcing mode, the USB device goes dead again in the guest.

Please note that I also chowned the /dev/bus/usb/xxx/xxx device to qemu:qemu, because by default it is owned by root:root, having permissions: crw-rw-r--. This way qemu can't access the device even with SELinux disabled. I haven't heard anyone before about this issue, so I wonder how did all the other people in this bugreport handle this?
Comment 24 Daniel Walsh 2010-03-30 16:33:09 EDT
Do you see any avc messages?
Comment 25 Erik Logtenberg 2010-03-30 18:20:51 EDT
When I use the ausearch command in the same way other people in this bug report did, then I get no results:

# ausearch -m AVC -ts recent
<no matches>

However, my /var/log/messages fills up with lots of messages like these:

Mar 31 00:18:13 bak kernel: type=1400 audit(1269987493.598:17379): avc:  denied  { ioctl } for  pid=27001 comm="qemu-kvm" path="/dev/bus/usb/002/024" dev=devtmpfs ino=248675106 scontext=system_u:system_r:svirt_t:s0:c53,c743 tcontext=system_u:object_r:usb_device_t:s0 tclass=chr_file                                                        
Mar 31 00:18:13 bak kernel: type=1400 audit(1269987493.598:17380): avc:  denied  { read write } for  pid=27001 comm="qemu-kvm" path="/dev/bus/usb/002/024" dev=devtmpfs ino=248675106 scontext=system_u:system_r:svirt_t:s0:c53,c743 tcontext=system_u:object_r:usb_device_t:s0 tclass=chr_file
Comment 26 Daniel Walsh 2010-03-31 10:26:56 EDT
That means libvirt did not label it correctly
Comment 27 Erik Logtenberg 2010-04-07 03:51:13 EDT
Is there any additional information that I can provide to help fix this issue? If so, please let me know.
Comment 28 Daniel Walsh 2010-04-07 10:10:00 EDT
*** Bug 579744 has been marked as a duplicate of this bug. ***
Comment 29 Erik Logtenberg 2010-04-14 08:15:43 EDT
In the mean time, is there a way for me to work around this bug? For instance by manually labelling the device for the time being.
What is the SELinux label that the device should have? (I'm not very familiar with SELinux labels, what is the exact command that I should enter to set the right label?)
Thanks in advance!
Comment 30 Daniel Walsh 2010-04-14 09:55:15 EDT
You could label the device to match your svirt image

chcon -t svirt_image_t -l MCSLEVELFROMSVIRT_T /dev/bus/usb/002/024

Or you can just add rules to allow svirt_t to read/write usb_devices.


# grep avc /var/log/audit/audit.log | audit2allow -M myvirt
# semodule -i myvirt.pp
Comment 31 Cole Robinson 2010-05-12 11:31:03 EDT
There is another workaround for this issue, it is manually specifying the USB device in the guest XML by bus:addr, which libvirt in F12 knows how to relabel.

Use 'lsusb' to find the device you want:

Bus 012 Device 345: ID 062a:0001 Creative Labs Notebook Optical Mouse

As root, use 'EDITOR=gedit virsh edit YOUR-VM-NAME' and add the following XML to the <devices> block

    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <address bus='12' device='345'/>
      </source>
    </hostdev>

Make sure you remove any leading 0 from the bus value returned by lsusb. Then start your guest as usual.

The downside to this, is that a USB devices bus/dev numbers are not stable: unplugging and replugging the device won't give it the same values, so you might have to edit the XML again if you replug the device on the host machine
Comment 32 Cole Robinson 2010-05-17 15:06:06 EDT
*** Bug 572354 has been marked as a duplicate of this bug. ***
Comment 33 Cole Robinson 2010-05-17 15:06:12 EDT
*** Bug 579728 has been marked as a duplicate of this bug. ***
Comment 34 Cole Robinson 2010-05-17 16:09:02 EDT
There is another (slimmer) bug tracking this issue. Duping there.

*** This bug has been marked as a duplicate of bug 537227 ***