Hot-unplug scsi storage controller under switch is also failed.

Didn't hit the issue with comment0 steps 
kernel-3.10.0-382.el7.x86_64
qemu-kvm-rhev-2.5.0-4.el7.x86_64 
seabios-1.9.1-2.el7.x86_64

As noted in <https://github.com/tianocore/edk2/issues/32#issuecomment-195038183>, the PCIe hotplug fixes (with Marcel's guidance) for OVMF are the upstream commit range 7e869ee..7daf240.

The downstream OVMF version used by QE for this BZ is d7c0dfa, which does not contain the above upstream commit range. (The upstream commits are dated about a month later than the d7c0dfa downstream fork point.)

I've not been backporting upstream OVMF patches to downstream partly due to lack of capacity, partly due to OVMF being Tech Preview. Instead of backports, I've been doing regular rebases (once per 4-7 weeks approximately). Once OVMF becomes fully supported by us, this could (... likely will) change, with the package becoming more stable (less frequent rebases, more backports).

For now, a new rebase for downstream OVMF has been pending for more than a month. That rebase ("ovmf-20160419-1.git90bb4c5.el7") will contain the above-noted commit range. RHEL-7 distro bug 1329559 has been blocking the rebase from getting an official build, unfortunately.

If you think it's appropriate, feel free to reassign this bug to the OVMF component. In that case however:

(1) Please retest the issue first against the pending OVMF rebase.
    (I'll provide a long-term link for the Brew build in the next comment.)

(2) Please make this BZ depend on bug 1329559 explicitly.

Thanks
Laszlo

Created attachment 1172856 [details]
"lspci" after hot-plug cd

Created attachment 1172857 [details]
"tailf /var/log/messages" in the guest

Created attachment 1172858 [details]
OVMF debug log

OVMF does not allocate IO port apertures for bridges (for example: PCI downstream ports and root ports) that have no devices with IO BARs behind them at boot.

Normally this behavior is welcome (it conserves IO port space), but based on the guest dmesg (comment 14) this time around it causes problems. The guest kernel complains that it cannot satisfy the IO port reguirements of the (legacy) virtio-scsi-pci device.

I have no clue how this is supposed to work even in theory. If we convince the edk2 PCI bus driver in OVMF somehow to allocate the 4KB IO aperture for *all* bridges (in expectation of PCIe hotplug), then we'll run out of IO port space super quick, just with a low number of bridges / downstream ports / root ports.

I have zero good ideas here. OVMF preserves IO port space, and therefore it runs into this BZ. Whereas SeaBIOS allocates IO port space generously (so PCIe hotplug works), but then again, SeaBIOS runs into bug 1271457.

I'm thinking "you can either have your cake, or eat it", but maybe I just don't know enough about this stuff...

In summary: while I don't know how to change the behavior of the edk2 PCI bus driver in regard to this, I'm not even sure we should *try*.

FWIW, in "MdeModulePkg/MdeModulePkg.dec", we have the following default (inherited by OVMF without changes):

>   ## Indicates if PciBus driver supports the hot plug device.<BR><BR>
>   #   TRUE  - PciBus driver supports the hot plug device.<BR>
>   #   FALSE - PciBus driver doesn't support the hot plug device.<BR>
>   # @Prompt Enable PciBus hot plug device support.
>   gEfiMdeModulePkgTokenSpaceGuid.PcdPciBusHotplugDeviceSupport|TRUE|BOOLEAN|0x0001003d

* Can you please test the following case instead / in addition: please make sure that the virtio-scsi-pci device being hot-plugged is a modern device (--> disable-legacy=on,disable-modern=off). In this case it should only require MMIO resources, which -- I hope? -- shouldn't cause problems.

* Another tip (inspired by <https://github.com/tianocore/edk2/issues/32>): what happens if you try to hotplug the device (legacy, as reported, not modern) into a root port (ioh3420) rather than a downstream port? At least according to the upstream tracker item, that should work. Thanks.

> * Can you please test the following case instead / in addition: please make
> sure that the virtio-scsi-pci device being hot-plugged is a modern device
> (--> disable-legacy=on,disable-modern=off). In this case it should only
> require MMIO resources, which -- I hope? -- shouldn't cause problems.

Do not hot-plug the CD-ROM successfully but the guest does not hang.

{"execute":"__com.redhat_drive_add","arguments":{"file":"/home/cd.iso","format":"raw","id":"cd","media":"cdrom"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","bus":"downstream2","id":"scsi_pci_bus1","disable-legacy":true,"disable-modern":false}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-disk","drive":"cd","id":"cd0"}}
{"return": {}}

OVMF debug log as attachment "modern device hotplug".

"lspci" in guest:
#lspci
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio Controller (rev 01)
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:05.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.4 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
02:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:00.0 SCSI storage controller: Red Hat, Inc Virtio SCSI
04:00.0 SCSI storage controller: Red Hat, Inc Device 1048 (rev 01)

"tailf /var/log/messages" in guest:
Jun 29 04:23:11 dhcp-10-78 kernel: pciehp 0000:02:01.0:pcie24: Button pressed on Slot(0-1)
Jun 29 04:23:11 dhcp-10-78 kernel: pciehp 0000:02:01.0:pcie24: Card present on Slot(0-1)
Jun 29 04:23:11 dhcp-10-78 kernel: pciehp 0000:02:01.0:pcie24: PCI slot #0-1 - powering on due to button press
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: no space for [mem size 0x01000000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: failed to assign [mem size 0x01000000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: no space for [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: failed to assign [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: no space for [mem size 0x00c00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: failed to assign [mem size 0x00c00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: no space for [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: failed to assign [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 4: no space for [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 4: failed to assign [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 1: no space for [mem size 0x00001000]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 1: failed to assign [mem size 0x00001000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Jun 29 04:23:12 dhcp-10-78 journal: Device 0000:04:00.0 not found: could not access /sys/bus/pci/devices/0000:04:00.0/config: No such file or directory
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0:   bridge window [mem 0x98400000-0x985fffff 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:00.0: PCI bridge to [bus 03]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:01:00.0: PCI bridge to [bus 02-04]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x01000000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x01000000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: no space for [mem size 0x00e00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: failed to assign [mem size 0x00e00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: no space for [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: failed to assign [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: no space for [mem size 0x00c00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 14: failed to assign [mem size 0x00c00000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: no space for [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: BAR 13: failed to assign [io  size 0x1000]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 4: no space for [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 4: failed to assign [mem size 0x00800000 64bit pref]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 1: no space for [mem size 0x00001000]
Jun 29 04:23:12 dhcp-10-78 kernel: pci 0000:04:00.0: BAR 1: failed to assign [mem size 0x00001000]
Jun 29 04:23:12 dhcp-10-78 kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Jun 29 04:23:12 dhcp-10-78 kernel: virtio-pci 0000:04:00.0: virtio_pci: leaving for legacy driver
Jun 29 04:23:59 dhcp-10-78 dbus-daemon: dbus[721]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Jun 29 04:23:59 dhcp-10-78 dbus[721]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Jun 29 04:23:59 dhcp-10-78 dbus-daemon: dbus[721]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Jun 29 04:23:59 dhcp-10-78 dbus[721]: [system] Successfully activated service 'com.redhat.SubscriptionManager'




> * Another tip (inspired by <https://github.com/tianocore/edk2/issues/32>):
> what happens if you try to hotplug the device (legacy, as reported, not
> modern) into a root port (ioh3420) rather than a downstream port? At least
> according to the upstream tracker item, that should work. Thanks.

Hotplug CD-ROM into a root port succeed.

-device ioh3420,bus=pcie.0,id=root.1,slot=2 \

{"execute":"__com.redhat_drive_add","arguments":{"file":"/home/cd.iso","format":"raw","id":"cd","media":"cdrom"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","bus":"root.1","id":"scsi_pci_bus1"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-disk","drive":"cd","id":"cd0"}}
{"return": {}}

OVMF debug log as attachment "hotplug into root port".

"tailf /var/log/messages" in guest:

Jun 29 03:58:22 dhcp-10-78 kernel: pciehp 0000:00:05.0:pcie04: Button pressed on Slot(2)
Jun 29 03:58:22 dhcp-10-78 kernel: pciehp 0000:00:05.0:pcie04: Card present on Slot(2)
Jun 29 03:58:22 dhcp-10-78 kernel: pciehp 0000:00:05.0:pcie04: PCI slot #2 - powering on due to button press
Jun 29 03:58:23 dhcp-10-78 kernel: pci 0000:05:00.0: BAR 1: assigned [mem 0x98600000-0x98600fff]
Jun 29 03:58:23 dhcp-10-78 kernel: pci 0000:05:00.0: BAR 0: assigned [io  0x1000-0x103f]
Jun 29 03:58:23 dhcp-10-78 kernel: pcieport 0000:00:05.0: PCI bridge to [bus 05]
Jun 29 03:58:23 dhcp-10-78 kernel: pcieport 0000:00:05.0:   bridge window [io  0x1000-0x1fff]
Jun 29 03:58:23 dhcp-10-78 journal: Device 0000:05:00.0 not found: could not access /sys/bus/pci/devices/0000:05:00.0/config: No such file or directory
Jun 29 03:58:23 dhcp-10-78 kernel: pcieport 0000:00:05.0:   bridge window [mem 0x98600000-0x987fffff]
Jun 29 03:58:23 dhcp-10-78 kernel: pcieport 0000:00:05.0:   bridge window [mem 0x98800000-0x989fffff 64bit pref]
Jun 29 03:58:23 dhcp-10-78 kernel: virtio-pci 0000:05:00.0: enabling device (0000 -> 0003)
Jun 29 03:58:23 dhcp-10-78 kernel: virtio-pci 0000:05:00.0: virtio_pci: leaving for legacy driver
Jun 29 03:58:23 dhcp-10-78 kernel: scsi host7: Virtio SCSI HBA
Jun 29 03:58:57 dhcp-10-78 kernel: scsi 7:0:0:0: CD-ROM            QEMU     QEMU CD-ROM      2.5+ PQ: 0 ANSI: 5
Jun 29 03:58:57 dhcp-10-78 kernel: scsi 7:0:0:0: Attached scsi generic sg1 type 5
Jun 29 03:58:57 dhcp-10-78 kernel: sr 7:0:0:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 cdda tray
Jun 29 03:58:57 dhcp-10-78 kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Jun 29 03:58:57 dhcp-10-78 dbus-daemon: dbus[715]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
Jun 29 03:58:57 dhcp-10-78 dbus[715]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
Jun 29 03:58:57 dhcp-10-78 systemd: Starting Hostname Service...
Jun 29 03:58:57 dhcp-10-78 dbus[715]: [system] Successfully activated service 'org.freedesktop.hostname1'
Jun 29 03:58:57 dhcp-10-78 dbus-daemon: dbus[715]: [system] Successfully activated service 'org.freedesktop.hostname1'
Jun 29 03:58:57 dhcp-10-78 systemd: Started Hostname Service.
Jun 29 03:58:57 dhcp-10-78 udisksd[2935]: Mounted /dev/sr0 at /run/media/root/CDROM on behalf of uid 0
Jun 29 03:59:05 dhcp-10-78 gnome-session: (nautilus:3431): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -15 and height 34
Jun 29 03:59:08 dhcp-10-78 dbus-daemon: dbus[715]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Jun 29 03:59:08 dhcp-10-78 dbus[715]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper)
Jun 29 03:59:08 dhcp-10-78 dbus[715]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Jun 29 03:59:08 dhcp-10-78 dbus-daemon: dbus[715]: [system] Successfully activated service 'com.redhat.SubscriptionManager'
Jun 29 04:00:01 dhcp-10-78 systemd: Started Session 2 of user root.
Jun 29 04:00:01 dhcp-10-78 systemd: Starting Session 2 of user root.
Jun 29 04:01:01 dhcp-10-78 systemd: Started Session 3 of user root.
Jun 29 04:01:01 dhcp-10-78 systemd: Starting Session 3 of user root.

Created attachment 1173723 [details]
modern device hotplug

Created attachment 1173724 [details]
hotplug into root port

Upstream query: http://thread.gmane.org/gmane.comp.bios.edk2.devel/13923

Posted upstream series: http://thread.gmane.org/gmane.comp.bios.edk2.devel/13952

Posted upstream v2 series:
http://thread.gmane.org/gmane.comp.bios.edk2.devel/14344

Upstream commit range 2eb358986052..8aba40b79267.

Fix included in ovmf-20160608-3.git988715a.el7

1. Reproduce with OVMF-20160419-2.git90bb4c5.el7.noarch.rpm and qemu-kvm-rhev-2.5.0-4.el7.x86_64, same result with comment12

2. Tested it on OVMF-20160608-3.git988715a.el7.noarch.rpm and qemu-kvm-rhev-2.6.0-19.el7.x86_64, hit another issue
  1) no respone when input command in the guest (like "lsblk" "lspci -vvv -t" "ifconfig")

  2) qemu core dump when quit directly in hmp or shutdown guest after "system_reset"

(qemu) red_channel_client_disconnect_dummy: rcc=0x7fb2bcbd1000 (channel=0x7fb2bbb84940 type=5 id=0)
qemu: qemu_mutex_lock: Invalid argument
bug1332408: line 40: 24736 Aborted                 (core dumped) /usr/libexec/qemu-kvm -S -name 'rhel7.3-64' -machine q35,accel=kvm,usb=off,vmport=off -drive file=/usr/share/OVMF/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/home/OVMF_VARS.fd,if=pflash,format=raw,unit=1 -m 4096 -smp 4,maxcpus=4,cores=2,threads=2,sockets=1 -cpu SandyBridge,enforce -rtc base=localtime,clock=host,driftfix=slew -nodefaults -vga qxl -device AC97,bus=pcie.0 -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=idSWJ5gV -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait -device isa-serial,chardev=serial_id_serial0 -chardev socket,id=seabioslog_id_20151214-111528-C6FB1EaX,path=/tmp/seabios-20151214-111528-C6FB1EaX,server,nowait -device isa-debugcon,chardev=seabioslog_id_20151214-111528-C6FB1EaX,iobase=0x402 -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 -device usb-tablet,id=usb-tablet1 -netdev tap,id=netdev0,vhost=on -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0,bootindex=3,disable-modern=off,disable-legacy=off -device ioh3420,bus=pcie.0,id=root.0,slot=1 -device x3130-upstream,bus=root.0,id=upstream1 -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 -device virtio-scsi-pci,bus=downstream1,id=scsi_pci_bus0 -drive file=/home/pxb-ovmf.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=1,physical_block_size=512,logical_block_size=512,serial=12345678900987654321,ver=SYSDISK,wwn=0x123,channel=0,scsi-id=0,lun=0 -enable-kvm -monitor stdio -spice port=5900,disable-ticketing -qmp tcp:0:6666,server,nowait


Following is the detailed steps:
1) Boot guest with following cmd

/usr/libexec/qemu-kvm \
 -S \
 -name 'rhel7.3-64' \
 -machine q35,accel=kvm,usb=off,vmport=off \
 -drive file=/usr/share/OVMF/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on \
 -drive file=/home/OVMF_VARS.fd,if=pflash,format=raw,unit=1 \
 -m 4096 \
 -smp 4,maxcpus=4,cores=2,threads=2,sockets=1 \
 -cpu SandyBridge,enforce \
 -rtc base=localtime,clock=host,driftfix=slew \
 -nodefaults \
 -vga qxl \
 -device AC97,bus=pcie.0 \
 -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \
 -mon chardev=qmp_id_qmpmonitor1,mode=control \
 -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \
 -mon chardev=qmp_id_catch_monitor,mode=control \
 -device pvpanic,ioport=0x505,id=idSWJ5gV \
 -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \
 -device isa-serial,chardev=serial_id_serial0 \
 -chardev socket,id=seabioslog_id_20151214-111528-C6FB1EaX,path=/tmp/seabios-20151214-111528-C6FB1EaX,server,nowait \
 -device isa-debugcon,chardev=seabioslog_id_20151214-111528-C6FB1EaX,iobase=0x402 \
 -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \
 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \
 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \
 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \
 -device usb-tablet,id=usb-tablet1 \
 -netdev tap,id=netdev0,vhost=on \
 -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0,bootindex=3,disable-modern=off,disable-legacy=off \
 -device ioh3420,bus=pcie.0,id=root.0,slot=1 \
 -device x3130-upstream,bus=root.0,id=upstream1 \
 -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
 -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
 -device virtio-scsi-pci,bus=downstream1,id=scsi_pci_bus0 \
 -drive file=/home/pxb-ovmf.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \
 -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=1,physical_block_size=512,logical_block_size=512,serial=12345678900987654321,ver=SYSDISK,wwn=0x123,channel=0,scsi-id=0,lun=0 \
 -enable-kvm \
 -monitor stdio \
 -spice port=5900,disable-ticketing \
 -qmp tcp:0:6666,server,nowait

2) check info in hmp
(qemu) info block
pflash0 (#block159): /usr/share/OVMF/OVMF_CODE.secboot.fd (raw, read-only)
    Cache mode:       writeback

pflash1 (#block372): /home/OVMF_VARS.fd (raw)
    Cache mode:       writeback

drive_sysdisk (#block523): /home/pxb-ovmf.qcow2 (qcow2)
    Cache mode:       writeback, direct

3) hot plug scsi controller in qmp
{"execute":"qmp_capabilities"}
{"return": {}}
{"timestamp": {"seconds": 1470896828, "microseconds": 604319}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/machine/peripheral/net0/virtio-backend"}}

{"execute":"__com.redhat_drive_add","arguments":{"file":"/home/virtio-win.iso","format":"raw","id":"cd","media":"cdrom"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","bus":"downstream2","id":"scsi_pci_bus1"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-disk","drive":"cd","id":"cd0"}}
{"return": {}}

4) check the block info in hmp
(qemu) info block
pflash0 (#block159): /usr/share/OVMF/OVMF_CODE.secboot.fd (raw, read-only)
    Cache mode:       writeback

pflash1 (#block372): /home/OVMF_VARS.fd (raw)
    Cache mode:       writeback

drive_sysdisk (#block523): /home/pxb-ovmf.qcow2 (qcow2)
    Cache mode:       writeback, direct

cd (#block772): /home/virtio-win.iso (raw, read-only)
    Removable device: not locked, tray closed
    Cache mode:       writeback


5) In guest, execute some command and no response

6) quit in hmp and qemu core dump
[root@localhost ~]# tailf /var/log/messages 
Aug 11 22:27:53 localhost systemd: Starting Session 2 of user root.
Aug 11 22:27:53 localhost dbus-daemon: dbus[734]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Aug 11 22:27:53 localhost dbus[734]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Aug 11 22:27:53 localhost dbus[734]: [system] Successfully activated service 'org.freedesktop.problems'
Aug 11 22:27:53 localhost dbus-daemon: dbus[734]: [system] Successfully activated service 'org.freedesktop.problems'
Aug 11 22:27:54 localhost systemd-logind: Removed session c1.
Aug 11 22:27:54 localhost systemd: Removed slice user-42.slice.
Aug 11 22:27:54 localhost systemd: Stopping user-42.slice.
Aug 11 22:27:55 localhost dhclient[3548]: DHCPDISCOVER on virbr0 to 255.255.255.255 port 67 interval 15 (xid=0x4008b8f9)
Aug 11 22:27:58 localhost fprintd: ** Message: No devices in use, exit
Aug 11 22:28:00 localhost dbus-daemon: dbus[734]: [system] Failed to activate service 'org.bluez': timed out
Aug 11 22:28:00 localhost dbus[734]: [system] Failed to activate service 'org.bluez': timed out
Aug 11 22:28:00 localhost pulseaudio: GetManagedObjects() failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Aug 11 22:28:10 localhost dhclient[3548]: DHCPDISCOVER on virbr0 to 255.255.255.255 port 67 interval 15 (xid=0x4008b8f9)
Aug 11 22:28:11 localhost dhclient[3548]: DHCPDISCOVER on virbr0-nic to 255.255.255.255 port 67 interval 13 (xid=0x7f86917c)
Aug 11 22:28:13 localhost kernel: pciehp 0000:02:01.0:pcie24: Button pressed on Slot(0-1)
Aug 11 22:28:13 localhost kernel: pciehp 0000:02:01.0:pcie24: Card present on Slot(0-1)
Aug 11 22:28:13 localhost kernel: pciehp 0000:02:01.0:pcie24: PCI slot #0-1 - powering on due to button press
Aug 11 22:28:14 localhost journal: libvirt version: 2.0.0, package: 3.el7 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2016-07-22-10:33:43, x86-024.build.eng.bos.redhat.com)
Aug 11 22:28:14 localhost journal: hostname: localhost.localdomain
Aug 11 22:28:14 localhost journal: Device 0000:04:00.0 not found: could not access /sys/bus/pci/devices/0000:04:00.0/config: No such file or directory
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x01000000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x01000000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 4: no space for [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 4: failed to assign [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 1: assigned [mem 0x98000000-0x98000fff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0:   bridge window [io  0x6000-0x6fff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0:   bridge window [mem 0x98000000-0x981fffff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:00.0: PCI bridge to [bus 03]
Aug 11 22:28:14 localhost kernel: pcieport 0000:01:00.0: PCI bridge to [bus 02-04]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x01000000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x01000000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 14: no space for [mem size 0x00400000]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 14: failed to assign [mem size 0x00400000]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: no space for [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 15: failed to assign [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 14: assigned [mem 0x98000000-0x981fffff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: BAR 14: [mem 0x98000000-0x981fffff] (failed to expand by 0x200000)
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 4: no space for [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 4: failed to assign [mem size 0x00800000 64bit pref]
Aug 11 22:28:14 localhost kernel: pci 0000:04:00.0: BAR 1: assigned [mem 0x98000000-0x98000fff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0: PCI bridge to [bus 04]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0:   bridge window [io  0x6000-0x6fff]
Aug 11 22:28:14 localhost kernel: pcieport 0000:02:01.0:   bridge window [mem 0x98000000-0x981fffff]
Aug 11 22:28:14 localhost kernel: virtio-pci 0000:04:00.0: enabling device (0000 -> 0002)
Aug 11 22:28:14 localhost kernel: virtio-pci 0000:04:00.0: virtio_pci: leaving for legacy driver
Aug 11 22:28:24 localhost dhclient[3548]: DHCPDISCOVER on virbr0-nic to 255.255.255.255 port 67 interval 14 (xid=0x7f86917c)
Aug 11 22:28:25 localhost dhclient[3548]: DHCPDISCOVER on virbr0 to 255.255.255.255 port 67 interval 20 (xid=0x4008b8f9)
Aug 11 22:28:27 localhost systemd: Job sys-devices-virtual-misc-vmbus\x21hv_kvp.device/start timed out.
Aug 11 22:28:27 localhost systemd: Timed out waiting for device sys-devices-virtual-misc-vmbus\x21hv_kvp.device.
Aug 11 22:28:27 localhost systemd: Dependency failed for Hyper-V KVP daemon.
Aug 11 22:28:27 localhost systemd: Job hypervkvpd.service/start failed with result 'dependency'.
Aug 11 22:28:27 localhost systemd: Job sys-devices-virtual-misc-vmbus\x21hv_kvp.device/start failed with result 'timeout'.
Aug 11 22:28:27 localhost systemd: Job sys-devices-virtual-misc-vmbus\x21hv_fcopy.device/start timed out.
Aug 11 22:28:27 localhost systemd: Timed out waiting for device sys-devices-virtual-misc-vmbus\x21hv_fcopy.device.

Add info:

  No core file, so I didn't put the detailed core file info

  Laszlo, should I file a new issue for tracking the above issue or it's the same reason with this bz?

Thanks
Jing Zhao

Host kernel: 3.10.0-488.el7.x86_64
guest kernel: 3.10.0-481.el7.x86_64

Host kernel: 3.10.0-488.el7.x86_64
guest kernel: 3.10.0-481.el7.x86_64

Comment 27 brings up two separate problems that interfere with testing:

First,

> 2) qemu core dump when quit directly in hmp or shutdown guest after
> "system_reset"

I think this could be related to bug 1299876. See
- bug 1299876 comment 4
- bug 1361487 (most likely duplicate).

The crash that is experienced in comment 27 could be a duplicate, or just
something quite similar. Either way it's a qemu-kvm-rhev issue, so a
separate BZ would be justified for triaging at least.

Second,

> 5) In guest, execute some command and no response

On the surface this looks identical to the original report in comment 0.
However, from the dmesg in comment 27 we can see that now we only run out of
MEM resources, not IO.

This change has occurred because in comment 0,
"qemu-kvm-rhev-2.5.0-4.el7.x86_64" was used for testing, but in comment 27,
"qemu-kvm-rhev-2.6.0-19.el7.x86_64" is used.

The latter includes the fix for qemu-kvm-rhev bug 1360664 (a backport of
upstream commit 9a4c0e220d8a), which enables transitional (legacy + modern)
virtio by default.

Commit 9a4c0e220d8a changes the virtio behavior: it enables transitional
devices by default (= legacy + modern at the same time), which causes Linux
to want to allocate MEM resources for them. When I tested the OVMF patches,
only IO was necessary.

Note that upstream commit 9a4c0e220d8a was committed on 2016-07-28, ten days
*after* I posted my downstream OVMF series (see comment 24). I could not
expect the same QEMU command line to silently change behavior when I was
testing the OVMF patches.

So, the second issue is a duplicate of bug 1365619.


Here's what I propose:
- Regarding the crash, I think we should just stick with existent bug
  1361487.

- Regarding the original report, please retest
  "OVMF-20160608-3.git988715a.el7.noarch" with one of the following
  settings:
  - *either* use qemu-kvm-rhev-2.6.0-18 (note the build number: 18),
  - *or* use build -19, *but* explicitly specify
    "disable-legacy=off,disable-modern=on" on the device being hot-plugged
    (that is, make it a legacy-only device)

  The reason is that the OVMF patches should be verified against the
  *original* scope, that is, under the circumstances for which the issue was
  originally reported (= legacy only). If you change the circumstances
  asynchronously (= transitional), you can't expect the original fix to
  work!

  I realize we should ultimately make it "just work", but for that, we
  already have bug 1365619. Thanks.

For what is worth, I agree with Laszlo's assessment.

Thanks,
Marcel

*** Bug 1343301 has been marked as a duplicate of this bug. ***

Has verified, it has been resolved,so change its status to "Verified".

Verified version:
  Kernel Version:3.10.0-500.el7.x86_64
  qemu-kvm-version:qemu-kvm-rhev-2.6.0-22.el7.x86_64
  OVMF Version:OVMF-20160608-3.git988715a.el7.noarch

Verified Steps:
1.Start guest with cmds:
/usr/libexec/qemu-kvm \
-name 'rhel7.3-64' \
-boot menu=on,splash-time=12000 \
-machine q35,accel=kvm,usb=off,vmport=off \
-drive file=/usr/share/OVMF/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on \
-drive file=/usr/share/OVMF/OVMF_Client_VARS.fd,if=pflash,format=raw,unit=1 \
-m 4096 \
-smp 4,maxcpus=4,cores=2,threads=2,sockets=1 \
-cpu SandyBridge,enforce \
-rtc base=localtime,clock=host,driftfix=slew \
-nodefaults \
-vga qxl \
-device AC97,bus=pcie.0 \
-chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/mm,server,nowait \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \
-mon chardev=qmp_id_catch_monitor,mode=control \
-device pvpanic,ioport=0x505,id=idSWJ5gV \
-chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \
-device isa-serial,chardev=serial_id_serial0 \
-chardev socket,id=seabioslog_id_20151214-111528-C6FB1EaX,path=/tmp/ovmf_hot,server,nowait \
-device isa-debugcon,chardev=seabioslog_id_20151214-111528-C6FB1EaX,iobase=0x402 \
-device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \
-device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \
-device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \
-device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \
-device usb-tablet,id=usb-tablet1 \
-netdev tap,id=netdev0,vhost=on \
-device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \
-device ioh3420,bus=pcie.0,id=root.0,slot=1 \
-device x3130-upstream,bus=root.0,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device virtio-scsi-pci,bus=downstream1,id=scsi_pci_bus0 \
-drive file=/home/73test/script/ovmf/img/rhel72/rhel72_64.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \
-device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,bootindex=1,id=device_sysdisk,physical_block_size=512,logical_block_size=512,serial=12345678900987654321,ver=SYSDISK,wwn=0x123,channel=0,scsi-id=0,lun=0,disable-modern=on,disable-legacy=off \
-enable-kvm \
-monitor stdio \
-spice port=5900,disable-ticketing \
-qmp tcp:0:6666,server,nowait

2.Check block info in hmp
 (qemu) info block
pflash0 (#block157): /usr/share/OVMF/OVMF_CODE.secboot.fd (raw, read-only)
    Cache mode:       writeback

pflash1 (#block300): /usr/share/OVMF/OVMF_Client_VARS.fd (raw)
    Cache mode:       writeback

drive_sysdisk (#block549): /home/73test/script/ovmf/img/rhel72/rhel72_64.qcow2 (qcow2)
    Cache mode:       writeback, direct

3.In guest,execute "lspci":
 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio Controller (rev 01)
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.4 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
02:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:00.0 SCSI storage controller: Red Hat, Inc Virtio SCSI


4.Do legacy scsi-cd hotplug
[root@dhcp-8-210 OVMF]# nc -U /tmp/mm
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 6, "major": 2}, "package": " (qemu-kvm-rhev-2.6.0-22.el7)"}, "capabilities": []}}
{"execute":"qmp_capabilities"}
{"return": {}}

{"execute":"__com.redhat_drive_add","arguments":{"file":"/home/kvm_autotest_root/iso/linux/RHEL-7.2-20151030.0-Server-x86_64-dvd1.iso","format":"raw","id":"cd","media":"cdrom"}}
{"return": {}}

{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","bus":"downstream2","id":"scsi_pci_bus1","disable-modern":"on","disable-legacy":"off"}}
{"return": {}}

{"execute":"device_add","arguments":{"driver":"scsi-disk","drive":"cd","id":"cd0"}}
{"return": {}}

5.Check block info in hmp:
 (qemu) info block
pflash0 (#block157): /usr/share/OVMF/OVMF_CODE.secboot.fd (raw, read-only)
    Cache mode:       writeback

pflash1 (#block300): /usr/share/OVMF/OVMF_Client_VARS.fd (raw)
    Cache mode:       writeback

drive_sysdisk (#block549): /home/73test/script/ovmf/img/rhel72/rhel72_64.qcow2 (qcow2)
    Cache mode:       writeback, direct

cd (#block717): /home/kvm_autotest_root/iso/linux/RHEL-7.2-20151030.0-Server-x86_64-dvd1.iso (raw, read-only)
    Removable device: not locked, tray closed
    Cache mode:       writeback

6.In guest,execute "lspci":
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio Controller (rev 01)
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.4 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
02:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:00.0 SCSI storage controller: Red Hat, Inc Virtio SCSI
04:00.0 SCSI storage controller: Red Hat, Inc Virtio SCSI

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2608.html