Bug 1129211

Summary: Guest system installed on virtio-scsi fails to boot as virtio-blk
Product: Red Hat Enterprise Linux 7 Reporter: ShupingCui <scui>
Component: qemu-kvm-rhevAssignee: Fam Zheng <famz>
Status: CLOSED WORKSFORME QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 7.1CC: coli, hhuang, huding, juzhang, michen, shuang, sluo, virt-maint, xfu, xuhan
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.1.0-2.el7.x86_64 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-19 08:41:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
serial log none

Description ShupingCui 2014-08-12 10:16:34 UTC
Description of problem:
install rhel7 guest with virtio_scsi, then boot up with virtio_blk, guest cannot boot up with:
systemd-fsck[282]: fsck: error 2 (No such file or directory) while executing fsck.ext2 for /dev/mapper/rhel_unused-root

Version-Release number of selected component (if applicable):
# uname -r
3.10.0-142.el7.x86_64
# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.1.0-1.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. install rhel7 guest with virtio_scsi
/bin/qemu-kvm \
    -name 'virt-tests-vm1'  \
    -sandbox off  \
    -M pc  \
    -nodefaults  \
    -vga qxl  \
    -global qxl-vga.vram_size=33554432 \
    -device intel-hda,bus=pci.0,addr=03 \
    -device hda-duplex  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140805-105741-5Mo1JeBh,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140805-105741-5Mo1JeBh,server,nowait \
    -device isa-serial,chardev=serial_id_serial0 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04  \
    -chardev socket,id=devvs,path=/tmp/virtio_port-vs-20140805-105741-5Mo1JeBh,server,nowait \
    -device virtserialport,chardev=devvs,name=vs,id=vs,bus=virtio_serial_pci0.0  \
    -chardev socket,id=seabioslog_id_20140805-105741-5Mo1JeBh,path=/tmp/seabios-20140805-105741-5Mo1JeBh,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140805-105741-5Mo1JeBh,iobase=0x402 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=05 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=06 \
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/RHEL-Server-7.0-64-virtio.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:a6:a7:a8:a9:aa,id=idgAoeCh,vectors=4,netdev=idEFK6HS,bus=pci.0,addr=07  \
    -netdev tap,id=idEFK6HS,vhost=on,vhostfd=23,fd=22  \
    -m 8192  \
    -smp 4,cores=2,threads=1,sockets=2  \
    -cpu 'SandyBridge',+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/linux/RHEL7.0-Server-x86_64.iso \
    -device ide-cd,id=cd1,drive=drive_cd1,bus=ide.0,unit=0 \
    -drive id=drive_unattended,if=none,snapshot=off,aio=native,media=cdrom,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/rhel70-64/ks.iso \
    -device ide-cd,id=unattended,drive=drive_unattended,bus=ide.0,unit=1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -kernel '/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/rhel70-64/vmlinuz'  \
    -append 'ksdevice=link ks=cdrom:/dev/sr1:/ks.cfg nicdelay=60 console=ttyS0,115200 console=tty0'  \
    -initrd '/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/rhel70-64/initrd.img'  \
    -spice port=3000,password=123456,addr=0,tls-port=3200,x509-dir=/tmp/spice_x509d,tls-channel=main,tls-channel=inputs,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off  \
    -no-kvm-pit-reinjection \
    -no-shutdown \
    -enable-kvm

2. boot up with virtio_blk after installation successfully
/bin/qemu-kvm \
    -name 'virt-tests-vm1'  \
    -sandbox off  \
    -M pc  \
    -nodefaults  \
    -vga qxl  \
    -global qxl-vga.vram_size=33554432 \
    -device intel-hda,bus=pci.0,addr=03 \
    -device hda-duplex  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140806-031148-naaCrkbi,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140806-031148-naaCrkbi,server,nowait \
    -device isa-serial,chardev=serial_id_serial0 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04  \
    -chardev socket,id=devvs,path=/tmp/virtio_port-vs-20140806-031148-naaCrkbi,server,nowait \
    -device virtserialport,chardev=devvs,name=vs,id=vs,bus=virtio_serial_pci0.0  \
    -chardev socket,id=seabioslog_id_20140806-031148-naaCrkbi,path=/tmp/seabios-20140806-031148-naaCrkbi,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140806-031148-naaCrkbi,iobase=0x402 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=05 \
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/RHEL-Server-7.0-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=06 \
    -device virtio-net-pci,mac=9a:7e:7f:80:81:82,id=idId3JMe,vectors=4,netdev=idReBNst,bus=pci.0,addr=07  \
    -netdev tap,id=idReBNst,vhost=on,vhostfd=23,fd=22  \
    -m 8192  \
    -smp 4,cores=2,threads=1,sockets=2  \
    -cpu 'SandyBridge',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -spice port=3000,password=123456,addr=0,tls-port=3200,x509-dir=/tmp/spice_x509d,tls-channel=main,tls-channel=inputs,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off  \
    -no-kvm-pit-reinjection \
    -enable-kvm
3.

Actual results:
cannot boot up with:
systemd-fsck[282]: fsck: error 2 (No such file or directory) while executing fsck.ext2 for /dev/mapper/rhel_unused-root

Expected results:
boot up successfully

Additional info:
guest ks file:
install
KVM_TEST_MEDIUM
graphical
poweroff
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp
rootpw redhat
firewall --enabled --ssh
selinux --enforcing
timezone --utc America/New_York
firstboot --disable
bootloader --location=mbr --append="console=tty0 console=ttyS0,115200 rootfstype=xfs"
zerombr
clearpart --all --initlabel
autopart
xconfig --startxonboot

%packages --ignoremissing
@base
@core
@development
@additional-devel
@debugging-tools
@network-tools
@basic-desktop
@desktop-platform
@fonts
@general-desktop
@graphical-admin-tools
@x11
lftp
gcc
gcc-c++
patch
make
git
nc
NetworkManager
ntpdate
redhat-lsb
numactl-libs
numactl
sg3_utils
hdparm
lsscsi
libaio-devel
perl-Time-HiRes
flex
prelink
%end

%post
echo "OS install is completed" > /dev/ttyS0
echo "remove rhgb quiet by grubby" > /dev/ttyS0
grubby --remove-args="rhgb quiet" --update-kernel=$(grubby --default-kernel)
echo "dhclient" > /dev/ttyS0
dhclient
echo "get repo" > /dev/ttyS0
wget http://fileshare.englab.nay.redhat.com/pub/section2/repo/epel/rhel-autotest.repo -O /etc/yum.repos.d/rhel-autotest.repo
echo "yum makecache" > /dev/ttyS0
yum makecache
echo "yum install -y stress" > /dev/ttyS0
yum install -y stress
echo "chkconfig sshd on" > /dev/ttyS0
chkconfig sshd on
echo "iptables -F" > /dev/ttyS0
iptables -F
echo "echo 0 > selinux/enforce" > /dev/ttyS0
echo 0 > /selinux/enforce
echo "chkconfig NetworkManager on" > /dev/ttyS0
chkconfig NetworkManager on
echo "update ifcfg-eth0" > /dev/ttyS0
sed -i "/^HWADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
echo "Disable lock cdrom udev rules" > /dev/ttyS0
sed -i "/--lock-media/s/^/#/" /usr/lib/udev/rules.d/60-cdrom_id.rules 2>/dev/null>&1
echo 'Post set up finished' > /dev/ttyS0
echo Post set up finished > /dev/hvc0
%end

Comment 1 ShupingCui 2014-08-12 10:17:32 UTC
Created attachment 926005 [details]
serial log

Test with qemu-kvm-1.5.3-67.el7.x86_64, not met this issue.

Comment 3 Fam Zheng 2014-08-18 04:50:37 UTC
initrd is not updated after guest device configuration change, that may make a different device path, or missing virtio_blk module when booting. Could you show the output of "lsmod | grep virtio" and "ls -R /dev/disk /dev/mapper" after it drops to dracut shell?

Fam

Comment 4 Fam Zheng 2014-08-18 10:28:09 UTC
This is a duplicate of bug 1127151 but just the other way around. So close it too.

*** This bug has been marked as a duplicate of bug 1127151 ***

Comment 5 Fam Zheng 2014-08-18 11:06:39 UTC
(In reply to ShupingCui from comment #1)
> Created attachment 926005 [details]
> serial log
> 
> Test with qemu-kvm-1.5.3-67.el7.x86_64, not met this issue.

This is an open question that needs some more investigation.

Fam

Comment 6 Fam Zheng 2014-08-19 08:41:35 UTC
Can't reproduce after testing again with qemu-kvm-rhev-2.1.0-2.el7.x86_64.

When generating initrd, during either system installation or kernel update, dracut takes care of installing *both* virtio-blk and virtio-scsi drivers into the initramfs image, as long as it finds itself inside a virtual machine. (that is detected with running `systemd-detect-virt --vm` in guest). This logic is guaranteed by the qemu module (/usr/lib/dracut/modules.d/90qemu).

For both qemu-kvm and qemu-kvm-rhev, the output of systemd-detect-virt command is "kvm".

I checked that in the test env, these look fine and work as expected.

So interchanging the system disk type between virtio-blk and virtio-scsi for RHEL 7 guest should just work. If in some way it is broken and only one of the two drivers is present in initrd, systemd-detect-virt, dracut and anaconda are the points to check.

Fam