Bug 608548
| Summary: | QEMU doesn't respect hardware sector size of underlying block device when doing O_DIRECT | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Johnny Liu <jialiu> | ||||
| Component: | qemu-kvm | Assignee: | chellwig <chellwig> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 6.0 | CC: | dyuan, hbrock, Jes.Sorensen, llim, ltroan, michen, mkenneth, szhou, tburke, virt-maint, xen-maint, yoyzhang | ||||
| Target Milestone: | rc | Keywords: | Reopened | ||||
| Target Release: | 6.1 | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | qemu-kvm-0.12.1.2-2.124.el6 | Doc Type: | Bug Fix | ||||
| Doc Text: |
Cause: qemu did not align memory properly for O_DIRECT support
Consequence: I/O to device with large sector sizes like CDROMs dit not work in cache=none mode
Fix: qemu was changed to use properly aligned memory for I/O requests
Result: I/O to devices with large sector sizes like CDROMs work in cache=none mode
|
Story Points: | --- | ||||
| Clone Of: | |||||||
| : | 616480 (view as bug list) | Environment: | |||||
| Last Closed: | 2011-05-19 11:28:32 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 580566, 580954, 616480 | ||||||
| Attachments: |
|
||||||
# virsh dumpxml cdrom_installation
<domain type='kvm' id='4'>
<name>cdrom_installation</name>
<uuid>093b482e-a02e-f5cf-563d-95d226d422b8</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64' machine='rhel6.0.0'>hvm</type>
<boot dev='cdrom'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>destroy</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/cdrom_installation.img'/>
<target dev='hda' bus='ide'/>
<alias name='ide0-0-0'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<disk type='block' device='cdrom'>
<driver name='qemu' cache='none'/>
<source dev='/dev/cdrom'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<alias name='ide0-1-0'/>
<address type='drive' controller='0' bus='1' unit='0'/>
</disk>
<controller type='ide' index='0'>
<alias name='ide0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<interface type='network'>
<mac address='52:54:00:be:fb:69'/>
<source network='default'/>
<target dev='vnet0'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/1'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target port='0'/>
<alias name='serial0'/>
</console>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='5900' autoport='yes' keymap='en-us'/>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
</devices>
<seclabel type='dynamic' model='selinux'>
<label>system_u:system_r:svirt_t:s0:c487,c559</label>
<imagelabel>system_u:object_r:svirt_image_t:s0:c487,c559</imagelabel>
</seclabel>
</domain>
Log in /var/log/libvirt/qemu/cdrom_installation.log:
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name cdrom_installation -uuid 093b482e-a02e-f5cf-563d-95d226d422b8 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/cdrom_installation.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -no-reboot -boot d -drive file=/var/lib/libvirt/images/cdrom_installation.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/dev/cdrom,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,cache=none -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=23,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:be:fb:69,bus=pci.0,addr=0x4 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3
char device redirected to /dev/pts/1
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux major release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Major release. This request is not yet committed for inclusion. Don't see how this could be a libvirt BZ, reassigning to QEMU Today I install a fresh HOST os from rel-eng build (RHEL6.0-20100622.1), this issue dose NOT occur. So it look like a environment issue, the nightly build is not stable. So I close this bug. The problem is reproduced again after updated some packages, such as: libvirt, qemu-kvm, python-virtinst, selinux-policy, virt-manager, an so on. I yum these package from brewweb repo. # cat /etc/yum.repos.d/brewroot.repo [brewroot] name = brewroot_repo baseurl = http://porkchop.devel.redhat.com/brewroot/repos/RHEL-6-build/latest/$basearch enabled = 1 gpgcheck = 0 Not only one tester encountered this issue at our side, but now we still did not find the root cause, so I reopen this bug for tracking this issue. reproduce the bug with <disk type='block' device='cdrom'> <driver cache='none'/> in domain xml, when set it to writethrough or writeback or default, boot successfully via cdrom. It's the new feature of virt-manager-0.8.4-4.el6. no-cache should be the default (refer to the comment in bug 594080), but I'm not sure why it became 'default' occasionally, that make us very faint to reproduce it before. reproduce the bug with <disk type='block' device='cdrom'> <driver cache='none'/> in domain xml, when set it to writethrough or writeback or default, boot successfully via cdrom. kyla. It's the new feature of virt-manager-0.8.4-4.el6. no-cache should be the default (refer to the comment in bug 594080), but I'm not sure why it became 'default' occasionally, that make us very faint to reproduce it before. dyuan. *** This bug has been marked as a duplicate of bug 558256 *** I do not think this bug is a a duplicate of bug 558256. In this bug, the guest even can NOT I do not think this bug is a a duplicate of bug 558256. Obviously in BZ#558256, guest can find the bootable media, and already start the installation process. But in this bug, the guest even can NOT find a bootable media, the result is worse than 558256. Maybe the summary is not clear, so I changed it to be more clear. This issue has been proposed when we are only considering blocker issues in the current Red Hat Enterprise Linux release. It has been denied for the current Red Hat Enterprise Linux release. ** If you would still like this issue considered for the current release, ask your support representative to file as a blocker on your behalf. Otherwise ask that it be considered for the next Red Hat Enterprise Linux release. ** "Boot failed: Could not read from CDROM (code 0003)" is from seabios/boot.c. It's almost certainly not a duplicate of bug 558256. I can reproduce it here when trying to use virt-manager to install from a real DVD on RHEL6 as of July 19. If I go into virt-manager and change the cache mode for the DVD drive to writeback it works fine. I'll look at whats wrong with qemu for this, but I believe libvirt shouldn't set DVD drives as cache=none in the first place either. Jes Ok so the problem is as follows: cache=none means the device, in this case the CDROM/DVD is opened with O_DIRECT. With O_DIRECT all I/O needs to be sector size aligned, and all request must be a multiple of the sector size. This works fine for files on a file system and raw disk partitions where the sector size is 512 bytes, but for a CDROM/DVD drive the sector size is normally 2048 bytes. Checking whether we are on a CDROM/DVD device, and checking the block size is pretty complicated, plus the 512 byte sector size is hard coded all over the place in QEMU, so it is not really feasible to try and fix this in QEMU itself. The only reasonable solution for RHEL6 is to make sure that libvirt doesn't set cache=none on CDROM/DVD drives. Jes Christoph tells that we should fix the alignment to 4k for supporting 4k block io physical disks Moving to 6.1 as 616480 is a workaround for the CDROM problem. Change bug description to match actual problem that QEMU expects all devices to have 512 byte sectors with O_DIRECT, which doesn't for devices with 2K or 4K sectors. Reassign to Christoph as he said he was looking into solving the problem the right way. I've submitted a patch to allow O_DIRECT I/O for non-512byte block sizes, including automatically enabling it for CDROMs. I'll backport it as soon as it's been accepted upstream. *** Bug 616509 has been marked as a duplicate of this bug. *** Reproduce this bug with 113 with cdrom option: -drive file=/dev/sr0,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,cache=none -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 Boot failed with error code 0003. Verify this bug with qemu-kvm-0.12.1.2-2.128.el6.x86_64.rpm as same cdrom option, guest can booting from CD-Rom ok. So this bug has been fixed. move to verified based on comment#31
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
Cause: qemu did not align memory properly for O_DIRECT support
Fix: qemu was changed to use properly aligned memory for I/O requests
Consequence: I/O to device with large sector sizes like CDROMs dit not work in cache=none mode
Result: I/O to devices with large sector sizes like CDROMs work in cache=none mode
Technical note updated. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
Diffed Contents:
@@ -1,6 +1,7 @@
Cause: qemu did not align memory properly for O_DIRECT support
+Consequence: I/O to device with large sector sizes like CDROMs dit not work in cache=none mode
+
Fix: qemu was changed to use properly aligned memory for I/O requests
-Consequence: I/O to device with large sector sizes like CDROMs dit not work in cache=none mode
Result: I/O to devices with large sector sizes like CDROMs work in cache=none mode
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0534.html An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0534.html |
Created attachment 427301 [details] Screenshot - installation failed form cdrom device. Description of problem: When trying to install guest form cdrom device, installation failed. The following error message is seen in virt-viewer: Boot failed: Could not read from CDROM (code 0003) No bootable device The screen-shot will be attached. Version-Release number of selected component (if applicable): qemu-kvm-0.12.1.2-2.82.el6.x86_64 libvirt-0.8.1-10.el6.x86_64 seabios-0.5.1-0.15.20100108git669c991.el6.x86_64 # uname -a Linux dhcp-66-70-179.nay.redhat.com 2.6.32-37.el6.x86_64 #1 SMP Sun Jun 20 19:29:35 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux How reproducible: Always Steps to Reproduce: 1. Run virt-install to install guest from cdrom device. # virt-install -n cdrom_installation -r 1024 --disk path=/var/lib/libvirt/images/cdrom_installation.img,size=5 -c /dev/cdrom 2. Check the message in pop-up virt-viewer windows. Actual results: Installation failed. The following error message is seen in virt-viewer: Boot failed: Could not read from CDROM (code 0003) No bootable device NOTE: I can make sure my DVD is a good media, because I can install host OS from this DVD media successfully. Expected results: Installation from cdrom device should be finished successfully. Additional info: If install the guest from ISO file, the installation works fine.