Bug 510612

Summary: QEMU driver does not honour read-only flag for disks
Product: Red Hat Enterprise Linux 5 Reporter: Aron Griffis <aron.griffis>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 5.6CC: adaora.onyia, alex_williamson, amy.griffis, berrange, bzeranski, crobinso, dallan, doug.chapman, dwa, dyuan, hbrock, id.2ndr, jdenemar, linda.knippers, llim, martine.silbermann, mra, mzhan, panormitis, roland.friedwagner, shawn.pagan, shengliang.lv, stillwell, tao, virt-maint, weizhan, xen-maint, xhu, zhenfei.peng
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: hp:dl785solblk
Fixed In Version: libvirt-0.8.2-9.el5 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-01-13 22:51:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 510630, 645798    
Bug Blocks: 562684    

Description Aron Griffis 2009-07-09 21:16:47 UTC
Description of problem:
Theoretically one can share read-only disk images between virtual machines, but specifying perms=ro has no effect with kvm-based VMs.  The disks are unexpectedly presented to the VMs as writeable.  I discovered this when my shared /usr volume was corrupted by receiving writes from multiple VMs.

Version-Release number of selected component (if applicable):
RHEL 5.4 snapshot 1

Additional info:
Unless this is somehow my error, I suspect the libvirt folks are already aware of the problem, but let me know if more information is needed.

Comment 1 Aron Griffis 2009-07-09 21:35:26 UTC
Fwiw, I don't think there is a corresponding qemu option.  Longer-term, that would be nice to fix.  For the moment, this bug is really about the fact that virt-install accepts the perms=ro option and silently creates the VM with a writeable disk instead of aborting.

Comment 3 Daniel Berrangé 2009-07-09 21:50:35 UTC
Yes, this is a known limitation, since underlying QEMU has no way to be told to treat a disk as read only. THis badly needs addressing in QEMU.

FYI, the 'shareable' flag is supported, in that we disable host caching, such that disks can be shared to many guests without data consistency issues across hosts. Obviously this would still need a cluster filesystem, so not a replacement for readonly mode.

Comment 4 Aron Griffis 2009-07-10 16:54:02 UTC
Daniel, FYI I tried using a file-based image hosted on a filesystem mounted readonly to protect it from qemu writing to it.  Additionally in the guest fstab, /usr is mounted -o ro.  However in my experiments this causes the VM to hang on boot.  Unfortunately this means that RHEL 5.4 provides no recourse for sharing filesystem images r/o between guests.

Maybe easier to explain as commands:

$ ls /mnt/tmp
shared-usr.img

$ mount -o remount,ro /mnt/tmp

$ virsh start --console myguest
[...]
INIT: version 2.86 booting
		Welcome to Red Hat Enterprise Linux Server
		Press 'I' to enter interactive startup.
Setting clock  (utc): Fri Jul 10 12:06:19 EDT 2009 [  OK  ]
Starting udev: [  OK  ]
Loading default keymap (us): [  OK  ]
Setting hostname localhost.localdomain:  [  OK  ]
No devices found
Setting up Logical Volume Management: [  OK  ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/vda1 
/: clean, 6311/263232 files, 104505/263056 blocks
[/sbin/fsck.ext3 (1) -- /usr] fsck.ext3 -a /dev/vdb1 
/usr: clean, 122279/350592 files, 640656/700827 blocks
(HUNG AT THIS POINT)

$ virsh destroy myguest

$ mount -o remount,rw /mnt/tmp

$ virsh start --console myguest
INIT: version 2.86 booting
		Welcome to Red Hat Enterprise Linux Server
		Press 'I' to enter interactive startup.
Setting clock  (utc): Fri Jul 10 12:39:49 EDT 2009 [  OK  ]
Starting udev: [  OK  ]
Loading default keymap (us): [  OK  ]
Setting hostname localhost.localdomain:  [  OK  ]
No devices found
Setting up Logical Volume Management: [  OK  ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/vda1 
/: clean, 6311/263232 files, 104505/263056 blocks
[/sbin/fsck.ext3 (1) -- /usr] fsck.ext3 -a /dev/vdb1 
/usr: clean, 122279/350592 files, 640656/700827 blocks
[  OK  ]
Remounting root filesystem in read-write mode:  [  OK  ]
Mounting local filesystems:  [  OK  ]
Enabling local filesystem quotas:  [  OK  ]
Enabling /etc/fstab swaps:  [  OK  ]
(ETC)

Comment 8 Daniel Veillard 2009-12-16 14:30:19 UTC
I'm not sure QEmu has been extended to support a read-only flag yet, in any case this sounds too late for Update 5 at this point so reassigning to 5.6

Daniel

Comment 9 Id2ndR 2010-01-11 17:02:54 UTC
I used to use -snapshot option with kvm. It seems to be the equivalent of <readonly/> for disk device in libvirt. However the readonly option in libvirt doesn't prevent to write to the disk.
Do I misunderstand the readonly option or it is not implemented yet ?

Comment 12 Jiri Denemark 2010-09-02 11:58:09 UTC
Fixed in libvirt-0.8.2-1.el5

Comment 14 weizhang 2010-10-14 11:41:14 UTC
test on rhel5.6

with 'Permissive' state of selinux the vm with <readonly/> flag on disk can be started but I can still write on the disk; with 'Enforcing' state, it has an error: Failed to start domain vm
error: internal error Process exited while reading console log output: qemu: could not open disk image   

qemu kernel-2.6.18-225.el5
libvirt-0.8.2-6.el5
kvm-qemu-img-83-200.el5

Comment 15 dyuan 2010-10-19 04:51:16 UTC
As this bug fixed in rhel5.6, will the readonly option in libvirt prevent to write to the disk ? Should we re-open this bug according to comment 14 ? Thanks.

Comment 16 Jiri Denemark 2010-10-22 11:44:32 UTC
*** Bug 644763 has been marked as a duplicate of this bug. ***

Comment 17 Jiri Denemark 2010-10-22 11:46:11 UTC
Switching back to ASSIGNED

Comment 18 Jiri Denemark 2010-10-22 12:14:59 UTC
The problem is that libvirt uses readonly=on flag only when qemu supports -device. RHEL-5 qemu added this flag to the existing -drive option (without adding it to -help output for extra fun), which we need to teach libvirt about

Comment 19 Jiri Denemark 2010-10-22 13:51:52 UTC
A bug 645798 requests adding the readonly option to help output of qemu-kvm.

Comment 22 Min Zhan 2010-10-28 09:14:41 UTC
Verified with Passed in below environment:
RHEL5.6-Server-x86_64-KVM
libvirt-0.8.2-9.el5
kvm-qemu-img-83-207.el5
kernel-2.6.18-228.el5

# getenforce
Permissive


For enforcing mode if readonly flag works well, I will track bug 644763 to check.

Comment 23 Min Zhan 2010-10-28 09:38:08 UTC
Verified with Passed in below environment:
RHEL5.6-Server-x86_64-KVM
libvirt-0.8.2-9.el5
kvm-qemu-img-83-207.el5
kernel-2.6.18-228.el5

# getenforce
Enforcing

Detail Steps:
1. use virt-manager to add storage for guest 
2. add readonly flag for this storage
# virsh edit <guest>
...
   <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/libvirt/images/test-cdrom.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='floppy'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/test-net.img'/>
      <target dev='fda' bus='fdc'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
...
3. # virsh start <guest>
Domain start successfully

4. check readonly flag, should have readonly=on
# ps -ef |grep qemu|grep kvm
root      9233     1 49 09:24 ?        00:00:05 /usr/libexec/qemu-kvm -S -M rhel5.4.0 -m 512 -smp 1 -name test-cdrom -uuid 00e790e6-3669-d1bb-9c8b-9e1526ba51d4 -monitor unix:/var/lib/libvirt/qemu/test-cdrom.monitor,server,nowait -boot c -drive file=/var/lib/libvirt/images/test-cdrom.img,if=virtio,boot=on,format=raw,cache=none -drive file=/var/lib/libvirt/images/test-net.img,if=virtio,readonly=on,format=raw -net nic,macaddr=54:52:00:02:d3:ff,vlan=0,model=virtio -net tap,fd=18,vlan=0 -serial pty -parallel none -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus -balloon virtio

5. In guest,
# mount /dev/fd0 /mnt
mount:block device /dev/fd0 is write-protected,mounting read-only
# dd if =/dev/zero of=/mnt/write_test bs=1024 size=100
create file failed.

Comment 24 Peng ZhenFei 2010-11-29 07:15:05 UTC
pick the follwoing combination:
RHEL5.6-Server-x86_64-KVM
libvirt-0.8.2-11.el5
kvm-qemu-img-83-207.el5
kernel-2.6.18-231.el5

# getenforce
Enforcing

try add readonly flag to the xml then

[root@min ~]# virsh start guest
error: Failed to start domain guest
error: internal error Process exited while reading console log output: qemu: readonly flag not supported for drive of this interface

Comment 25 Jiri Denemark 2010-11-29 08:33:16 UTC
This is a limitation of qemu. According to qemu sources, it only supports readonly flag for scsi, virtio, and floppy drives. So you can't use readonly in combination with ide drives.

Comment 26 Min Zhan 2010-11-29 08:35:38 UTC
According to comment 25, I turn the bug status from ON_QA to Verified.

Comment 27 Peng ZhenFei 2010-11-30 07:25:03 UTC
Pick the following combination
RHEL5.6-Server-x86_64-KVM
libvirt-0.8.2-12.el5
kvm-qemu-img-83-207.el5
kernel-2.6.18-232.el5

After few try to add file base scsi device in rhel5.6 KVM, we finally found it is not support in 2.6.18.x kernel O_o

Then we kick off two more testing with fdc and virtio and readonly flag look good.

This is work for me

Comment 29 errata-xmlrpc 2011-01-13 22:51:32 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0060.html