RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 607919 - "Permission denied" is seen when trying to start vm with a nfs disk image after libivrtd restart.
Summary: "Permission denied" is seen when trying to start vm with a nfs disk image aft...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.0
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Laine Stump
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-06-25 08:01 UTC by Johnny Liu
Modified: 2010-06-29 03:45 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-06-29 03:45:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
guest log (8.31 KB, text/plain)
2010-06-25 10:09 UTC, Johnny Liu
no flags Details
qemu.conf (7.31 KB, text/plain)
2010-06-25 10:10 UTC, Johnny Liu
no flags Details
log file for comment 8 (20.87 KB, text/plain)
2010-06-25 11:03 UTC, Johnny Liu
no flags Details
guest log - ver 2 (16.99 KB, text/plain)
2010-06-25 11:04 UTC, Johnny Liu
no flags Details
qemu.conf - ver 2 (7.31 KB, text/plain)
2010-06-25 11:04 UTC, Johnny Liu
no flags Details

Description Johnny Liu 2010-06-25 08:01:53 UTC
Description of problem:
Restart libvirtd service, try to start a vm with disk image on a nfs share, the following error is seen:
# virsh start test
error: Failed to start domain test
error: internal error Process exited while reading console log output: char device redirected to /dev/pts/3
qemu: could not open disk image /mnt/jialiu/vm1.img: Permission denied


Version-Release number of selected component (if applicable):
libvirt-0.8.1-10.el6.x86_64
# uname -a
Linux dhcp-66-70-86.nay.redhat.com 2.6.32-37.el6.x86_64 #1 SMP Sun Jun 20 19:29:35 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
qemu-kvm-0.12.1.2-2.81.el6.x86_64
# rpm -qa|grep selinux
selinux-policy-targeted-3.7.19-27.el6.noarch
libselinux-2.0.94-1.el6.x86_64
libselinux-python-2.0.94-1.el6.x86_64
selinux-policy-3.7.19-27.el6.noarch
libselinux-utils-2.0.94-1.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Install a fresh os from nightly build - RHEL6.0-20100622.n.0
2. mount nfs share
# setenforce 0
# getenforce 
Permissive
# iptables -F
# mount 10.66.90.113:/vol/xenimage /mnt (10.66.90.113 is a share nfs)
3. Define a vm, and start it.
# virsh dumpxml test
<domain type='kvm'>
  <name>test</name>
  <uuid>89731ff2-def2-9f1a-0708-5cac6b051fab</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.0.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' cache='none'/>
      <source file='/mnt/jialiu/vm1.img'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:29:e3:76'/>
      <source network='default'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
  </devices>
</domain>
# virsh start test
Domain test started

# virsh destroy test
Domain test destroyed

4. restart libvirtd, and try to start the vm again:
# service libvirtd restart
Stopping libvirtd daemon:                                  [  OK  ]
Starting libvirtd daemon:                                  [  OK  ]
# virsh start test
error: Failed to start domain test
error: internal error Process exited while reading console log output: char device redirected to /dev/pts/3
qemu: could not open disk image /mnt/jialiu/vm1.img: Permission denied

Actual results:
Fail to start vm, and "Permission denied" is seen.

Expected results:
vm should be started successfully.

Additional info:

Comment 2 RHEL Program Management 2010-06-25 08:22:54 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 3 Daniel Berrangé 2010-06-25 08:58:05 UTC
Can you attach the /etc/libvirt/qemu.conf file, /var/log/libvirt/qemu/$GUEST.log and provide output of

 ls -l /mnt/jialiu/vm1.img
 ls -dl /mnt/
 ls -dl /mnt/jialiu/

Comment 4 dyuan 2010-06-25 09:58:31 UTC
start domain with 'Permission denied' error both with enforcing and permissive mode. test version is the same as orignal description.

qemu: could not open disk image /mnt/test/dyuan/img/rhel5u4-x86_64.img: Permission denied

10.66.90.113:/vol/xenimage on /mnt/test type nfs (rw,addr=10.66.90.113)

# getsebool -a|grep virt_use_nfs
virt_use_nfs --> on

# ll -dZ /mnt/
drwxr-xr-x. root root system_u:object_r:mnt_t:s0       /mnt/
# ll -dZ /mnt/test/
drwx-----x. root root system_u:object_r:nfs_t:s0       /mnt/test/
# ll -dZ /mnt/test/dyuan/
drwxrwxr-x. qemu qemu system_u:object_r:nfs_t:s0       /mnt/test/dyuan/
# ll -dZ /mnt/test/dyuan/img/
drwxrwxr-x. qemu qemu system_u:object_r:nfs_t:s0       /mnt/test/dyuan/img/
# ll -dZ /mnt/test/dyuan/img/rhel5u4-x86_64.img 
-rw-r--r--. root root system_u:object_r:nfs_t:s0       /mnt/test/dyuan/img/rhel5u4-x86_64.img

Comment 5 Johnny Liu 2010-06-25 10:05:19 UTC
# ls -l /mnt/jialiu/vm1.img
-rw-------. 1 root root 5242880000 Jun 25 06:03 /mnt/jialiu/vm1.img
[root@dhcp-66-70-86 ~]# ls -dl /mnt/
drwx-----x. 17 root root 4096 Jun 24 10:20 /mnt/
[root@dhcp-66-70-86 ~]# ls -dl /mnt/jialiu/
drwxr-xr-x. 2 qemu qemu 4096 Jun 24 21:28 /mnt/jialiu/

The log file will be attached later.

Comment 6 Johnny Liu 2010-06-25 10:09:32 UTC
Created attachment 426831 [details]
guest log

Comment 7 Johnny Liu 2010-06-25 10:10:15 UTC
Created attachment 426832 [details]
qemu.conf

Comment 8 Daniel Berrangé 2010-06-25 10:40:53 UTC
Can you set 

 log_filters="1:util 1:security 1:qemu"
 log_outputs="1:file:/var/log/libvirt/libvirtd.log"

in /etc/libvirt/libvirtd.conf and restart libvirtd and try the guest again, then attach that log file to this bug.

Comment 9 Johnny Liu 2010-06-25 11:02:23 UTC
Sorry for my mistake, the attachments in comment 6 and comment 7 are not correct ones.

So I attached them again and with the log file generated in comment 8.

Comment 10 Johnny Liu 2010-06-25 11:03:25 UTC
Created attachment 426845 [details]
log file for comment 8

Comment 11 Johnny Liu 2010-06-25 11:04:23 UTC
Created attachment 426847 [details]
guest log - ver 2

Comment 12 Johnny Liu 2010-06-25 11:04:59 UTC
Created attachment 426848 [details]
qemu.conf - ver 2

Comment 13 Hugh Brock 2010-06-28 16:13:53 UTC
Not sure if this is misconfiguration or for real, but in any case if it's NFS it's critical-path for VDSM and must be fixed.

Comment 14 Laine Stump 2010-06-29 03:45:13 UTC
In order to store the image files on a root-squashed NFS share, dynamic_ownership must be set to 0 in qemu.conf. This is not the case here.

When I set dynamic_ownership=1 in my own test setup, failure occurs as indicated here. When I set dynamic_ownership=0, it the domain can be successfully created and destroyed multiple times.

(this did point out a separate bug in in a patch I just pushed today, though, so it's actually good this bug was filed ;-)


Note You need to log in before you can comment on or make changes to this bug.