Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 624447 - [vdsm] [libvirt] permission error on run vm task when using NFS storage (libvirt log!)
[vdsm] [libvirt] permission error on run vm task when using NFS storage (libv...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.1
All Linux
low Severity medium
: rc
: ---
Assigned To: Laine Stump
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-08-16 09:54 EDT by Haim
Modified: 2014-01-12 19:46 EST (History)
14 users (show)

See Also:
Fixed In Version: libvirt-0.9.10-1.el6
Doc Type: Bug Fix
Doc Text:
In some configurations, libvirt or RHEV users would see log messages such as this: "warning : virDomainDiskDefForeachPath:7654 : Ignoring open failure on xxx.xxx" These messages were harmless, but sometimes caused concern. They should no longer occur unless there really is a problem that needs handling.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 02:24:18 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:0748 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2012-06-19 15:31:38 EDT

  None (edit)
Description Haim 2010-08-16 09:54:30 EDT
Description of problem:

watching libvirt logs I see permission errors when running vm (on each run vm command scenarios). 
currently. there is no actual affect on the system, though I would like to know why it occurs, if we need to deal with it, and if not, remove it from log. 


16:50:58.366: warning : virDomainDiskDefForeachPath:7654 : Ignoring open failure on /rhev/data-center│pU
UI│/dff3b690-519b-4c05-b790-0b52837f40c3/250afad0-bed7-4bce-8841-73906e1c3e14/images/207f524b-c93a-4433-│
Th│96e5-6544ba38821e/6224acd4-d93a-435c-ba0c-c9ee4297add6: Permission denied                            │es
po│16:50:58.376: info : qemudDispatchSignalEvent:397 : Received unexpected signal 17                    │}
Th│16:50:58.401: info : brProbeVnetHdr:449 : Enabling IFF_VNET_HDR                                      │s:
 (│16:50:58.411: info : qemudDispatchSignalEvent:397 : Received unexpected signal 17                    │
Th│Thread-2962::WARNING::2010-08-16 16:50:58,672::libvirtvm::633::vds.vmlog.68fccbbf-cad6-4d94-aebc-f5fc│in
s:│40c6ac8c::_readPauseCode unsupported by libvirt vm

2.6.32-59.1.el6.x86_64
libvirt-0.8.1-23.el6.x86_64
vdsm-4.9-12.3.x86_64
device-mapper-multipath-0.4.9-25.el6.x86_64
lvm2-2.02.72-4.el6.x86_64
qemu-kvm-0.12.1.2-2.109.el6.x86_64


1) run new vm
Comment 2 Dan Kenigsberg 2010-09-06 08:13:46 EDT
Please verify if the problem's gone with virt_use_nfs=on.

Since it's libvirt.log-related, why is it open on vdsm? I suspect that behavior will be the same if you use virsh directly.
Comment 3 Haim 2010-09-20 04:50:25 EDT
no, still hit same behivior.
look at the attached log. 

11:15:11.051: debug : qemuSetupDiskPathAllow:3539 : Process path /rhev/data-center/02f12ae5-7121-4e67-b19c-b2c293bc6206/0e3f4ad3-87fb-4c67-b660-50b9e4584216/images/a26c9ff2-24be-4cf9-b64e-84e3b0c0b9e9/9faac66d-5ca
2-4bad-8b2c-d5b4210adcb4 for disk
11:15:11.051: debug : qemuSetupDiskPathAllow:3545 : Ignoring EINVAL for /rhev/data-center/02f12ae5-7121-4e67-b19c-b2c293bc6206/0e3f4ad3-87fb-4c67-b660-50b9e4584216/images/a26c9ff2-24be-4cf9-b64e-84e3b0c0b9e9/9faac
66d-5ca2-4bad-8b2c-d5b4210adcb4
11:15:11.052: warning : virDomainDiskDefForeachPath:7654 : Ignoring open failure on /rhev/data-center/02f12ae5-7121-4e67-b19c-b2c293bc6206/0e3f4ad3-87fb-4c67-b660-50b9e4584216/images/a26c9ff2-24be-4cf9-b64e-84e3b0
c0b9e9/9faac66d-5ca2-4bad-8b2c-d5b4210adcb4: Permission denied


virt_use_nfs --> on

[root@magenta-vdsc ~]#  getenforce 
Enforcing
Comment 4 Dan Kenigsberg 2010-09-25 18:16:22 EDT
thanks Haim, but what about my second question: why did you open this minor libvirt logging issue on vdsm? Without compelling explanation I would have to close this bug, or move it libvirt.
Comment 5 Haim 2010-09-26 11:10:43 EDT
so please move to libvirt. it reproduces all time.
Comment 6 Laine Stump 2011-01-31 15:44:54 EST
None of these logs are error logs; the one mentioned in the summary (and the one message common between both comments that list log messages) is warning level.

It is reporting that libvirt was unable to open a backing store file while trying to determine the volume type for informational purposes. This happens because that code runs as root, but the backing store is on a root-squashed disk owned by a different uid (not readable by root).
Comment 7 Daniel Berrange 2011-02-01 05:16:06 EST
We kept these warning messages because while in VDSM usage they may be expected, in other usage scenarios they are a sign of real problems. Without the warning messages we'd not be able to identify real problems that arise in non-VDSM usage.
Comment 13 zhpeng 2012-01-04 05:13:31 EST
The packages of comment 0 are too old that i can't use them to register to the RHEVM.

But, i used newer packages:
libvirt-0.9.8-1.el6.x86_64
qemu-kvm-0.12.1.2-2.213.el6.x86_64
vdsm-4.9-110.el6.x86_64

with virt_use_nfs=on

There are no such errors like comment 0 and comment 3.

So i need your help to confirm whether issue still exists in your env

BR
Comment 16 Laine Stump 2012-01-04 13:30:13 EST
Note that it is important for the NFS share to be mounted with root_squash, and the qemu user (in /etc/libvirt/qemu.conf) to be set to something other than "root", in order to see this bug (presumably, the files on the share would be owned by the qemu user:group).

Although I can't confirm right now that the message is still output, I can confirm that the code that would cause those two messages (the 
ignoring ..." messages) is still present in the source.
Comment 20 Laine Stump 2012-01-25 13:07:01 EST
(zhpeng: never mind, I set up my own root-squash NFS system and saw that both messages still appear)

I recently posted patches upstream that eliminate the Warning message (but not the debug message, since it's only seen when the log level is set to debug, and fixing that to not fail wouldn't solve any problem anyway). It's still stuck in review, but will hopefully be in libvirt-0.9.10 so that it can be in the next rebase for RHEL.

  https://www.redhat.com/archives/libvir-list/2012-January/msg00673.html
Comment 21 Laine Stump 2012-02-03 10:56:34 EST
A new version of the upstream patches, once again waiting for review/ACK:

  https://www.redhat.com/archives/libvir-list/2012-February/msg00098.html
Comment 22 Laine Stump 2012-02-03 16:54:16 EST
Patches to eliminate the warning message have been pushed upstream, so they will be in 0.9.10:

commit 90e4d681bc30074466912e9ae733b0bf0b9d5a03
Author: Laine Stump <laine@laine.org>
Date:   Fri Jan 13 15:26:45 2012 -0500

    util: refactor virFileOpenAs
    
    virFileOpenAs previously would only try opening a file as the current
    user, or as a different user, but wouldn't try both methods in a
    single call. This made it cumbersome to use as a replacement for
    open(2). Additionally, it had a lot of historical baggage that led to
    it being difficult to understand.
    
    This patch refactors virFileOpenAs in the following ways:
    
    * reorganize the code so that everything dealing with both the parent
      and child sides of the "fork+setuid+setgid+open" method are in a
      separate function. This makes the public function easier to understand.
    
    * Allow a single call to virFileOpenAs() to first attempt the open as
      the current user, and if that fails to automatically re-try after
      doing fork+setuid (if deemed appropriate, i.e. errno indicates it
      would now be successful, and the file is on a networkFS). This makes
      it possible (in many, but possibly not all, cases) to drop-in
      virFileOpenAs() as a replacement for open(2).
    
      (NB: currently qemuOpenFile() calls virFileOpenAs() twice, once
      without forking, then again with forking. That unfortunately can't
      be changed without at least some discussion of the ramifications,
      because the requested file permissions are different in each case,
      which is something that a single call to virFileOpenAs() can't deal
      with.)
    
    * Add a flag so that any fchown() of the file to a different uid:gid
      is explicitly requested when the function is called, rather than it
      being implied by the presence of the O_CREAT flag. This just makes
      for less subtle surprises to consumers. (Commit
      b1643dc15c5de886fefe56ad18608d65f1325a2c added the check for O_CREAT
      before forcing ownership. This patch just makes that restriction
      more explicit.)
    
    * If either the uid or gid is specified as "-1", virFileOpenAs will
      interpret this to mean "the current [gu]id".
    
    All current consumers of virFileOpenAs should retain their present
    behavior (after a few minor changes to their setup code and
    arguments).

commit c18a88ac489c5356734bb7cff59f8fc73c9d1227
Author: Laine Stump <laine@laine.org>
Date:   Thu Jan 12 13:24:45 2012 -0500

    qemu: eliminate "Ignoring open failure" when using root-squash NFS
    
    This eliminates the warning message reported in:
    
     https://bugzilla.redhat.com/show_bug.cgi?id=624447
    
    It was caused by a failure to open an image file that is not
    accessible by root (the uid libvirtd is running as) because it's on a
    root-squash NFS share, owned by a different user, with permissions of
    660 (or maybe 600).
    
    The solution is to use virFileOpenAs() rather than open(). The
    codepath that generates the error is during qemuSetupDiskCGroup(), but
    the actual open() is in a lower-level generic function called from
    many places (virDomainDiskDefForeachPath), so some other pieces of the
    code were touched just to add dummy (or possibly useful) uid and gid
    arguments.
    
    Eliminating this warning message has the nice side effect that the
    requested operation may even succeed (which in this case isn't
    necessary, but shouldn't hurt anything either).
Comment 25 Huang Wenlong 2012-02-14 03:32:15 EST
Verify this bug with : 
libvirt-0.9.10-1.el6.x86_64
vdsm-4.9.4-0.el6



1)  update libvirt pkg to libvirt-0.9.10-1.el6.x86_64

2)  restart libvirtd service 

#initctl restart libvirtd 

3)  in the rhev-m  start a NFS guest  

4) check the libvirtd.log 

#cat /var/log/libvirtd.log |grep -i permission  


Note : there is not any error about permission
Comment 26 Laine Stump 2012-05-08 15:24:10 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
In some configurations, libvirt or RHEV users would see log messages such as this:

 "warning : virDomainDiskDefForeachPath:7654 : Ignoring open
failure on xxx.xxx"

These messages were harmless, but sometimes caused concern. They should no longer occur unless there really is a problem that needs handling.
Comment 28 errata-xmlrpc 2012-06-20 02:24:18 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html

Note You need to log in before you can comment on or make changes to this bug.