RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 680880 - The defined NFS pool can not be started
Summary: The defined NFS pool can not be started
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.1
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-02-28 10:09 UTC by Jason Chen
Modified: 2012-06-20 06:26 UTC (History)
12 users (show)

Fixed In Version: libvirt-0.9.9-1.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-06-20 06:26:14 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:0748 0 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2012-06-19 19:31:38 UTC

Description Jason Chen 2011-02-28 10:09:54 UTC 
Description of problem:
   it can not start the defined nfs pool with the command "pool-start ..." , and all ways complaining the following:
" cannot open volume '/var/lib/libvirt/images/nfs/fcoemon.dcbd.1613': No such device or address "


Version-Release number of selected component (if applicable):

[1] OS version:
------------------------
[root@dhcp-93-122 test]# uname -a
Linux dhcp-93-122.nay.redhat.com 2.6.32-117.el6.x86_64 #1 SMP Tue Feb 15 16:29:34 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

[2] libvirt version:
------------------------
[root@dhcp-93-122 test]# rpm -qa |grep libvirt
libvirt-0.8.7-7.el6.x86_64
libvirt-debuginfo-0.8.7-7.el6.x86_64
libvirt-client-0.8.7-7.el6.x86_64
libvirt-devel-0.8.7-7.el6.x86_64
libvirt-python-0.8.7-7.el6.x86_64

[3] qemu-kvm version:
------------------------
[root@dhcp-93-122 test]# rpm -qa |grep qemu
qemu-kvm-debuginfo-0.12.1.2-2.146.el6.x86_64
qemu-kvm-tools-0.12.1.2-2.146.el6.x86_64
qemu-img-0.12.1.2-2.146.el6.x86_64
gpxe-roms-qemu-0.9.7-6.4.el6.noarch
qemu-kvm-0.12.1.2-2.146.el6.x86_64

[4] kernel version:
------------------------
[root@dhcp-93-122 test]# rpm -qa|grep kernel
dracut-kernel-004-36.el6.noarch
kernel-firmware-2.6.32-117.el6.noarch
abrt-addon-kerneloops-1.1.15-1.el6.x86_64
kernel-2.6.32-94.el6.x86_64
kernel-2.6.32-117.el6.x86_64
kernel-headers-2.6.32-117.el6.x86_64

How reproducible:


Steps to Reproduce:
1. create one pool config file as below:
---------------------------------------------------------------
[root@dhcp-93-122 test]# more nfs-pool.xml 
<pool type="netfs">
     <name>test-nfs</name>
          <source>
              <host name="10.66.90.121"/>
              <dir path="/vol/S3/libvirtmanual"/>
          </source>
          <target>
              <path>/var/lib/libvirt/images/nfs/</path>
          </target>
</pool>

2. define it 
---------------------------------------------------------------
virsh # pool-define nfs-pool.xml 


3. start it, and it will hit this bug
---------------------------------------------------------------
virsh # pool-start test-nfs

  
Actual results:

virsh # pool-start test-nfs
error: Failed to start pool test-nfs
error: cannot open volume '/var/lib/libvirt/images/nfs/fcoemon.dcbd.1613': No such device or address

Expected results:
the pool defined should be started

Additional info:
Comment 2 Daniel Berrangé 2011-02-28 11:25:15 UTC 
Please show the output of 'ls -al /vol/S3/libvirtmanual' on the NFS server
Comment 3 Jason Chen 2011-03-01 02:44:11 UTC 
(In reply to comment #2)
> Please show the output of 'ls -al /vol/S3/libvirtmanual' on the NFS server

Hi
   i mount it to my local host and show it as i am not authorized to login the NFS server by SSH,the result is as below:
==========================================================================
[root@dhcp-93-23 ~]# mkdir -p /var/lib/libvirt/images/nfs/
[root@dhcp-93-23 ~]# mount 10.66.90.121:/vol/S3/libvirtmanual /var/lib/libvirt/images/nfs/
[root@dhcp-93-23 ~]# ls -al /var/lib/libvirt/images/nfs/
total 96527772
drwxrwxrwx. 18 root  root       12288 Feb 28 10:39 .
drwxr-xr-x.  3 root  root        4096 Mar  1 10:36 ..
-rw-r--r--.  1 root  root         863 Feb 16 17:57 ca-cert.pem
-rw-r--r--.  1 root  root         963 Feb 16 17:57 ca-key.pem
-rw-r--r--.  1 root  root  2147483649 Feb 23 10:04 disk1.img
drwxrwxrwt.  2 root  root        4096 Nov 19 10:11 .esd
drwx------.  2 jason jason       4096 Feb 22 12:47 .esd-500
srwxrwxrwx.  1 root  root           0 Jan 28 18:14 fcoemon.dcbd.1613
-rw-------.  1 jason jason    2670440 Feb 23 16:03 FlashXXgeHOi7
drwxrwxrwt.  2 root  root        4096 Nov 19 10:11 .ICE-unix
drwxr-xr-x.  2 root  root        4096 Dec 21 14:14 iozone3_353
-rw-r--r--.  1 root  root      213984 Feb 25 18:19 Libvirt_IEEE_Test_Plan.txt
drwx------.  2 root  root        4096 Feb 17 14:10 orbit-root
drwx------.  2 jason jason       4096 Jan  1  1970 orbit-yoyo
drwx------.  2 jason jason       4096 Feb 24 12:45 plugtmp
drwx------.  2 jason jason       4096 Feb 22 12:47 pulse-W37QXbxpdaYG
-rw-------.  1 root  root   291160064 Dec 15 15:57 RHEL5.5_i386.img
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_10.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:19 rhel61_i386_10.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_11.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:19 rhel61_i386_11.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_12.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:19 rhel61_i386_12.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_13.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:19 rhel61_i386_13.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_14.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:18 rhel61_i386_14.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:03 rhel61_i386_15.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:18 rhel61_i386_15.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_16.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:18 rhel61_i386_16.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_17.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:18 rhel61_i386_17.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_18.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:18 rhel61_i386_18.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_19.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:17 rhel61_i386_19.xml
-rw-------.  1 root  root  8589934592 Feb 25 14:31 rhel61_i386_1.img
-rw-r--r--.  1 root  root        1489 Feb 24 17:52 rhel61_i386_1.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_20.img
-rw-r--r--.  1 root  root        1472 Feb 22 11:17 rhel61_i386_20.xml
-rw-------.  1 root  root  8589934592 Feb 23 14:14 rhel61_i386_2.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_2.xml
-rw-------.  1 root  root  8589934592 Feb 23 17:11 rhel61_i386_3.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_3.xml
-rw-------.  1 root  root  8589934592 Feb 22 15:22 rhel61_i386_4.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_4.xml
-rw-------.  1 root  root  8589934592 Feb 25 19:46 rhel61_i386_5.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_5.xml
-rw-------.  1 root  root  8589934592 Feb 24 16:24 rhel61_i386_6.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_6.xml
-rw-------.  1 root  root  8589934592 Feb 24 13:44 rhel61_i386_7.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:21 rhel61_i386_7.xml
-rw-------.  1   107   107 8589934592 Feb 25 17:44 rhel61_i386_8.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:19 rhel61_i386_8.xml
-rw-------.  1 root  root  8589934592 Feb 22 16:04 rhel61_i386_9.img
-rw-r--r--.  1 root  root        1470 Feb 22 11:19 rhel61_i386_9.xml
-rw-------.  1 root  root  8589934592 Feb 23 17:01 rhel61_i386_graphic.img
-rw-------.  1 root  root  8589934592 Feb 24 13:45 rhel61_i386_no_swap.img
-rw-------.  1 root  root  8589934592 Feb 25 13:48 rhel61_x86_64.img
-rw-r--r--.  1 root  root        1745 Feb 25 13:50 rhel61_x86_64.xml
-rw-------.  1   107   107 1915944960 Feb 25 14:43 rhel6_x86_64_qcow2.img
-rw-r--r--.  1 root  root        1656 Feb 25 14:20 rhel6_x86_64_qcow2.xml
drwxr-xr-x.  2 root  root        4096 Feb 28 22:13 RHEL_KVM_images
drwxr-xr-x.  2 root  root        4096 Jan  5 20:10 RHEL-KVM-Longevity_images
-rw-r--r--.  1 root  root         741 Feb 16 17:57 server-cert.pem
-rw-r--r--.  1 root  root         603 Feb 16 17:57 server-key.csr
-rw-r--r--.  1 root  root         887 Feb 16 17:57 server-key.pem
-rw-r--r--.  1 root  root         887 Feb 16 17:57 server-key.pem.secure
drwxr-xr-x. 18 root  root        4096 Feb 23 15:08 Share
drwxrwxrwx. 10 root  root        4096 Mar  1 08:00 .snapshot
-rw-r--r--.  1 root  root        1575 Feb 16 17:56 test.sh
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm10.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm11.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm12.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm13.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm14.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm15.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm16.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm17.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm18.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm19.img
-rw-r--r--.  1 root  root  5368709120 Feb 24 10:27 vm1.img
-rw-r--r--.  1 root  root  5368709120 Feb 16 17:06 vm20.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm2.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm3.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm4.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:38 vm5.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:38 vm6.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm7.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:38 vm8.img
-rw-r--r--.  1 root  root  5368709120 Feb 11 11:39 vm9.img
drwxr-xr-x. 12 root  root        4096 Nov 19 21:04 VMware
drwxr-xr-x.  2 root  root        4096 Nov 19 15:07 Windows_KVM_images
drwxrwxrwt.  2 root  root        4096 Nov 19 10:13 .X11-unix
drwxr-xr-x.  2 root  root        8192 Dec 21 11:19 Xen_images
[root@dhcp-93-23 ~]#
Comment 4 Daniel Berrangé 2011-03-01 16:24:02 UTC 
> srwxrwxrwx.  1 root  root           0 Jan 28 18:14 fcoemon.dcbd.1613

This is the file libvirt is tripping up on. It is a UNIX socket. libvirt should skip those files when enumerating a storage pool.
Comment 5 RHEL Program Management 2011-04-04 02:07:00 UTC 
Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
Comment 6 Dave Allan 2011-06-10 02:25:43 UTC 
IIRC, this has been fixed; Jiri, can you confirm?
Comment 9 Jiri Denemark 2011-06-15 08:43:08 UTC 
Heh, I thought I already replied and I did but somehow I didn't hit "save changes" button :-)

I think this has been fixed but I wasn't able to find the exact commit. I was able to find a commit (4a1abb3f50b8cc4b32661fe978382014b93e4d7f) that allowed libvirt to ignore sockets, broken links and similar stuff that used to fail pool start. However, this commit is few month older than libvirt-0.8.7 release...
Comment 10 Dave Allan 2011-06-20 20:36:40 UTC 
Jason, can you retest with the latest 6.2 package just to make sure it's still broken?
Comment 11 Jason Chen 2011-06-21 02:12:58 UTC 
(In reply to comment #10)
> Jason, can you retest with the latest 6.2 package just to make sure it's still
> broken?

(In reply to comment #10)
> Jason, can you retest with the latest 6.2 package just to make sure it's still
> broken?

ok, i will take a try and update this bug when i finished it
Comment 12 Jason Chen 2011-06-21 06:53:24 UTC 
(In reply to comment #10)
> Jason, can you retest with the latest 6.2 package just to make sure it's still
> broken?

(In reply to comment #10)
> Jason, can you retest with the latest 6.2 package just to make sure it's still
> broken?

Dave, 
   
   i retested it with the latest 6.2 package and it's still broken.some information is as below:
 
1 steps & outputs:
-----------------------
> [root@localhost ~]# more nfs-pool.xml 
> <pool type="netfs">
>   <name>test-nfs</name>
>   <source>
>     <host name="10.66.90.121"/>
>     <dir path="/vol/S3/libvirtmanual"/>
>   </source>
>   <target>
>     <path>/var/lib/libvirt/images/nfs/</path>
>   </target>
> </pool>
> 
> [root@localhost ~]# virsh pool-define nfs-pool.xml 
> Pool test-nfs defined from nfs-pool.xml
> 
> [root@localhost ~]# virsh pool-list --all
> Name                 State      Autostart 
> -----------------------------------------
> test-nfs             inactive   no        
> 
> [root@localhost ~]# virsh pool-start test-nfs
> error: Failed to start pool test-nfs
> error: internal error '/bin/mount 10.66.90.121:/vol/S3/libvirtmanual /var/lib> /libvirt/images/nfs' exited unexpectedly: exit status 1

(the NFS source "10.66.90.121:/vol/S3/libvirtmanual" is OK)

2 test target version info:
----------------------------
> libvirt-0.9.2-1.el6.x86_64
> qemu-img-0.12.1.2-2.165.el6.x86_64
> qemu-kvm-0.12.1.2-2.165.el6.x86_64
> OS: Linux localhost.localdomain 2.6.32-158.el6.x86_64 #1 SMP Tue Jun 14 > 11:21:34 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
Comment 15 Michal Privoznik 2011-11-25 09:21:59 UTC 
Moving to POST:

commit f88de3eb5166d26bcab4ac1586cc940f20b83b73
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Nov 24 15:16:19 2011 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Nov 25 08:07:43 2011 +0100

    storage: Skip socket and fifo on pool-start
    
    If pool directory contains special files like FIFO or sockets
    we want to skip those on pool-start or pool-refresh otherwise
    open() will get an error.


v0.9.7-159-gf88de3e
Comment 16 Michal Privoznik 2011-11-25 12:47:54 UTC 
Actually, that patch removed a call to fstat() which turned out to be important. This commit fixes that issue:

commit 3ba949e8f4a77aa0d770bb9c7d71225181d88d9a
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Nov 25 13:25:19 2011 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Nov 25 13:45:36 2011 +0100

    storage: Refetch file status after open
    
    This partly reverts my previous patch f88de3eb. We need to
    get file status after open, as given path could have been symlink,
    so fstat() will operate on different file than lstat().

v0.9.7-161-g3ba949e
Comment 19 tingting zheng 2012-01-10 11:07:23 UTC 
Verified the bug with:
libvirt-0.9.9-1.el6.x86_64
qemu-kvm-0.12.1.2-2.209.el6_2.1.x86_64
qemu-img-0.12.1.2-2.209.el6_2.1.x86_64
OS:2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

Steps:
1. create one pool config file as below:
# cat nfs-pool.xml
<pool type="netfs">
     <name>test-nfs</name>
          <source>
              <host name="10.66.90.121"/>
              <dir path="/vol/S3/libvirtmanual"/>
          </source>
          <target>
              <path>/var/lib/libvirt/images/nfs/</path>
          </target>
</pool>

Check the source dir,there is a UNIX socket file:
srw-------  1 root root             0 Nov 21 02:09 fcoemon.dcbd.1673

2. define the pool. 
# virsh  pool-define nfs-pool.xml 
Pool test-nfs defined from nfs-pool.xml

3. start the pool.
# virsh pool-start test-nfs
Pool test-nfs started
Comment 21 errata-xmlrpc 2012-06-20 06:26:14 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html
Note You need to log in before you can comment on or make changes to this bug.