Bug 680880
| Summary: | The defined NFS pool can not be started | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Jason Chen <jiachen> |
| Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.1 | CC: | berrange, dallan, dyuan, eblake, gren, jiachen, llim, mzhan, rwu, tzheng, whuang, xen-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-0.9.9-1.el6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2012-06-20 06:26:14 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Please show the output of 'ls -al /vol/S3/libvirtmanual' on the NFS server (In reply to comment #2) > Please show the output of 'ls -al /vol/S3/libvirtmanual' on the NFS server Hi i mount it to my local host and show it as i am not authorized to login the NFS server by SSH,the result is as below: ========================================================================== [root@dhcp-93-23 ~]# mkdir -p /var/lib/libvirt/images/nfs/ [root@dhcp-93-23 ~]# mount 10.66.90.121:/vol/S3/libvirtmanual /var/lib/libvirt/images/nfs/ [root@dhcp-93-23 ~]# ls -al /var/lib/libvirt/images/nfs/ total 96527772 drwxrwxrwx. 18 root root 12288 Feb 28 10:39 . drwxr-xr-x. 3 root root 4096 Mar 1 10:36 .. -rw-r--r--. 1 root root 863 Feb 16 17:57 ca-cert.pem -rw-r--r--. 1 root root 963 Feb 16 17:57 ca-key.pem -rw-r--r--. 1 root root 2147483649 Feb 23 10:04 disk1.img drwxrwxrwt. 2 root root 4096 Nov 19 10:11 .esd drwx------. 2 jason jason 4096 Feb 22 12:47 .esd-500 srwxrwxrwx. 1 root root 0 Jan 28 18:14 fcoemon.dcbd.1613 -rw-------. 1 jason jason 2670440 Feb 23 16:03 FlashXXgeHOi7 drwxrwxrwt. 2 root root 4096 Nov 19 10:11 .ICE-unix drwxr-xr-x. 2 root root 4096 Dec 21 14:14 iozone3_353 -rw-r--r--. 1 root root 213984 Feb 25 18:19 Libvirt_IEEE_Test_Plan.txt drwx------. 2 root root 4096 Feb 17 14:10 orbit-root drwx------. 2 jason jason 4096 Jan 1 1970 orbit-yoyo drwx------. 2 jason jason 4096 Feb 24 12:45 plugtmp drwx------. 2 jason jason 4096 Feb 22 12:47 pulse-W37QXbxpdaYG -rw-------. 1 root root 291160064 Dec 15 15:57 RHEL5.5_i386.img -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_10.img -rw-r--r--. 1 root root 1472 Feb 22 11:19 rhel61_i386_10.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_11.img -rw-r--r--. 1 root root 1472 Feb 22 11:19 rhel61_i386_11.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_12.img -rw-r--r--. 1 root root 1472 Feb 22 11:19 rhel61_i386_12.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_13.img -rw-r--r--. 1 root root 1472 Feb 22 11:19 rhel61_i386_13.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_14.img -rw-r--r--. 1 root root 1472 Feb 22 11:18 rhel61_i386_14.xml -rw-------. 1 root root 8589934592 Feb 22 16:03 rhel61_i386_15.img -rw-r--r--. 1 root root 1472 Feb 22 11:18 rhel61_i386_15.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_16.img -rw-r--r--. 1 root root 1472 Feb 22 11:18 rhel61_i386_16.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_17.img -rw-r--r--. 1 root root 1472 Feb 22 11:18 rhel61_i386_17.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_18.img -rw-r--r--. 1 root root 1472 Feb 22 11:18 rhel61_i386_18.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_19.img -rw-r--r--. 1 root root 1472 Feb 22 11:17 rhel61_i386_19.xml -rw-------. 1 root root 8589934592 Feb 25 14:31 rhel61_i386_1.img -rw-r--r--. 1 root root 1489 Feb 24 17:52 rhel61_i386_1.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_20.img -rw-r--r--. 1 root root 1472 Feb 22 11:17 rhel61_i386_20.xml -rw-------. 1 root root 8589934592 Feb 23 14:14 rhel61_i386_2.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_2.xml -rw-------. 1 root root 8589934592 Feb 23 17:11 rhel61_i386_3.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_3.xml -rw-------. 1 root root 8589934592 Feb 22 15:22 rhel61_i386_4.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_4.xml -rw-------. 1 root root 8589934592 Feb 25 19:46 rhel61_i386_5.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_5.xml -rw-------. 1 root root 8589934592 Feb 24 16:24 rhel61_i386_6.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_6.xml -rw-------. 1 root root 8589934592 Feb 24 13:44 rhel61_i386_7.img -rw-r--r--. 1 root root 1470 Feb 22 11:21 rhel61_i386_7.xml -rw-------. 1 107 107 8589934592 Feb 25 17:44 rhel61_i386_8.img -rw-r--r--. 1 root root 1470 Feb 22 11:19 rhel61_i386_8.xml -rw-------. 1 root root 8589934592 Feb 22 16:04 rhel61_i386_9.img -rw-r--r--. 1 root root 1470 Feb 22 11:19 rhel61_i386_9.xml -rw-------. 1 root root 8589934592 Feb 23 17:01 rhel61_i386_graphic.img -rw-------. 1 root root 8589934592 Feb 24 13:45 rhel61_i386_no_swap.img -rw-------. 1 root root 8589934592 Feb 25 13:48 rhel61_x86_64.img -rw-r--r--. 1 root root 1745 Feb 25 13:50 rhel61_x86_64.xml -rw-------. 1 107 107 1915944960 Feb 25 14:43 rhel6_x86_64_qcow2.img -rw-r--r--. 1 root root 1656 Feb 25 14:20 rhel6_x86_64_qcow2.xml drwxr-xr-x. 2 root root 4096 Feb 28 22:13 RHEL_KVM_images drwxr-xr-x. 2 root root 4096 Jan 5 20:10 RHEL-KVM-Longevity_images -rw-r--r--. 1 root root 741 Feb 16 17:57 server-cert.pem -rw-r--r--. 1 root root 603 Feb 16 17:57 server-key.csr -rw-r--r--. 1 root root 887 Feb 16 17:57 server-key.pem -rw-r--r--. 1 root root 887 Feb 16 17:57 server-key.pem.secure drwxr-xr-x. 18 root root 4096 Feb 23 15:08 Share drwxrwxrwx. 10 root root 4096 Mar 1 08:00 .snapshot -rw-r--r--. 1 root root 1575 Feb 16 17:56 test.sh -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm10.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm11.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm12.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm13.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm14.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm15.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm16.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm17.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm18.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm19.img -rw-r--r--. 1 root root 5368709120 Feb 24 10:27 vm1.img -rw-r--r--. 1 root root 5368709120 Feb 16 17:06 vm20.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm2.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm3.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm4.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:38 vm5.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:38 vm6.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm7.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:38 vm8.img -rw-r--r--. 1 root root 5368709120 Feb 11 11:39 vm9.img drwxr-xr-x. 12 root root 4096 Nov 19 21:04 VMware drwxr-xr-x. 2 root root 4096 Nov 19 15:07 Windows_KVM_images drwxrwxrwt. 2 root root 4096 Nov 19 10:13 .X11-unix drwxr-xr-x. 2 root root 8192 Dec 21 11:19 Xen_images [root@dhcp-93-23 ~]# > srwxrwxrwx. 1 root root 0 Jan 28 18:14 fcoemon.dcbd.1613
This is the file libvirt is tripping up on. It is a UNIX socket. libvirt should skip those files when enumerating a storage pool.
Since RHEL 6.1 External Beta has begun, and this bug remains unresolved, it has been rejected as it is not proposed as exception or blocker. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux. IIRC, this has been fixed; Jiri, can you confirm? Heh, I thought I already replied and I did but somehow I didn't hit "save changes" button :-) I think this has been fixed but I wasn't able to find the exact commit. I was able to find a commit (4a1abb3f50b8cc4b32661fe978382014b93e4d7f) that allowed libvirt to ignore sockets, broken links and similar stuff that used to fail pool start. However, this commit is few month older than libvirt-0.8.7 release... Jason, can you retest with the latest 6.2 package just to make sure it's still broken? (In reply to comment #10) > Jason, can you retest with the latest 6.2 package just to make sure it's still > broken? (In reply to comment #10) > Jason, can you retest with the latest 6.2 package just to make sure it's still > broken? ok, i will take a try and update this bug when i finished it (In reply to comment #10) > Jason, can you retest with the latest 6.2 package just to make sure it's still > broken? (In reply to comment #10) > Jason, can you retest with the latest 6.2 package just to make sure it's still > broken? Dave, i retested it with the latest 6.2 package and it's still broken.some information is as below: 1 steps & outputs: ----------------------- > [root@localhost ~]# more nfs-pool.xml > <pool type="netfs"> > <name>test-nfs</name> > <source> > <host name="10.66.90.121"/> > <dir path="/vol/S3/libvirtmanual"/> > </source> > <target> > <path>/var/lib/libvirt/images/nfs/</path> > </target> > </pool> > > [root@localhost ~]# virsh pool-define nfs-pool.xml > Pool test-nfs defined from nfs-pool.xml > > [root@localhost ~]# virsh pool-list --all > Name State Autostart > ----------------------------------------- > test-nfs inactive no > > [root@localhost ~]# virsh pool-start test-nfs > error: Failed to start pool test-nfs > error: internal error '/bin/mount 10.66.90.121:/vol/S3/libvirtmanual /var/lib> /libvirt/images/nfs' exited unexpectedly: exit status 1 (the NFS source "10.66.90.121:/vol/S3/libvirtmanual" is OK) 2 test target version info: ---------------------------- > libvirt-0.9.2-1.el6.x86_64 > qemu-img-0.12.1.2-2.165.el6.x86_64 > qemu-kvm-0.12.1.2-2.165.el6.x86_64 > OS: Linux localhost.localdomain 2.6.32-158.el6.x86_64 #1 SMP Tue Jun 14 > 11:21:34 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux Moving to POST:
commit f88de3eb5166d26bcab4ac1586cc940f20b83b73
Author: Michal Privoznik <mprivozn>
AuthorDate: Thu Nov 24 15:16:19 2011 +0100
Commit: Michal Privoznik <mprivozn>
CommitDate: Fri Nov 25 08:07:43 2011 +0100
storage: Skip socket and fifo on pool-start
If pool directory contains special files like FIFO or sockets
we want to skip those on pool-start or pool-refresh otherwise
open() will get an error.
v0.9.7-159-gf88de3e
Actually, that patch removed a call to fstat() which turned out to be important. This commit fixes that issue:
commit 3ba949e8f4a77aa0d770bb9c7d71225181d88d9a
Author: Michal Privoznik <mprivozn>
AuthorDate: Fri Nov 25 13:25:19 2011 +0100
Commit: Michal Privoznik <mprivozn>
CommitDate: Fri Nov 25 13:45:36 2011 +0100
storage: Refetch file status after open
This partly reverts my previous patch f88de3eb. We need to
get file status after open, as given path could have been symlink,
so fstat() will operate on different file than lstat().
v0.9.7-161-g3ba949e
Verified the bug with:
libvirt-0.9.9-1.el6.x86_64
qemu-kvm-0.12.1.2-2.209.el6_2.1.x86_64
qemu-img-0.12.1.2-2.209.el6_2.1.x86_64
OS:2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
Steps:
1. create one pool config file as below:
# cat nfs-pool.xml
<pool type="netfs">
<name>test-nfs</name>
<source>
<host name="10.66.90.121"/>
<dir path="/vol/S3/libvirtmanual"/>
</source>
<target>
<path>/var/lib/libvirt/images/nfs/</path>
</target>
</pool>
Check the source dir,there is a UNIX socket file:
srw------- 1 root root 0 Nov 21 02:09 fcoemon.dcbd.1673
2. define the pool.
# virsh pool-define nfs-pool.xml
Pool test-nfs defined from nfs-pool.xml
3. start the pool.
# virsh pool-start test-nfs
Pool test-nfs started
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-0748.html |
Description of problem: it can not start the defined nfs pool with the command "pool-start ..." , and all ways complaining the following: " cannot open volume '/var/lib/libvirt/images/nfs/fcoemon.dcbd.1613': No such device or address " Version-Release number of selected component (if applicable): [1] OS version: ------------------------ [root@dhcp-93-122 test]# uname -a Linux dhcp-93-122.nay.redhat.com 2.6.32-117.el6.x86_64 #1 SMP Tue Feb 15 16:29:34 EST 2011 x86_64 x86_64 x86_64 GNU/Linux [2] libvirt version: ------------------------ [root@dhcp-93-122 test]# rpm -qa |grep libvirt libvirt-0.8.7-7.el6.x86_64 libvirt-debuginfo-0.8.7-7.el6.x86_64 libvirt-client-0.8.7-7.el6.x86_64 libvirt-devel-0.8.7-7.el6.x86_64 libvirt-python-0.8.7-7.el6.x86_64 [3] qemu-kvm version: ------------------------ [root@dhcp-93-122 test]# rpm -qa |grep qemu qemu-kvm-debuginfo-0.12.1.2-2.146.el6.x86_64 qemu-kvm-tools-0.12.1.2-2.146.el6.x86_64 qemu-img-0.12.1.2-2.146.el6.x86_64 gpxe-roms-qemu-0.9.7-6.4.el6.noarch qemu-kvm-0.12.1.2-2.146.el6.x86_64 [4] kernel version: ------------------------ [root@dhcp-93-122 test]# rpm -qa|grep kernel dracut-kernel-004-36.el6.noarch kernel-firmware-2.6.32-117.el6.noarch abrt-addon-kerneloops-1.1.15-1.el6.x86_64 kernel-2.6.32-94.el6.x86_64 kernel-2.6.32-117.el6.x86_64 kernel-headers-2.6.32-117.el6.x86_64 How reproducible: Steps to Reproduce: 1. create one pool config file as below: --------------------------------------------------------------- [root@dhcp-93-122 test]# more nfs-pool.xml <pool type="netfs"> <name>test-nfs</name> <source> <host name="10.66.90.121"/> <dir path="/vol/S3/libvirtmanual"/> </source> <target> <path>/var/lib/libvirt/images/nfs/</path> </target> </pool> 2. define it --------------------------------------------------------------- virsh # pool-define nfs-pool.xml 3. start it, and it will hit this bug --------------------------------------------------------------- virsh # pool-start test-nfs Actual results: virsh # pool-start test-nfs error: Failed to start pool test-nfs error: cannot open volume '/var/lib/libvirt/images/nfs/fcoemon.dcbd.1613': No such device or address Expected results: the pool defined should be started Additional info: