Bug 1167145

Summary: networkMigrateStateFiles function does not work on xfs file system due to using unsupported t_type field
Product: Red Hat Enterprise Linux 7 Reporter: Hu Jianwei <jiahu>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.1CC: dyuan, honzhang, jiahu, mzhan, rbalakri, zhwang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.8-10.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:47:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hu Jianwei 2014-11-24 01:54:22 UTC
Description of problem:
networkMigrateStateFiles function does not work on xfs file system due to using unsupported t_type field

Version-Release number of selected component (if applicable):
libvirt-1.2.8-7.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
Upgrading libvirt, but libvirt can not save network status files to the new location, and remove them from the old location.

[root@localhost libvirt_wget]# rpm -q libvirt
libvirt-1.1.1-29.el7.x86_64
[root@localhost libvirt_wget]#
[root@localhost libvirt_wget]# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 bridge-net           active     no            yes
 default              active     yes           yes
 route-net            active     no            yes

[root@localhost libvirt_wget]# ll /var/lib/libvirt/network/
total 12
-rw-------. 1 root root 501 Nov 21 16:56 bridge-net.xml
-rw-------. 1 root root 760 Nov 21 16:56 default.xml
-rw-------. 1 root root 738 Nov 21 16:56 route-net.xml
[root@localhost libvirt_wget]# ll /var/run/libvirt/network/
total 8
-rw-r--r--. 1 root root 6 Nov 21 16:56 default.pid
-rw-r--r--. 1 root root 0 Nov 21 16:56 nwfilter.leases
-rw-r--r--. 1 root root 6 Nov 21 16:56 route-net.pid

[root@localhost libvirt-1.2.8]# rpm -q libvirt
libvirt-1.2.8-7.el7.x86_64

[root@localhost libvirt-1.2.8-7.el7]# service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service
[root@localhost libvirt-1.2.8-7.el7]# virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 bridge-net           inactive   no            yes
 default              inactive   yes           yes
 route-net            inactive   no            yes

[root@localhost libvirt-1.2.8-7.el7]# ll /var/lib/libvirt/network/
total 12
-rw-------. 1 root root 501 Nov 21 17:12 bridge-net.xml
-rw-------. 1 root root 760 Nov 21 17:11 default.xml
-rw-------. 1 root root 738 Nov 21 17:11 route-net.xml
[root@localhost libvirt-1.2.8-7.el7]# ll /var/run/libvirt/network/
total 8
-rw-r--r--. 1 root root 6 Nov 21 17:11 default.pid
-rw-r--r--. 1 root root 0 Nov 21 18:14 nwfilter.leases
-rw-r--r--. 1 root root 6 Nov 21 17:11 route-net.pid


My debugging info:
[root@localhost libvirt-1.2.8]# rpm -q libvirt
libvirt-1.2.8-7.el7.x86_64

Breakpoint 2, networkStateInitialize (privileged=true, callback=0x55555556ade0 <daemonInhibitCallback>, opaque=0x5555557e6f70) at network/bridge_driver.c:559
510	    while ((direrr = virDirRead(dir, &entry, oldStateDir)) > 0) {
(gdb) 
512	        if (entry->d_type != DT_REG ||
(gdb) p *entry
$28 = {
  d_ino = 135884899, 
  d_off = 25, 
  d_reclen = 32, 
  d_type = 0 '\000', 
  d_name = "default.xml\000\000dp\031\b\000\000\000\000#\000\000\000\000\000\000\000(\000\000route-net.xml\000^^^^^^\000hp\031\b\000\000\000\000\000\002\000\000\000\000\000\000(\000\000bridge-net.xml\000^^^^^\000", '^' <repeats 163 times>
}
(gdb)

[root@localhost libvirt-1.2.8]# cat /etc/fstab 
#
# /etc/fstab
# Created by anaconda on Tue Nov 18 09:45:04 2014
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/rhel-root   /                       xfs     defaults        0 0
UUID=40589125-bead-4e52-8ccf-c52991546bed /boot                   xfs     defaults        0 0
/dev/mapper/rhel-home   /home                   xfs     defaults        0 0
/dev/mapper/rhel-swap   swap                    swap    defaults        0 0


Actual results:
As shown above steps, the d_type return 0(unknown) for a regular file.

Expected results:
Fix it.

Additional info:
       The d_type field is implemented since Linux 2.6.4.  It occupies a
       space that was previously a zero-filled padding byte in the
       linux_dirent structure.  Thus, on kernels before 2.6.3, attempting to
       access this field always provides the value 0 (DT_UNKNOWN).

       Currently, only some filesystems (among them: Btrfs, ext2, ext3, and
       ext4) have full support for returning the file type in d_type.  All
       applications must properly handle a return of DT_UNKNOWN.
from http://man7.org/linux/man-pages/man2/getdents.2.html

Comment 2 Jiri Denemark 2014-11-27 09:05:58 UTC
Fixed upstream by v1.2.10-209-gdabb23e:

commit dabb23e6d95dc7d81e7fb2a3f6c942167f4c45af
Author: Jiri Denemark <jdenemar>
Date:   Wed Nov 26 16:24:27 2014 +0100

    network: Fix upgrade from libvirt older than 1.2.4
    
    Starting from libvirt-1.2.4, network state XML files moved to another
    directory (see commit b9e95491) and libvirt automatically migrates the
    network state files to a new location. However, the code used
    dirent.d_type which is not supported by all filesystems. Thus, when
    libvirt was upgraded on a host which used such filesystem, network state
    XMLs were not properly moved and running networks disappeared from
    libvirt.
    
    This patch falls back to lstat() whenever dirent.d_type is DT_UNKNOWN to
    fix this issue.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1167145
    Signed-off-by: Jiri Denemark <jdenemar>

Comment 5 Hu Jianwei 2014-12-04 02:55:49 UTC
Verify as below:

[root@ibm-x3850x5-06 ~]# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Sep 16 08:16:36 2014
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=04c33841-7f7e-479b-b7f2-41855520d345 /                       xfs     defaults        1 1
UUID=e9ce0f57-651a-43fb-ac35-4e660926184f /tmp                    xfs     defaults        1 2
UUID=3679443b-b4b5-4f84-9079-bce34cac58fa swap                    swap    defaults        0 0

[root@ibm-x3850x5-06 ~]# rpm -q libvirt
libvirt-1.1.1-29.el7.x86_64
[root@ibm-x3850x5-06 ~]# ll /var/lib/libvirt/network/
total 12
-rw-------. 1 root root 501 Dec  4 09:55 bridge-net.xml
-rw-------. 1 root root 860 Dec  4 09:55 default.xml
-rw-------. 1 root root 741 Dec  4 09:55 route-net.xml
[root@ibm-x3850x5-06 ~]# ll /var/run/libvirt/network/
total 8
-rw-r--r--. 1 root root 5 Dec  4 09:55 default.pid
-rw-r--r--. 1 root root 5 Dec  4 09:55 route-net.pid


[root@ibm-x3850x5-06 ~]# rpm -q libvirt
libvirt-1.2.8-10.el7.x86_64
[root@ibm-x3850x5-06 ~]# 
[root@ibm-x3850x5-06 ~]# virsh net-list 
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 bridge-net           active     no            yes
 default              active     yes           yes
 route-net            active     no            yes

[root@ibm-x3850x5-06 ~]# ll /var/lib/libvirt/network/
total 0
[root@ibm-x3850x5-06 ~]# ll /var/run/libvirt/network/
total 20
-rw-------. 1 root root 501 Dec  4 09:56 bridge-net.xml
-rw-r--r--. 1 root root   5 Dec  4 09:55 default.pid
-rw-------. 1 root root 860 Dec  4 09:56 default.xml
-rw-r--r--. 1 root root   0 Dec  4 09:56 nwfilter.leases
-rw-r--r--. 1 root root   5 Dec  4 09:55 route-net.pid
-rw-------. 1 root root 741 Dec  4 09:56 route-net.xml

We can get a right result on xfs file system, move to Verified.

Comment 7 errata-xmlrpc 2015-03-05 07:47:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html