Bug 1208619

Summary: Create directory for lease files if it's missing (i.e. if no net is autostarted)
Product: Red Hat Enterprise Linux 6 Reporter: Jan Kurik <jkurik>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.7CC: bazulay, danken, dkuznets, dyuan, ecohen, gklein, iheim, jherrman, jiahu, lpeer, lsurette, meverett, mprivozn, mzhan, oourfali, pm-eus, pstehlik, rbalakri, tlavigne, xuzhang, ybronhei, yeylon
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.10.2-46.el6_6.5 Doc Type: Bug Fix
Doc Text:
Prior to this update, when starting the libvirt network filter (nwfilter) while no network was available, the nwfilter driver initialization failed. With this update, nwfilter automatically creates a path for the nwfilter.leases file. This allows nwfilter to be independent from the network driver and to be able to start with no network available.
Story Points: ---
Clone Of: 1200991 Environment:
Last Closed: 2015-04-29 10:10:09 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1200991    
Bug Blocks:    

Description Jan Kurik 2015-04-02 17:09:56 UTC
This bug has been copied from bug #1200991 and has been proposed
to be backported to 6.6 z-stream (EUS).

Comment 5 Hu Jianwei 2015-04-10 07:20:34 UTC
I can not reproduce it on the latest version.

<1> Before restarting host OS, disabled all autostart virtual networks and removed /var/run/libvirt/network folder.
[root@intel-e31225-8-3 ~]# rpm -q libvirt
libvirt-0.10.2-46.el6_6.5.x86_64

[root@intel-e31225-8-3 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     no            yes

[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/network
ls: cannot access /var/run/libvirt/network: No such file or directory
[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/
total 4
srwxrwxrwx. 1 root root    0 Apr 10 14:58 libvirt-sock
srwxrwxrwx. 1 root root    0 Apr 10 14:58 libvirt-sock-ro
drwxr-xr-x. 2 root root 4096 Apr 10 14:59 qemu

[root@intel-e31225-8-3 ~]# 
[root@intel-e31225-8-3 ~]# reboot

Broadcast message from root.nay.redhat.com
	(/dev/pts/0) at 15:03 ...

The system is going down for reboot NOW!
[root@intel-e31225-8-3 ~]# Connection to 10.66.85.74 closed by remote host.
Connection to 10.66.85.74 closed.

<2> After starting the host OS again:
[hujianwei@localhost libvirt]$ ssh root.85.74 -X
root.85.74's password: 
Last login: Wed Apr  8 20:47:48 2015 from vpn1-6-169.pek2.redhat.com
[root@intel-e31225-8-3 ~]# cat /var/log/libvirt/libvirtd.log | grep "virNWFilterSnoopLeaseFileRefresh" -b5
<=== No error messages output(for detailed error message in last version, see bug 1200991)

[root@intel-e31225-8-3 ~]# ll /var/run/libvirt
total 8
srwxrwxrwx. 1 root root    0 Apr 10 15:06 libvirt-sock
srwxrwxrwx. 1 root root    0 Apr 10 15:06 libvirt-sock-ro
drwx------. 2 root root 4096 Apr 10 15:06 network
drwxr-xr-x. 2 root root 4096 Apr 10 15:04 qemu

[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/network/
total 0
-rw-r--r--. 1 root root 0 Apr 10 15:06 nwfilter.leases

[root@intel-e31225-8-3 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              inactive   no            yes

[root@intel-e31225-8-3 ~]# virsh dumpxml virt-tests-vm1 | grep "/interface" -B8
    <interface type='network'>
      <mac address='52:54:00:88:89:8a'/>
      <source network='default'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'>
        <parameter name='CTRL_IP_LEARNING' value='dhcp'/>
      </filterref>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

[root@intel-e31225-8-3 ~]# virsh net-start default
Network default started

[root@intel-e31225-8-3 ~]# virsh start virt-tests-vm1
Domain virt-tests-vm1 started

[root@intel-e31225-8-3 ~]# cat /var/run/libvirt/network/nwfilter.leases 
1428653349 a62c462d-bd34-742d-943a-643ba6322da7-52:54:00:88:89:8A 192.168.122.166 192.168.122.1
[root@intel-e31225-8-3 ~]# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j libvirt-I-vnet0

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j libvirt-O-vnet0

Bridge chain: libvirt-I-vnet0, entries: 9, policy: ACCEPT
-j I-vnet0-mac
-p IPv4 -j I-vnet0-ipv4-ip
-p IPv4 -j ACCEPT 
-p ARP -j I-vnet0-arp-mac
-p ARP -j I-vnet0-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet0-rarp
-p 0x835 -j ACCEPT 
-j DROP 

Bridge chain: libvirt-O-vnet0, entries: 4, policy: ACCEPT
-p IPv4 -j O-vnet0-ipv4
-p ARP -j ACCEPT 
-p 0x8035 -j O-vnet0-rarp
-j DROP 

Bridge chain: I-vnet0-mac, entries: 2, policy: ACCEPT
-s 52:54:0:88:89:8a -j RETURN 
-j DROP 

Bridge chain: I-vnet0-ipv4-ip, entries: 3, policy: ACCEPT
-p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
-p IPv4 --ip-src 192.168.122.166 -j RETURN 
-j DROP 

Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
-j ACCEPT 

Bridge chain: I-vnet0-arp-mac, entries: 2, policy: ACCEPT
-p ARP --arp-mac-src 52:54:0:88:89:8a -j RETURN 
-j DROP 

Bridge chain: I-vnet0-arp-ip, entries: 2, policy: ACCEPT
-p ARP --arp-ip-src 192.168.122.166 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -s 52:54:0:88:89:8a -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:88:89:8a --arp-mac-dst 52:54:0:88:89:8a -j ACCEPT 
-j DROP 

Bridge chain: O-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:88:89:8a --arp-mac-dst 52:54:0:88:89:8a -j ACCEPT 
-j DROP 

According to patch of comment 3, we can get expected results.

Comment 6 Hu Jianwei 2015-04-10 07:37:49 UTC
Hi Gil,

The original bug comes from vdsm/rhevm team, I've verified it from libvirt side.
Could you please have a try in your team's test environment using the fixed version? 

Thanks.

Comment 7 Gil Klein 2015-04-12 09:27:40 UTC
(In reply to Hu Jianwei from comment #6)
> Hi Gil,
> 
> The original bug comes from vdsm/rhevm team, I've verified it from libvirt
> side.
> Could you please have a try in your team's test environment using the fixed
> version? 
Hu, where can we get the relevant libvirt build for testing this fix?

Pavel, can you please give it a try as soon as we got the libvirt build for testing?

Comment 10 Hu Jianwei 2015-04-14 06:02:49 UTC
Appended new testing results for libvirt-0.10.2-46.el6_6.6.x86_64, PASS.

[root@intel-e31225-8-3 ~]# rpm -q libvirt
libvirt-0.10.2-46.el6_6.6.x86_64

[root@intel-e31225-8-3 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     no            yes

[root@intel-e31225-8-3 ~]# rm -rf /var/run/libvirt/network
[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/network
ls: cannot access /var/run/libvirt/network: No such file or directory
[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/
total 4
srwxrwxrwx. 1 root root    0 Apr 14 13:50 libvirt-sock
srwxrwxrwx. 1 root root    0 Apr 14 13:50 libvirt-sock-ro
drwxr-xr-x. 2 root root 4096 Apr 14 13:51 qemu
[root@intel-e31225-8-3 ~]# reboot

Broadcast message from root.nay.redhat.com
	(/dev/pts/0) at 13:53 ...

The system is going down for reboot NOW!
[root@intel-e31225-8-3 ~]# Connection to 10.66.85.74 closed by remote host.
Connection to 10.66.85.74 closed.
[hujianwei@localhost libvirt]$ 
[hujianwei@localhost libvirt]$ ssh root.85.74 -X
ssh: connect to host 10.66.85.74 port 22: Connection refused
[hujianwei@localhost libvirt]$ ssh root.85.74 -X
root.85.74's password: 
Last login: Mon Apr 13 10:36:37 2015 from 10.66.7.130
[root@intel-e31225-8-3 ~]# cat /var/log/libvirt/libvirtd.log | grep "virNWFilterSnoopLeaseFileRefresh" -b5
<=== No output from above command line
[root@intel-e31225-8-3 ~]# ll /var/run/libvirt
total 8
srwxrwxrwx. 1 root root    0 Apr 14 13:56 libvirt-sock
srwxrwxrwx. 1 root root    0 Apr 14 13:56 libvirt-sock-ro
drwx------. 2 root root 4096 Apr 14 13:56 network
drwxr-xr-x. 2 root root 4096 Apr 14 13:54 qemu
[root@intel-e31225-8-3 ~]# ll /var/run/libvirt/network/
total 0
-rw-r--r--. 1 root root 0 Apr 14 13:56 nwfilter.leases
[root@intel-e31225-8-3 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              inactive   no            yes

[root@intel-e31225-8-3 ~]# virsh dumpxml virt-tests-vm1 | grep "/interface" -B8
    <interface type='network'>
      <mac address='52:54:00:88:89:8a'/>
      <source network='default'/>
      <model type='virtio'/>
      <filterref filter='clean-traffic'>
        <parameter name='CTRL_IP_LEARNING' value='dhcp'/>
      </filterref>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
[root@intel-e31225-8-3 ~]# virsh net-start default
Network default started

[root@intel-e31225-8-3 ~]# virsh start virt-tests-vm1
Domain virt-tests-vm1 started

[root@intel-e31225-8-3 ~]# cat /var/run/libvirt/network/nwfilter.leases
1428994703 a62c462d-bd34-742d-943a-643ba6322da7-52:54:00:88:89:8A 192.168.122.166 192.168.122.1
[root@intel-e31225-8-3 ~]# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j libvirt-I-vnet0

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j libvirt-O-vnet0

Bridge chain: libvirt-I-vnet0, entries: 9, policy: ACCEPT
-j I-vnet0-mac
-p IPv4 -j I-vnet0-ipv4-ip
-p IPv4 -j ACCEPT 
-p ARP -j I-vnet0-arp-mac
-p ARP -j I-vnet0-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet0-rarp
-p 0x835 -j ACCEPT 
-j DROP 

Bridge chain: libvirt-O-vnet0, entries: 4, policy: ACCEPT
-p IPv4 -j O-vnet0-ipv4
-p ARP -j ACCEPT 
-p 0x8035 -j O-vnet0-rarp
-j DROP 

Bridge chain: I-vnet0-mac, entries: 2, policy: ACCEPT
-s 52:54:0:88:89:8a -j RETURN 
-j DROP 

Bridge chain: I-vnet0-ipv4-ip, entries: 3, policy: ACCEPT
-p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
-p IPv4 --ip-src 192.168.122.166 -j RETURN 
-j DROP 

Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
-j ACCEPT 

Bridge chain: I-vnet0-arp-mac, entries: 2, policy: ACCEPT
-p ARP --arp-mac-src 52:54:0:88:89:8a -j RETURN 
-j DROP 

Bridge chain: I-vnet0-arp-ip, entries: 2, policy: ACCEPT
-p ARP --arp-ip-src 192.168.122.166 -j RETURN 
-j DROP 

Bridge chain: I-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -s 52:54:0:88:89:8a -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:88:89:8a --arp-mac-dst 52:54:0:88:89:8a -j ACCEPT 
-j DROP 

Bridge chain: O-vnet0-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:88:89:8a --arp-mac-dst 52:54:0:88:89:8a -j ACCEPT 
-j DROP

Comment 11 dyuan 2015-04-16 02:31:21 UTC
(In reply to Gil Klein from comment #7)
> (In reply to Hu Jianwei from comment #6)
> > Hi Gil,
> > 
> > The original bug comes from vdsm/rhevm team, I've verified it from libvirt
> > side.
> > Could you please have a try in your team's test environment using the fixed
> > version? 
> Hu, where can we get the relevant libvirt build for testing this fix?
> 
> Pavel, can you please give it a try as soon as we got the libvirt build for
> testing?

Restore the needinfo to Pavel for double check, thanks.

Comment 14 errata-xmlrpc 2015-04-29 10:10:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0916.html