Bug 974197 - ip requests 192.168.4.1 but interface gets 192.168.4.0
ip requests 192.168.4.1 but interface gets 192.168.4.0
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: quagga (Show other bugs)
19
x86_64 Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Michal Sekletar
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-13 11:53 EDT by Erik Logtenberg
Modified: 2014-12-08 10:38 EST (History)
20 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-12-08 10:38:54 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Erik Logtenberg 2013-06-13 11:53:18 EDT
Description of problem:

I configured virbr0 to have IP address 192.168.4.1 but it gets assigned 192.168.4.0.

Version-Release number of selected component (if applicable):
libvirt-1.0.5.1-1.fc19.x86_64

My configuration of the default network:

<network>
  <name>default</name>
  <uuid>cd32c418-1f1e-42ff-abba-38969495f9a5</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:33:45:f9'/>
  <ip address='192.168.4.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.4.2' end='192.168.4.254' />
    </dhcp>
  </ip>
</network>

Actual result:

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.4.0  netmask 255.255.255.0  broadcast 192.168.4.255
        ether 52:54:00:33:45:f9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 2503 (2.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I used this same configuration with Fedora 18 and previous versions, and it always worked fine this way. With Fedora 19 I get these results however.

If I net-destroy default and then net-start default with virsh, virbr0 gets removed and re-added with still the wrong IP-address.
Comment 1 Laine Stump 2013-06-13 14:29:16 EDT
I'm not sure what to make of this. I just tried the same xml (cut-pasted from the bug description) on an updated F19 system with libvirt-1.0.5.1-1 and it properly set the IP address to 192.168.4.1.

My system has both the updates and updates-testing repos enabled, and was updated just an hour or so ago. Is yours similarly up to date? If not, can you enable those repos, update, then try again? 

Can anyone think of a reason outside libvirt that might cause this?
Comment 2 Erik Logtenberg 2013-06-13 17:32:14 EDT
Thanks for trying to help out. I have both repos you mention also enabled and fully updated earlier today as well.
I posted this bug under libvirt but I'm not sure if it's really libvirt causing this. If there's any way I can get useful debug information for you please let me know and I'll provide it.
Comment 3 Erik Logtenberg 2013-07-05 10:32:58 EDT
Is there any more information that I can provide to help solve this issue?

In the mean time I've updated to libvirt-1.0.5.2-1.fc19.x86_64 but the problem still remains.
Comment 4 Daniel Berrange 2013-07-05 10:43:44 EDT
I've just tested this config and, like laine, I can't reproduce the problem described.

Please run this as root in one shell:

 # strace -p <pid-of-libvirtd-daemon> -q -e trace=execve -s 1000 -f


then in another shell run:

 # virsh net-start default


One of the last lines printed should be

[pid  9223] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0


Be interested to know if the IP address is correct in that command line for you
Comment 5 Erik Logtenberg 2013-07-05 11:40:23 EDT
[pid 28020] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0

Interestingly, that line appears to be correct. It does say 192.168.4.1/24 in that command line, however this is the resulting interface:

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.4.0  netmask 255.255.255.0  broadcast 192.168.4.255
        ether 52:54:00:33:45:f9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 19  bytes 3744 (3.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Comment 6 Daniel Berrange 2013-07-05 11:48:46 EDT
So as far as libvirt is concerned it has done the right thing. Either ip command or the kernel is screwing up the address, or some other program is changing it after the fact.

I assume you've made sure every RPM is fully up2date wrt final Fedora 19 release packages & rebooted to ensure running the latest kernel ?
Comment 7 Erik Logtenberg 2013-07-05 12:06:40 EDT
Actually I was running kernel-3.9.4-301.fc19.x86_64, but I had already installed kernel-3.9.8-300.fc19.x86_64, so I've just rebooted to try if that helps. Unfortunately no sigar.

Still same result with latest kernel.
Comment 8 Cole Robinson 2013-08-30 19:00:49 EDT
Erik, can you still reproduce with up to date F19?
What OS is the guest running?
Can you post the XML config of your guest?
Comment 9 Erik Logtenberg 2013-09-02 03:58:35 EDT
Yes, every time the machine boots, virbr0 gets the 192.168.4.0 address instead of 192.168.4.1. I manually set it right every time.

You ask for my guest config, but this is without any guest running. There are several different guests, both Linux and Windows, that I successfully run (after fixing the right IP adress for virbr0), but the issue is already there before I start any guest.

I would provide the /etc/sysconfig/network-scripts/ifcfg-virbr0 file if I could, but it doesn't exist. The interface is managed by libvirt and not started by the regular network starting scripts.
So the xml part in my opening post seems to be the only configuration for this interface.
Comment 10 Cole Robinson 2013-09-09 09:03:17 EDT
Sorry about the confusion wrt guest/host. As mentioned by danpb it certainly looks like libvirt is doing everything it can here. Reassigning to kernel
Comment 11 Michele Baldessari 2013-09-09 10:12:16 EDT
Hi Erik,

so libvirt is doing a "[pid 28020] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0".

Can you confirm that running the command 'ip a a 192.168.1.4/24 broadcast 192.168.4.255 dev virbr0' gives you the same result (i.e. 192.168.4.0/24 assigned to the interface)?

Could you share the full output of 'ip a' ?

Thanks,
Michele
Comment 12 Josh Boyer 2013-09-18 16:38:27 EDT
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.11.1-200.fc19.  Please test this kernel update and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.
Comment 13 Michele Baldessari 2013-09-20 03:55:18 EDT
Hi Eric,

can you let me know the info asked in c#11? If the issue persists please try with 3.11.1-200.fc19 and get me the output of those commands as well please.

Thanks,
Michele
Comment 14 Michele Baldessari 2013-10-05 06:16:40 EDT
Ping Eric?
Comment 15 Neil Horman 2013-10-05 14:05:17 EDT
FWIW, This strikes me as though it might be some odd configuration error - As noted, from the xml configuration above this should be working, and is, by all attempts at reproduction here.  That said, I'd start looking at at other configurations - for instance, is there perhaps a configuration file for virbr0 in /etc/sysconfig/network-scripts or some such?  If the interface were already created by network manager and had a static ip assigned to it, the only error you might get is something about the entry already being there when libvirtd started.  Can you check that?  Also, can you attach a sosreport, or at least your /var/log/messages file so we can see if the interface throws any errors before you expect it?
Comment 16 Josh Boyer 2013-10-08 13:12:17 EDT
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 2 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.
Comment 17 Erik Logtenberg 2013-10-10 09:40:44 EDT
Hi, thanks for the extra questions, I'll try to find that out. I don't really reboot this machine often and I see this only after a fresh boot. I'll try to get some answers to Neil and Michele's questions.
Comment 18 Erik Logtenberg 2014-08-07 17:22:52 EDT
I finally finally found out what causes this problem.

It turns out it has nothing to do with virtualization at all. It's an issue with zebra (part of quagga). In the systemd unit file for zebra, it says:

Before=network.target

The big problem with this is, that starting zebra before the interface is up, causes the interface to be brought up in the wrong way, namely with the network address (in this case 192.168.4.0) as primary ip address.
As soon as the network officially gets started, the real ip address is also added, but as secondary. With ifconfig you can't see this but with "ip addr" you actually can.

If zebra gets started after the interface is already properly brought up, it doesn't add the wrong ip address. This is why I couldn't reproduce this bug after the machine had already been booted.

The fix is easy, just change the zebra unit file to:

After=network.target

Now, it all works just fine.
Comment 19 Michal Sekletar 2014-08-08 04:15:01 EDT
I am not sure that ordering quagga after network.target is what we want in general. quagga should be considered as a service which implements networking, thus it should pull the network.target in and order itself before it.
Comment 20 Erik Logtenberg 2014-08-08 04:23:01 EDT
Well, this is the workaround that got it fixed quickly for me. I understand that in general it is better to solve the actual problem instead. Do you have enough information to track the underlying bug down or should I provide more?
Comment 21 Michal Sekletar 2014-08-08 04:32:21 EDT
I am at Flock right now, so I'll look into this probably sometime next week. I may or may not have further questions :)
Comment 22 Erik Logtenberg 2014-08-08 04:41:47 EDT
Okay, you will probably want to know what's in my /etc/quagga/zebra.conf, so here it is. My /etc/sysconfig/quagga file is unmodified.


hostname testbox

password xxx
enable password xxx
service password-encryption

debug zebra events
debug zebra kernel

interface lo
          description loopback

interface eth0
          description Office 192.168.46.0/24
          ip address 192.168.46.0/24

interface virbr0
          description testbox intern 192.168.4.0/24
          ip address 192.168.4.0/24

log file /var/log/quagga/zebra.log
Comment 23 Michal Sekletar 2014-12-08 10:38:54 EST
Sorry for not getting back to you earlier. Given configuration in comment #22 I don't consider described problem as a bug in quagga. Like I said earlier quagga should be ordered before network target as it is a service implementing network configuration.

If you don't like that quagga is configuring an ip address on the particular interface well then don't tell it to do so.

Note You need to log in before you can comment on or make changes to this bug.