Bug 974197

I'm not sure what to make of this. I just tried the same xml (cut-pasted from the bug description) on an updated F19 system with libvirt-1.0.5.1-1 and it properly set the IP address to 192.168.4.1.

My system has both the updates and updates-testing repos enabled, and was updated just an hour or so ago. Is yours similarly up to date? If not, can you enable those repos, update, then try again? 

Can anyone think of a reason outside libvirt that might cause this?

Thanks for trying to help out. I have both repos you mention also enabled and fully updated earlier today as well.
I posted this bug under libvirt but I'm not sure if it's really libvirt causing this. If there's any way I can get useful debug information for you please let me know and I'll provide it.

Is there any more information that I can provide to help solve this issue?

In the mean time I've updated to libvirt-1.0.5.2-1.fc19.x86_64 but the problem still remains.

I've just tested this config and, like laine, I can't reproduce the problem described.

Please run this as root in one shell:

 # strace -p <pid-of-libvirtd-daemon> -q -e trace=execve -s 1000 -f


then in another shell run:

 # virsh net-start default


One of the last lines printed should be

[pid  9223] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0


Be interested to know if the IP address is correct in that command line for you

[pid 28020] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0

Interestingly, that line appears to be correct. It does say 192.168.4.1/24 in that command line, however this is the resulting interface:

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.4.0  netmask 255.255.255.0  broadcast 192.168.4.255
        ether 52:54:00:33:45:f9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 19  bytes 3744 (3.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

So as far as libvirt is concerned it has done the right thing. Either ip command or the kernel is screwing up the address, or some other program is changing it after the fact.

I assume you've made sure every RPM is fully up2date wrt final Fedora 19 release packages & rebooted to ensure running the latest kernel ?

Actually I was running kernel-3.9.4-301.fc19.x86_64, but I had already installed kernel-3.9.8-300.fc19.x86_64, so I've just rebooted to try if that helps. Unfortunately no sigar.

Still same result with latest kernel.

Erik, can you still reproduce with up to date F19?
What OS is the guest running?
Can you post the XML config of your guest?

Yes, every time the machine boots, virbr0 gets the 192.168.4.0 address instead of 192.168.4.1. I manually set it right every time.

You ask for my guest config, but this is without any guest running. There are several different guests, both Linux and Windows, that I successfully run (after fixing the right IP adress for virbr0), but the issue is already there before I start any guest.

I would provide the /etc/sysconfig/network-scripts/ifcfg-virbr0 file if I could, but it doesn't exist. The interface is managed by libvirt and not started by the regular network starting scripts.
So the xml part in my opening post seems to be the only configuration for this interface.

Sorry about the confusion wrt guest/host. As mentioned by danpb it certainly looks like libvirt is doing everything it can here. Reassigning to kernel

Hi Erik,

so libvirt is doing a "[pid 28020] execve("/usr/sbin/ip", ["/usr/sbin/ip", "addr", "add", "192.168.4.1/24", "broadcast", "192.168.4.255", "dev", "virbr0"], [/* 3 vars */]) = 0".

Can you confirm that running the command 'ip a a 192.168.1.4/24 broadcast 192.168.4.255 dev virbr0' gives you the same result (i.e. 192.168.4.0/24 assigned to the interface)?

Could you share the full output of 'ip a' ?

Thanks,
Michele

*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.11.1-200.fc19.  Please test this kernel update and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Hi Eric,

can you let me know the info asked in c#11? If the issue persists please try with 3.11.1-200.fc19 and get me the output of those commands as well please.

Thanks,
Michele

Ping Eric?

FWIW, This strikes me as though it might be some odd configuration error - As noted, from the xml configuration above this should be working, and is, by all attempts at reproduction here.  That said, I'd start looking at at other configurations - for instance, is there perhaps a configuration file for virbr0 in /etc/sysconfig/network-scripts or some such?  If the interface were already created by network manager and had a static ip assigned to it, the only error you might get is something about the entry already being there when libvirtd started.  Can you check that?  Also, can you attach a sosreport, or at least your /var/log/messages file so we can see if the interface throws any errors before you expect it?

This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 2 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.

Hi, thanks for the extra questions, I'll try to find that out. I don't really reboot this machine often and I see this only after a fresh boot. I'll try to get some answers to Neil and Michele's questions.

I finally finally found out what causes this problem.

It turns out it has nothing to do with virtualization at all. It's an issue with zebra (part of quagga). In the systemd unit file for zebra, it says:

Before=network.target

The big problem with this is, that starting zebra before the interface is up, causes the interface to be brought up in the wrong way, namely with the network address (in this case 192.168.4.0) as primary ip address.
As soon as the network officially gets started, the real ip address is also added, but as secondary. With ifconfig you can't see this but with "ip addr" you actually can.

If zebra gets started after the interface is already properly brought up, it doesn't add the wrong ip address. This is why I couldn't reproduce this bug after the machine had already been booted.

The fix is easy, just change the zebra unit file to:

After=network.target

Now, it all works just fine.

I am not sure that ordering quagga after network.target is what we want in general. quagga should be considered as a service which implements networking, thus it should pull the network.target in and order itself before it.

Well, this is the workaround that got it fixed quickly for me. I understand that in general it is better to solve the actual problem instead. Do you have enough information to track the underlying bug down or should I provide more?

I am at Flock right now, so I'll look into this probably sometime next week. I may or may not have further questions :)

Okay, you will probably want to know what's in my /etc/quagga/zebra.conf, so here it is. My /etc/sysconfig/quagga file is unmodified.


hostname testbox

password xxx
enable password xxx
service password-encryption

debug zebra events
debug zebra kernel

interface lo
          description loopback

interface eth0
          description Office 192.168.46.0/24
          ip address 192.168.46.0/24

interface virbr0
          description testbox intern 192.168.4.0/24
          ip address 192.168.4.0/24

log file /var/log/quagga/zebra.log

Sorry for not getting back to you earlier. Given configuration in comment #22 I don't consider described problem as a bug in quagga. Like I said earlier quagga should be ordered before network target as it is a service implementing network configuration.

If you don't like that quagga is configuring an ip address on the particular interface well then don't tell it to do so.