Description of problem: If we change non-VDSM network devices with VDSM and then call `vdsm-tool restore nets`, non-VDSM devices are not restored. Unified restoration first remove everything in the running config and then tries to regenerate everything that's in the persisted config, problem is, that original devices are not persisted. Version-Release number of selected component (if applicable): Tested on VDSM 4.16. How reproducible: Always Steps to Reproduce: 1. vdsClient -s 0 setupNetworks "networks={ovirtmgmt:{bonding:bond0,bridged:true,ipaddr:${IP},netmask:${NETMASK},gateway:${GATEWAY}}}" "bondings={bond0:{nics:${ETH_NIC_NAME}}}" 2. vdsm-tool restore-nets Actual results: VDSM devices are removed, ifcfg-eth0 is removed, but not restored. Expected results: We want ifcfg-eth0 in the original state.
As a workaround for pre-defined bonding ifcfg files, look at https://bugzilla.redhat.com/show_bug.cgi?id=1154399#c3
In order to not make this more difficult for customers and GSS than it has to be, let's skip the linking to another bug and be explicit here. To my understanding, it's best to ask vdsm to own the existing bonds prior to upgrading to 3.5. This can be done by running: "persist /etc/sysconfig/network-scripts/ifcfg-<bond-name>" Dan, please corroborate, and also does one need to specify a path for the persist script? If this was not done prior to upgrade, then bonds will have to be re-constructed post upgrade. If there management still has connectivity to the host, that can be performed via the GUI; otherwise (say if the management network was on such a bond) physical access to the host is needed, where the bonds can be created using vdsm command line: "vdsClient -s 0 setupNetworks bondings='{<bond-name>:{nics:<nic1-name>+<nic2-name>+...}" Dan, what would be the following command to also configure the management network on the newly-created bond?
This bug may not be specific to bond interfaces. I just ran into the same thing last night with a plain, ordinary, old-fashioned eth2 interface. See https://access.redhat.com/discussions/1350873 for details. After an upgrade to 3.5 from 3.4, on each host, my eth0 interface connected to the rhevm network works as expected. But no automation I can find will make interface eth2 come alive. Interface eth2 is connected to a home-made network named storage. I have to to ifup eth2 by hand on each host to make it live. I tried setting up eth2 again in the RHEVM GUI - no luck. I noticed ifcfg-eth2 on the hosts have "ONBOOT=no". I tried editing, then "persist ifcfg-eth2", and then a reboot. Still no luck. And ifcfg-eth2 went back to the way it was, even though I persisted it. - Greg Scott
I was experimenting with this problem and I came to situation, where I had 2 bond interfaces with 8 VLANs attached to them, and RHEV bridges attached to them running (typed in), persisted. On every reboot all of those interfaces were changed "ONBOOT=no" no matter if I changes were persisted, saved via RHEV-M network configuration GUI. I saw those interfaces starting up during boot, and after login I got no network connectivity and all of them reconfigured as described above.
Vdsm intentionally sets ONBOOT=no in order to disable ifcfg-based network persistence; vdsm is expected to start the interface a bit later, as vdsmd starts. Does it happen, Greg? If not, could you supply your /var/lib/vdsm, and /var/log/vdsm/*log during the service startup process? Either way, your report does not seem to be much related to this bug: unlike this bug, you do not refer to pre-vdsm ifcfg file, are you?
In my case, interface eth2 does not start up until I do "ifup eth2" by hand. Both hosts in my RHEV 3.5 environment act this way. I have a 16 MB monster vdsm.log and I'm looking for a place here in bugzilla to upload it. - Greg
Created attachment 992928 [details] vdsm.log from my RHEV 3.5 upgrade last night
Aw nuts. Forget about that vdsm.log I just uploaded. It rolled over and doesn't have anything from last night. Hang on while I go find the right one. - Greg
Created attachment 992933 [details] vdsm.log archive dated midnight. This should have the eth2 issues GMT is 5 hours later than Central time. Or 6 hours this time of year? Look for entries around 9 PM Central time last night.
Created attachment 992934 [details] supervdsm.log in case it's useful. And I obsoleted the original vdsm.log.
Created attachment 992935 [details] connectivity.log with some info from last night.
Created attachment 992936 [details] mom.log from last night
Created attachment 992937 [details] and upgrade.log showing the 3.5 upgrade I hope.
Created attachment 992939 [details] The whole /var/lib/vdsm tree tar -cvzf /tmp/vdsm.tgz vdsm
Greg, this is the wrong place to discuss your bug, as it has nothing to do with ifcfg files that has been created outside vdsm. Please open a new bug about your issue. In particular, please include the output of `vdsClient -s 0 getVdsCaps` after reboot, as supervdsm.log reports that eth2 has been ifup'ed when vdsmd started.
Why do you assume I created an ifcfg file outside VDSM? Until things broke and my RHEV-H host was dead in the water, I did everything via the RHEV-M GUI. After....***AFTER*** I found host rhevb dead, I dug into the problem and found interface eth2 had no IP Address. Then I looked at ifcfg-eth2 and noticed it said ONBOOT=no. I change it by hand to ONBOOT=yes and persisted it. Then rebooted host rhevb and found it want back to the way it was. Then I looked at host rheva - which was not yet updated to rhevh-6.6 and found its ifcfg-eth2 was also set to ONBOOT=no. At any rate, I did not mess with any config files by hand until ***AFTER*** running into this problem. If supervdsm.log says eth2 was ifup'ed with vdsm started then either (1) supervdsm.log is not telling the truth, or (2) something else during boot ifdown'ed it. If I boot either of my hosts, the only way I will be able to get it back alive inside RHEV-M is to ifup eth2 by hand. - Greg
Created attachment 993061 [details] vdsClient -s 0 getVdsCaps on host rheva
Created attachment 993063 [details] vdsClient -s 0 getVdsCaps on host rhevb
3.5.1 is already full with bugs (over 80), and since none of these bugs were added as urgent for 3.5.1 release in the tracker bug, moving to 3.5.2
(In reply to Greg Scott from comment #16) > Why do you assume I created an ifcfg file outside VDSM? Greg, I don't. But this bug was explicitly opened to address the problem of vdsm removing manually-written ifcfg files. You use this bug to discuss another problem, which confused me, and is certain to confuse future readers of this bug. Hence I asked you to discuss this bug elsewhere.
OK, now I get it. Sorry for the confusion. I found this bug report through some community posts and it looked like the same problem. Do you still want me to open another bugzilla? - Greg
The backported patch is ready https://gerrit.ovirt.org/#/c/38240/; I'm waiting for qa_ack to clone.
Dan, Petr, please help. 1. I am trying to understand the relation of this current bug to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1197668 They both seem to talk about same problem, but one is closed errata and one is open. 2. If I understand correctly, this bug is related to this solution: https://access.redhat.com/solutions/1346443 Please confirm. 3. Comment#2 on this bug by Lior is not clear to me. Why do we need to run persist commands on a regular RHEL host. Or he is referring to RHEV-H hosts, that can be affected by this problem as well and we need to update our solution mentioned on #2? Bottom line what I am trying to understand is whether we fixed the correct bug and when it is making to the customers.
Verified on - 3.6.0-0.0.master.20150412172306.git55ba764.el6 with vdsm-4.17.0-632.git19a83a2.el7.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0362.html