Bug 1405838 - otopi updated iproute package during ovirt-host-mgmt
Summary: otopi updated iproute package during ovirt-host-mgmt
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: otopi
Classification: oVirt
Component: Plugins.packagers
Version: 1.5.2
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ovirt-4.1.2
: 1.6.2
Assignee: Yedidyah Bar David
QA Contact: Petr Matyáš
URL:
Whiteboard:
Depends On:
Blocks: 1444420
TreeView+ depends on / blocked
 
Reported: 2016-12-18 20:18 UTC by Derek Atkins
Modified: 2017-09-17 07:57 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: otopi has a plugin that always runs that checks basic dns/hostname status. It needs the command 'ip' from the package 'iproute'. It used to install or update this package in some early stage. Consequence: otopi is used (also) in tools from which some users do not expect to change the state of the machine as radically as installing or updating a package. Examples are ovirt-host-deploy in the mode of 'checking for updates' for oVirt's Update Manager, and 'engine-setup' when ran and then canceled by replying 'Cancel' to the final prompt before setup/upgrade. Fix: otopi was changed to not install or upgrade iproute. If the command 'ip' is not found, it will emit a warning and continue. Result: otopi no longer installs or updates iproute for its own use. Tools based on otopi that require iproute for more-critical stuff still can and do install or update it as needed.
Clone Of:
Environment:
Last Closed: 2017-05-23 08:18:00 UTC
oVirt Team: Integration
Embargoed:
ylavi: ovirt-4.1+
lsvaty: testing_ack+


Attachments (Terms of Use)
Engine Log showing the upgrade (23.25 KB, text/plain)
2016-12-18 20:18 UTC, Derek Atkins
no flags Details
ovirt-host-mgmt log (153.66 KB, text/plain)
2016-12-19 12:45 UTC, Derek Atkins
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 74386 0 otopi-1.6 MERGED plugins: hostname.py: Do not install/update iproute 2017-04-05 14:15:47 UTC
oVirt gerrit 75238 0 otopi-1.6 MERGED plugins: hostname.py: Do not install/update iproute 2017-04-06 12:54:17 UTC

Description Derek Atkins 2016-12-18 20:18:21 UTC
Created attachment 1233251 [details]
Engine Log showing the upgrade

Description of problem:

I woke up on Dec 13th to see the log of my ovirt host (Running CentOS 7.2) had updated the iproute package.  The update happened at 23:17; I was happily in bed at the time.   I have no other admins of my system and didn't initiate any updates, nor do I have any cron jobs set up to auto-update.

I'm trying to determine why this package was updated?  Specifically:

tail -5 /var/log/yum.log
Nov 25 10:46:11 Updated: 10:qemu-kvm-tools-ev-2.3.0-31.0.el7_2.21.1.x86_64
Nov 25 10:46:12 Updated: ovirt-hosted-engine-setup-2.0.3-1.el7.centos.noarch
Nov 25 10:46:12 Updated: ovirt-iso-uploader-4.0.2-1.el7.centos.noarch
Nov 25 10:46:12 Updated: ovirt-release40-4.0.5-2.noarch
Dec 12 23:17:17 Updated: iproute-3.10.0-74.el7.x86_64

Nov 25 was when I updated from 4.0.4 to 4.0.5.

Version-Release number of selected component (if applicable):

I'm running Ovirt-4.0.5.5 in hosted-engine mode on a single host machine.  Both host and hosted-engine VM are running CentOS 7.2.  The host is running:

vdsm-4.18.15.3-1.el7.centos.x86_64

The engine is running:

ovirt-engine-4.0.5.5-1.el7.centos.noarch


How reproducible:

No clue.  It only happened once.  I haven't tried downgrading iproute to see if it gets upgraded again.


Steps to Reproduce:
1. Unknown.  I had EL7.2 installed
2. EL7.3 got pushed to the mirror
3. Engine auto-updated the iproute package

Actual results:

iproute package was auto-updated on the host

Expected results:

No packages should get updated without admin approval.

Additional info:

I initially reported to the ovirt users list.  By the time I got a reply that I should file a bug report, the vdsm logs had rolled over (it only keeps 100 vdsm log files, which rolled over last night).  SO..... I have no VDSM log to upload.  Sorry.

I've got the engine log, which shows the upgrade!  I can upload the supervdsm log if you care, but I didn't see anything in there.

Thanks.

Comment 1 Martin Perina 2016-12-19 08:54:23 UTC
I looked at logs around the time you have mentioned and the host-deploy process you have mentioned is just checking for available upgrades:

2016-12-12 23:16:46,941 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (DefaultQuartzScheduler9) [4eb758cf] Connected to host OVIRT-0.IHTFP.ORG with SSH key fingerprint: SHA256:4PDfnXd9/9WD3YNPrtiTwKgKGjcvKHqd/v5c8qlDrj0                                                                                                                                             
2016-12-12 23:16:47,020 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (DefaultQuartzScheduler9) [4eb758cf] Installation of OVIRT-0.IHTFP.ORG. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x &&  "${MYTMP}"/ovirt-host-mgmt DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar
2016-12-12 23:16:47,020 INFO  [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (DefaultQuartzScheduler9) [4eb758cf] Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' refresh                      
2016-12-12 23:16:47,055 INFO  [org.ovirt.engine.core.uutils.ssh.SSHDialog] (DefaultQuartzScheduler9) [4eb758cf] SSH execute 'root.ORG' 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x &&  "${MYTMP}"/ovirt-host-mgmt DIALOG/dialect=str:machine DIALOG/customization=bool:True'
2016-12-12 23:16:47,787 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Initializing                                                                                 
2016-12-12 23:16:47,794 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Environment setup                                                                            
2016-12-12 23:16:47,805 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Environment packages setup                                                                   
2016-12-12 23:17:14,760 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Status: Downloading Packages                                                                    
2016-12-12 23:17:16,580 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Download/Verify: iproute-3.10.0-74.el7.x86_64                                                   
2016-12-12 23:17:16,588 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Status: Check Package Signatures                                                                
2016-12-12 23:17:16,589 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Status: Running Test Transaction                                                                
2016-12-12 23:17:16,714 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Status: Running Transaction                                                                     
2016-12-12 23:17:17,000 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum update: 1/2: iproute-3.10.0-74.el7.x86_64                                                       
2016-12-12 23:17:17,812 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum updated: 2/2: iproute                                                                           
2016-12-12 23:17:18,203 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Verify: 1/2: iproute.x86_64 0:3.10.0-74.el7 - u                                                 
2016-12-12 23:17:18,302 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Yum Verify: 2/2: iproute.x86_64 0:3.10.0-54.el7_2.1 - ud                                            
2016-12-12 23:17:18,609 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Programs detection                                                                           
2016-12-12 23:17:18,640 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Environment customization                                                                    
2016-12-12 23:17:18,686 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Logs at host located at: '/tmp/ovirt-host-mgmt-20161212231647-llmd01.log'                           
2016-12-12 23:17:18,695 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Setup validation                                                                             
2016-12-12 23:17:18,713 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Transaction setup                                                                            
2016-12-12 23:17:18,757 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Misc configuration                                                                           
2016-12-12 23:17:18,758 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Package installation                                                                         
2016-12-12 23:17:23,834 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Misc configuration                                                                           
2016-12-12 23:17:23,837 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Transaction commit                                                                           
2016-12-12 23:17:23,883 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Closing up                                                                                   
2016-12-12 23:17:23,885 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Pre-termination                                                                              
2016-12-12 23:17:23,893 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployPackagesUnit] (VdsDeploy) [4eb758cf] Available packages for update: device-mapper-1.02.135-1.el7_3.1,device-mapper-event-1.02.135-1.el7_3.1,device-mapper-event-libs-1.02.135-1.el7_3.1,device-mapper-libs-1.02.135-1.el7_3.1,device-mapper-persistent-data-0.6.3-1.el7,libnl3-3.2.28-2.el7,libnl3-cli-3.2.28-2.el7,libvirt-client-2.0.0-10.el7_3.2,libvirt-daemon-2.0.0-10.el7_3.2,libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.2,libvirt-daemon-driver-interface-2.0.0-10.el7_3.2,libvirt-daemon-driver-network-2.0.0-10.el7_3.2,libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.2,libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.2,libvirt-daemon-driver-qemu-2.0.0-10.el7_3.2,libvirt-daemon-driver-secret-2.0.0-10.el7_3.2,libvirt-daemon-driver-storage-2.0.0-10.el7_3.2,libvirt-daemon-kvm-2.0.0-10.el7_3.2,libvirt-lock-sanlock-2.0.0-10.el7_3.2,libvirt-python-2.0.0-2.el7,lvm2-2.02.166-1.el7_3.1,lvm2-libs-2.02.166-1.el7_3.1,sanlock-3.4.0-1.el7,sanlock-lib-3.4.0-1.el7,sanlock-python-3.4.0-1.el7
2016-12-12 23:17:23,904 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-host-mgmt-20161212231723-OVIRT-0.IHTFP.ORG-4eb758cf.log'                                                                                                                                         
2016-12-12 23:17:23,977 INFO  [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [4eb758cf] Stage: Termination                                                                                  
2016-12-12 23:17:24,085 INFO  [org.ovirt.engine.core.bll.host.HostUpgradeManager] (DefaultQuartzScheduler9) [4eb758cf] There are available packages (device-mapper-1.02.135-1.el7_3.1, device-mapper-event-1.02.135-1.el7_3.1, device-mapper-event-libs-1.02.135-1.el7_3.1, device-mapper-libs-1.02.135-1.el7_3.1, device-mapper-persistent-data-0.6.3-1.el7, libnl3-3.2.28-2.el7, libnl3-cli-3.2.28-2.el7, libvirt-client-2.0.0-10.el7_3.2, libvirt-daemon-2.0.0-10.el7_3.2, libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.2, libvirt-daemon-driver-interface-2.0.0-10.el7_3.2, libvirt-daemon-driver-network-2.0.0-10.el7_3.2, libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.2, libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.2, libvirt-daemon-driver-qemu-2.0.0-10.el7_3.2, libvirt-daemon-driver-secret-2.0.0-10.el7_3.2, libvirt-daemon-driver-storage-2.0.0-10.el7_3.2, libvirt-daemon-kvm-2.0.0-10.el7_3.2, libvirt-lock-sanlock-2.0.0-10.el7_3.2, libvirt-python-2.0.0-2.el7, lvm2-2.02.166-1.el7_3.1, lvm2-libs-2.02.166-1.el7_3.1, sanlock-3.4.0-1.el7, sanlock-lib-3.4.0-1.el7, sanlock-python-3.4.0-1.el7) for host 'ovirt-0'


But you are right, during the check for upgrades we have updated iproute package. More details about this action can be found at '/var/log/ovirt-engine/host-deploy/ovirt-host-mgmt-20161212231723-OVIRT-0.IHTFP.ORG-4eb758cf.log' on engine machine, could you please attach it to the bug?

Comment 2 Derek Atkins 2016-12-19 12:45:59 UTC
Created attachment 1233375 [details]
ovirt-host-mgmt log

Hi,
Here is the log file you requested
Thanks.

Comment 3 Martin Perina 2016-12-19 13:31:44 UTC
Hmm, looks like some internal part of ovirt-host-deploy/otopi upgraded iproute package, because this upgrade is not performed/required by "check for upgrade" flow. Sandro, could you please take a look?

Comment 4 Sandro Bonazzola 2016-12-19 16:46:51 UTC
(In reply to Martin Perina from comment #3)
> Hmm, looks like some internal part of ovirt-host-deploy/otopi upgraded
> iproute package, because this upgrade is not performed/required by "check
> for upgrade" flow. Sandro, could you please take a look?

That's weird. Host deploy should install it but not upgrade it. 
ovirt-host-deploy has iproute as pre-requirement package.
otopi takes care of pre-installing all the needed packages in order to run itself and then it restart itself.
It's like that since:

e9c98c67 (Alon Bar-Lev 2013-07-30 18:02:18 +0300 601)         self.packager.install(packages=('iproute',))
in ./src/plugins/ovirt-host-deploy/vdsm/bridge.py

packager.install shouldn't update since for that we have installUpdate.
Maybe something changed in yum internals and install became equivalent to update.

Moving to otopi for further investigation.

Comment 5 Yedidyah Bar David 2017-02-27 07:58:00 UTC
Changing the summary line for now.

If indeed it's a change in yum, we might do one of:
1. Open a bug on yum
2. Add an option to otopi to not do this and make ovirt-host-mgmt enable this option.

Didn't check yet. I know for sure that in the command line, 'yum install package' will update it if relevant, and this is also documented in its man page:

       install
              Is  used  to install the latest version of a package or group of
              packages while ensuring that  all  dependencies  are  satisfied.

Comment 6 Yedidyah Bar David 2017-03-09 10:24:38 UTC
It might have been a change in yum, not sure. Briefly skimmed through its git history and could not find a clue.

ovirt-host-deploy installs it (as Sandro mentioned), or updates (as mentioned, might be a bug/change in yum), but only in deploy ((re)install) mode. Not in ovirt-host-mgmt (check for updates). So the problem/bug is likely not there.

otopi itself also installs iproute, in src/plugins/otopi/network/hostname.py . It does that unconditionally, so every tool that uses otopi will get to install or update iproute (unless it uses an offlinepackager).

This plugin (hostname.py) does not set any environment key, so apart from general sanity testing of the machine (and debug logging of some relevant information) I can't see why it's needed. In principle we could have removed it. For current bug, I suggest to patch it to not install (or update) iproute, but still do its stuff if it manages to find the command 'ip'. Sandro?

Comment 7 Sandro Bonazzola 2017-03-31 06:26:09 UTC
(In reply to Yedidyah Bar David from comment #6)
> For current bug, I suggest to patch it to not install (or update)
> iproute, but still do its stuff if it manages to find the command 'ip'.
> Sandro?

Make sense to me.

Comment 8 Derek Atkins 2017-04-03 15:43:49 UTC
(In reply to Sandro Bonazzola from comment #7)
> (In reply to Yedidyah Bar David from comment #6)
> > For current bug, I suggest to patch it to not install (or update)
> > iproute, but still do its stuff if it manages to find the command 'ip'.
> > Sandro?
> 
> Make sense to me.

Or possibly do its stuff if it finds "ip", and if not then install iproute?

Comment 9 Yedidyah Bar David 2017-05-03 09:36:27 UTC
Note to QE - reproduction/verification flows:

I.

1. Install a system with oVirt repos and an older iproute package (e.g. from el7.2)
2. Install otopi
3. Run otopi

With unfixed builds, otopi will update iproute.
With fixed builds it will not.

II.

1. Install a system with oVirt repos and do not install iproute.
2. Install otopi
3. Run otopi

With unfixed builds, otopi will install iproute.
With fixed builds, it will emit a warning and continue, and not install iproute.

Comment 10 Yedidyah Bar David 2017-05-03 09:39:15 UTC
(In reply to Derek Atkins from comment #8)
> (In reply to Sandro Bonazzola from comment #7)
> > (In reply to Yedidyah Bar David from comment #6)
> > > For current bug, I suggest to patch it to not install (or update)
> > > iproute, but still do its stuff if it manages to find the command 'ip'.
> > > Sandro?
> > 
> > Make sense to me.
> 
> Or possibly do its stuff if it finds "ip", and if not then install iproute?

Of course we can do that, but I do not see a good reason to, and people like you (that care for their systems' health) will still wonder why it did, at least in some flows. But thanks for the suggestion :-)

Comment 11 Petr Matyáš 2017-05-15 14:44:00 UTC
Verified on 4.1.2-4

Comment 12 Yedidyah Bar David 2017-09-17 07:57:33 UTC
Sorry, wrong bug.


Note You need to log in before you can comment on or make changes to this bug.