Bug 1576352 - rhvm-4.2 reports "no updates found" although there is available updates
Summary: rhvm-4.2 reports "no updates found" although there is available updates
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-4.2.3-1
: ---
Assignee: Ondra Machacek
QA Contact: Pavol Brilla
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-09 09:16 UTC by Huijuan Zhao
Modified: 2022-07-09 13:23 UTC (History)
23 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 10:02:26 UTC
oVirt Team: Infra
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
All logs from rhvh (10.06 MB, application/x-gzip)
2018-05-09 09:16 UTC, Huijuan Zhao
no flags Details
Log from rhvm (33.82 KB, text/plain)
2018-05-09 09:18 UTC, Huijuan Zhao
no flags Details
Screenshot of rhvm can not detect available updates (207.14 KB, image/png)
2018-05-09 09:19 UTC, Huijuan Zhao
no flags Details
engine.log (10.64 MB, text/plain)
2018-05-09 14:46 UTC, Ying Cui
no flags Details
engine.log (8.00 KB, text/plain)
2018-05-09 14:54 UTC, Ying Cui
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github ansible ansible issues 39978 0 None closed yum: Don't list obsoleted packages 2020-09-14 12:52:46 UTC
Red Hat Issue Tracker RHV-47440 0 None None None 2022-07-09 13:23:04 UTC
Red Hat Product Errata RHSA-2018:2071 0 None None None 2018-06-27 10:03:53 UTC
oVirt gerrit 91156 0 master MERGED ansible: Hotfix for Ansible yum module 2020-09-14 12:52:46 UTC
oVirt gerrit 91163 0 ovirt-engine-4.2 MERGED ansible: Hotfix for Ansible yum module 2020-09-14 12:52:45 UTC
oVirt gerrit 91168 0 ovirt-engine-4.2.3.z MERGED ansible: Hotfix for Ansible yum module 2020-09-14 12:52:46 UTC

Description Huijuan Zhao 2018-05-09 09:16:59 UTC
Created attachment 1433648 [details]
All logs from rhvh

Description of problem:
Add rhvh-4.2.2-20180410.0 to rhvm-4.2(4.2.3.5-0.1.el7), setup local repos in rhvh which target to rhvh-4.2.3-20180508.0, click "Check for Upgrade", rhvm reported "no updates found".
It should report there is upgrade available.

At the same time, run "# yum update" from rhvh side can detect new build rhvh-4.2.3-20180508.0, and upgrade to new build rhvh-4.2.3 successful.
 

Version-Release number of selected component (if applicable):
From:rhvh-4.2.2.1-0.20180410.0
To:  rhvh-4.2.3.0-0.20180508.0

rhvm: Software Version:4.2.3.5-0.1.el7

How reproducible:
100%


Steps to Reproduce:
1. Install redhat-virtualization-host-4.2-20180410.0
2. Add it to rhvm-4.2.3
3. Setup local repos in rhvh, which target to redhat-virtualization-host-4.2-20180508.0
4. In rhvm side, choose the rhvh and click "Check for Upgrade"
5. Run "# yum update" from rhvh side

Actual results:
1. After step 4, rhvm reports "no updates found", so can not upgrade rhvh from rhvm side.
2. After step 5, can detect new build redhat-virtualization-host-4.2-20180508.0, and upgrade rhvh successful.

Expected results:
After step 4, rhvm should detect available upgrade, and can upgrade rhvh from rhvm successful.

Additional info:

Comment 1 Huijuan Zhao 2018-05-09 09:18:20 UTC
Created attachment 1433649 [details]
Log from rhvm

Comment 2 Huijuan Zhao 2018-05-09 09:19:23 UTC
Created attachment 1433650 [details]
Screenshot of rhvm can not detect available updates

Comment 4 Ryan Barry 2018-05-09 11:08:00 UTC
Can you please add engine.log?

Comment 5 Ondra Machacek 2018-05-09 13:37:37 UTC
The yum proceed just fine in your case and I can't reproduce as welll:

May  9 16:28:16 dell-per730-34 python: ansible-yum Invoked with allow_downgrade=False name=['ovirt-host'] validate_certs=True list=None disable_gpg_check=False conf_file=None install_repoquery=True state=latest disablerepo=None update_cache=True disable_plugin=[] enablerepo=None exclude=None security=False update_only=False enable_plugin=[] installroot=/ skip_broken=False
May  9 16:28:21 dell-per730-34 python: ansible-yum Invoked with allow_downgrade=False name=['*'] validate_certs=True list=None disable_gpg_check=False conf_file=None install_repoquery=True state=latest disablerepo=None update_cache=True disable_plugin=[] enablerepo=None exclude=None security=False update_only=False enable_plugin=[] installroot=/ skip_broken=False

Can you please send the whole output of running 'yum update' from step 5? Also please attach engine.log.

Comment 6 Ryan Barry 2018-05-09 14:05:42 UTC
Just FYI, Ondra, this is specifically about updating RHVH.

I'm also trying to set up a reproducer, but nothing in RHVH's update process has changed since 4.2 beta 1...

Comment 8 Ying Cui 2018-05-09 14:46:58 UTC
Created attachment 1433894 [details]
engine.log

Comment 9 Ying Cui 2018-05-09 14:54:42 UTC
Created attachment 1433899 [details]
engine.log

Comment 10 Ying Cui 2018-05-09 15:07:03 UTC
On the engine side, after clicking on the "Check for Upgrade", can NOT detect available upgrade on engine(4.2.3.5-0.1.el7). 

On Host, it can check the new version by yum update by local repo.

# yum update 
Loaded plugins: imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Repo rhel-7-server-rhvh-4-source-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-source-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Repo rhel-7-server-rhvh-4-beta-debug-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-beta-debug-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Repo rhel-7-server-rhvh-4-debug-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-debug-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Repo rhel-7-server-rhvh-4-beta-source-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-beta-source-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Repo rhel-7-server-rhvh-4-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Repo rhel-7-server-rhvh-4-beta-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877.pem
Repo rhel-7-server-rhvh-4-beta-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/5858345830546115877-key.pem
Resolving Dependencies
--> Running transaction check
---> Package redhat-virtualization-host-image-update.noarch 0:4.2-20180507.0.el7_5 will be obsoleting
---> Package redhat-virtualization-host-image-update-placeholder.noarch 0:4.2-2.1.el7 will be obsoleted
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package                                                           Arch                             Version                                         Repository                                                Size
===================================================================================================================================================================================================================
Installing:
 redhat-virtualization-host-image-update                           noarch                           4.2-20180507.0.el7_5                            rhel-7-server-rhvh-4-beta-rpms                           612 M
     replacing  redhat-virtualization-host-image-update-placeholder.noarch 4.2-2.1.el7

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 612 M
Is this ok [y/d/N]:

Comment 12 Huijuan Zhao 2018-05-10 00:03:09 UTC
(In reply to Ondra Machacek from comment #5)
 
> Can you please send the whole output of running 'yum update' from step 5?
> Also please attach engine.log.

Please see comment 9 and comment 10.
Thanks Ying to provide the info.

Comment 13 Ondra Machacek 2018-05-10 11:13:00 UTC
I still can't reproduce, can you please try to execute following commands on the engine:

$ echo "ip_of_host_that_doesnt_work" > /tmp/inv

$ ANSIBLE_STDOUT_CALLBACK=hostupgradeplugin /usr/bin/ansible-playbook --ssh-common-args='-F /var/lib/ovirt-engine/.ssh/config' --check --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/inv /usr/share/ovirt-engine/playbooks/ovirt-host-upgrade.yml

$ /usr/bin/ansible-playbook -vvvv --ssh-common-args='-F /var/lib/ovirt-engine/.ssh/config' --check --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/inv /usr/share/ovirt-engine/playbooks/ovirt-host-upgrade.yml

And share the output.

Comment 14 Ryan Barry 2018-05-10 12:00:44 UTC
[root@engine ~]# ANSIBLE_STDOUT_CALLBACK=hostupgradeplugin /usr/bin/ansible-playbook --ssh-common-args='-F /var/lib/ovirt-engine/.ssh/config' --check --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/inv /usr/share/o
virt-engine/playbooks/ovirt-host-upgrade.yml
[]
[root@engine ~]# /usr/bin/ansible-playbook -vvvv --ssh-common-args='-F /var/lib/ovirt-engine/.ssh/config' --check --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/inv /usr/share/ovirt-engine/playbooks/ovir[205/3614]
rade.yml
ansible-playbook 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /tmp/inv inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: ovirt-host-upgrade.yml *************************************************************************************************************************************************************************************************************
1 plays in /usr/share/ovirt-engine/playbooks/ovirt-host-upgrade.yml

PLAY [all] ***********************************************************************************************************************************************************************************************************************************
META: ran handlers

TASK [ovirt-host-upgrade : Install ovirt-host package if it isn't installed] *****************************************************************************************************************************************************************
task path: /usr/share/ovirt-engine/playbooks/roles/ovirt-host-upgrade/tasks/main.yml:8
Using module file /usr/lib/python2.7/site-packages/ansible/modules/packaging/os/yum.py
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<192.168.122.64> (0, '/root\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/root/.ansible
/cp/0948316aa6" does not exist\r\ndebug2: resolving "192.168.122.64" port 22\r\ndebug2: ssh_connect_direct: needpriv 0\r\ndebug1: Connecting to 192.168.122.64 [192.168.122.64] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 cl
earing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 10000 ms remain after connect\r\ndebug1: permanently_set_uid: 0/0\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /etc/pki/ovirt-engin
e/keys/engine_id_rsa type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /etc/pki/ovirt-engine/keys/engine_id_rsa-cert type -1\r\ndebug1: Enabling compatibility mode for protocol 2.0\r\ndebug1: Local ver
sion string SSH-2.0-OpenSSH_7.4\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_7.4\r\ndebug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to 192.1
68.122.64:22 as \'root\'\r\ndebug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from 192.168.122.64\r\nde
bug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521\r\ndebug3:
 send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.
org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-
group14-sha1,diffie-hellman-group1-sha1,ext-info-c\r\ndebug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-n
istp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01,ssh-rsa-cert-v01,ssh-dss-cert-v01,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss\r\ndebug2: ciphers ctos: chacha20-poly1305,aes128-ctr
,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,ae
s128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2
-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-2
56,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: zlib,zlib,none\r\ndebug2: compression stoc: zlib,zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: 
reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-gro
up16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1\r\ndebug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2
-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: ciph
ers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: MACs ctos: umac-64-etm,umac-1
28-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm,umac-128
-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,zlib\r\nde
bug2: compression stoc: none,zlib\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: 
ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: chacha20-poly1305 MAC: <implicit> compression: zlib\r\ndebug1: kex: client->server cipher: chacha20-poly1305 MAC: <implicit> compression: zlib@
openssh.com\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug
1: Server host key: ecdsa-sha2-nistp256 SHA256:bRKndLfrwtxInWkJmp9aBLGsBsFUflZ6IAJdUSl4gaA\r\ndebug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:1
\r\ndebug3: load_hostkeys: loaded 1 keys from 192.168.122.64\r\ndebug1: Host \'192.168.122.64\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /root/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newke
ys: mode 1\r\ndebug1: rekey after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: re
key after 134217728 blocks\r\ndebug2: key: /etc/pki/ovirt-engine/keys/engine_id_rsa ((nil)), explicit\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: se
rver-sig-algs=<rsa-sha2-256,rsa-sha2-512>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: 
Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,
publickey\r\ndebug3: authmethod_lookup gssapi-with-mic\r\ndebug3: remaining preferred: gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_lookup gssapi-keyex\r\ndebug3: remaining preferred: hostbased,publickey\r\ndebug3: authmethod_lo
okup publickey\r\ndebug3: remaining preferred: ,publickey\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /etc/pki/ovirt-engine/keys/engine_id_rsa\r\ndebug3: sign_
and_send_pubkey: RSA SHA256:KybMVBphLSeGI3+T4Jm9znrNt9yBvj01ZNJUlTMKK0U\r\ndebug3: send packet: type 50\r\ndebug2: we sent a publickey packet, wait for reply\r\ndebug3: receive packet: type 52\r\ndebug1: Enabling compression at level 6.\r
\ndebug1: Authentication succeeded (publickey).\r\nAuthenticated to 192.168.122.64 ([192.168.122.64]:22).\r\ndebug1: setting up multiplex master socket\r\ndebug3: muxserver_listen: temporary control path /root/.ansible/cp/0948316aa6.TnoWy
yWExp4Pfeh2\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug1: channel 0: new [/root/.ansible/cp/0948316aa6]\r\ndebug3: muxserver_listen: mux listener channel 0 fd 4\r\ndebug2: fd 3 s
etting TCP_NODELAY\r\ndebug3: ssh_packet_set_tos: set IP_TOS 0x08\r\ndebug1: control_persist_detach: backgrounding master process\r\ndebug2: control_persist_detach: background process is 3356\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug1: 
forking to background\r\ndebug1: Entering interactive session.\r\ndebug1: pledge: id\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug1: multiplexing control connection\r\ndebug2: fd 5 setting O_NONBLOCK\r\nde
bug3: fd 5 is O_NONBLOCK\r\ndebug1: channel 1: new [mux-control]\r\ndebug3: channel_post_mux_listener: new mux channel 1 fd 5\r\ndebug3: mux_master_read_cb: channel 1: hello sent\r\ndebug2: set_control_persist_exit_time: cancel scheduled 
exit\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x00000001 len 4\r\ndebug2: process_mux_master_hello: channel 1 slave version 4\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwa
rdings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000004 len 4\r\ndebug2: process_mux_alive_check: channel 1: a
live check\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000002 len 81\r\ndebug2: process_mux_new_session: channel
 1: request tty 0, X 0, agent 0, subsys 0, term "screen-256color", cmd "/bin/sh -c \'echo ~ && sleep 0\'", env 0\r\ndebug3: process_mux_new_session: got fds stdin 6, stdout 7, stderr 8\r\ndebug2: fd 7 setting O_NONBLOCK\r\ndebug2: fd 8 se
tting O_NONBLOCK\r\ndebug1: channel 2: new [client-session]\r\ndebug2: process_mux_new_session: channel_new: 2 linked to control channel 1\r\ndebug2: channel 2: send open\r\ndebug3: send packet: type 90\r\ndebug3: receive packet: type 80\
r\ndebug1: client_input_global_request: rtype hostkeys-00 want_reply 0\r\ndebug3: receive packet: type 91\r\ndebug2: callback start\r\ndebug2: client_session2_setup: id 2\r\ndebug1: Sending command: /bin/sh -c \'echo ~ && slee
p 0\'\r\ndebug2: channel 2: request exec confirm 1\r\ndebug3: send packet: type 98\r\ndebug3: mux_session_confirm: sending success reply\r\ndebug2: callback done\r\ndebug2: channel 2: open confirm rwindow 0 rmax 32768\r\ndebug1: mux_clien
t_request_session: master session id: 2\r\ndebug2: channel 2: rcvd adjust 2097152\r\ndebug3: receive packet: type 99\r\ndebug2: channel_input_status_confirm: type 99 id 2\r\ndebug2: exec request accepted on channel 2\r\ndebug3: receive pa
cket: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype exit-status reply 0\r\ndebug3: mux_exit_message: channel 2: exit message, exitval 0\r\ndebug3: receive packet: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype
 eow reply 0\r\ndebug2: channel 2: rcvd eow\r\ndebug2: channel 2: close_read\r\ndebug2: channel 2: input open -> closed\r\ndebug3: receive packet: type 96\r\ndebug2: channel 2: rcvd eof\r\ndebug2: channel 2: output open -> dra
in\r\ndebug2: channel 2: obuf empty\r\ndebug2: channel 2: close_write\r\ndebug2: channel 2: output drain -> closed\r\ndebug3: receive packet: type 97\r\ndebug2: channel 2: rcvd close\r\ndebug3: channel 2: will not send data after close\r\
ndebug2: channel 2: send close\r\ndebug3: send packet: type 97\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: gc: notify user\r\ndebug3: mux_master_session_cleanup_cb: entering for channel 2\r\ndebug2: channel 1: rcvd close\r\ndebug2
: channel 1: output open -> drain\r\ndebug2: channel 1: close_read\r\ndebug2: channel 1: input open -> closed\r\ndebug2: channel 2: gc: user detached\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: garbage collecting\r\ndebug1: channe
l 2: free: client-session, nchannels 3\r\ndebug3: channel 2: status: The following connections are open:\r\n  #1 mux-control (t16 r-1 i3/0 o1/16 fd 5/5 cc -1)\r\n  #2 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)\r\n\r\ndebug2: channel 
1: obuf empty\r\ndebug2: channel 1: close_write\r\ndebug2: channel 1: output drain -> closed\r\ndebug2: channel 1: is dead (local)\r\ndebug2: channel 1: gc: notify user\r\ndebug3: mux_master_control_cleanup_cb: entering for channel 1\r\nd
ebug2: channel 1: gc: user detached\r\ndebug2: channel 1: is dead (local)\r\ndebug2: channel 1: garbage collecting\r\ndebug1: channel 1: free: mux-control, nchannels 2\r\ndebug3: channel 1: status: The following connections are open:\r\n 
 #1 mux-control (t16 r-1 i3/0 o3/0 fd 5/5 cc -1)\r\n\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\
n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root                                                                                                                                                                            [125/3615]
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansibl
e/tmp/ansible-tmp-1525953407.2-43785370886990 `" && echo ansible-tmp-1525953407.2-43785370886990="` echo /root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990 `" ) && sleep 0'"'"''
<192.168.122.64> (0, 'ansible-tmp-1525953407.2-43785370886990=/root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh
/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_req
uest_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id
: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<192.168.122.64> PUT /root/.ansible/tmp/ansible-local-334528DNix/tmppHKeoF TO /root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990/yum.py
<192.168.122.64> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-ke
yex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 '[192.168.122.64]'
<192.168.122.64> (0, 'sftp> put /root/.ansible/tmp/ansible-local-334528DNix/tmppHKeoF /root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990/yum.py\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration 
data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 r
emote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_req
uest_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename" revision 1\r\ndebug2: Server supports extension "statvfs" revision 2\r\ndebug2: Server supports ex
tension "fstatvfs" revision 2\r\ndebug2: Server supports extension "hardlink" revision 1\r\ndebug2: Server supports extension "fsync" revision 1\r\ndebug3: Sent message fd 5 T:16 I:1\r\ndebug3: SSH_FXP_
REALPATH . -> /root size 0\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-334528DNix/tmppHKeoF\r\ndebug3: Sent message fd 5 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or d
irectory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990/yum.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4
 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:32757\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_
FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 32757 bytes at 65536\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit stat
us from master 0\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-152
5953407.2-43785370886990/ /root/.ansible/tmp/ansible-tmp-1525953407.2-43785370886990/yum.py && sleep 0'"'"''
<192.168.122.64> (0, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mu
x_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_a
live: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit stat
us from master 0\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 -tt 192.168.122.64 '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansib
le-tmp-1525953407.2-43785370886990/yum.py && sleep 0'"'"''
<192.168.122.64> (0, '\r\n{"changed": false, "results": ["All packages providing ovirt-host are up to date"], "msg": "", "rc": 0, "invocation": {"module_args": {"allow_downgrade": false, "name": ["ovirt-host"], "validate_certs": true, "li
st": null, "disable_gpg_check": false, "conf_file": null, "install_repoquery": true, "state": "latest", "disablerepo": null, "update_cache": true, "disable_plugin": [], "enablerepo": null, "exclude": null, "security": false, "update_only"
: false, "enable_plugin": [], "installroot": "/", "skip_broken": false}}, "changes": {"updated": [], "installed": []}}\r\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh
/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_req
uest_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id
: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to 192.168.122.64 closed.\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1525
953407.2-43785370886990/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.122.64> (0, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mu
x_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_a
live: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit stat
us from master 0\r\n')
ok: [192.168.122.64] => {                                                                                                                                                                                                                     
    "changed": false,                                                                                                                                                                                                                         
    "changes": {                                                                                                                                                                                                                              
        "installed": [],                                                                                                                                                                                                                      
        "updated": []
    }, 
    "invocation": {
        "module_args": {
            "allow_downgrade": false, 
            "conf_file": null, 
            "disable_gpg_check": false, 
            "disable_plugin": [], 
            "disablerepo": null, 
            "enable_plugin": [], 
            "enablerepo": null, 
            "exclude": null, 
            "install_repoquery": true, 
            "installroot": "/", 
            "list": null, 
            "name": [
                "ovirt-host"
            ], 
            "security": false, 
            "skip_broken": false, 
            "state": "latest", 
            "update_cache": true, 
            "update_only": false, 
            "validate_certs": true
        }
    }, 
    "msg": "", 
    "rc": 0, 
    "results": [
        "All packages providing ovirt-host are up to date"
    ]
}
                                                                                                                                                                                                                                              
TASK [ovirt-host-upgrade : Update system] ****************************************************************************************************************************************************************************************************
task path: /usr/share/ovirt-engine/playbooks/roles/ovirt-host-upgrade/tasks/main.yml:16                                                                                                                                                       
Using module file /usr/lib/python2.7/site-packages/ansible/modules/packaging/os/yum.py
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<192.168.122.64> (0, '/root\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndeb
ug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_re
quest_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received ex
it status from master 0\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansibl
e/tmp/ansible-tmp-1525953412.96-243665835667843 `" && echo ansible-tmp-1525953412.96-243665835667843="` echo /root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843 `" ) && sleep 0'"'"''
<192.168.122.64> (0, 'ansible-tmp-1525953412.96-243665835667843=/root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/
.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client
_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master sessio
n id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<192.168.122.64> PUT /root/.ansible/tmp/ansible-local-334528DNix/tmpQsECq_ TO /root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843/yum.py
<192.168.122.64> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-ke
yex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 '[192.168.122.64]'
<192.168.122.64> (0, 'sftp> put /root/.ansible/tmp/ansible-local-334528DNix/tmpQsECq_ /root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843/yum.py\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuratio
n data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0
 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_r
equest_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename" revision 1\r\ndebug2: Server supports extension "statvfs" revision 2\r\ndebug2: Server supports 
extension "fstatvfs" revision 2\r\ndebug2: Server supports extension "hardlink" revision 1\r\ndebug2: Server supports extension "fsync" revision 1\r\ndebug3: Sent message fd 5 T:16 I:1\r\ndebug3: SSH_FX
P_REALPATH . -> /root size 0\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-334528DNix/tmpQsECq_\r\ndebug3: Sent message fd 5 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or
 directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843/yum.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack f
or 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:32748\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: S
SH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 32748 bytes at 65536\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit 
status from master 0\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-152
5953412.96-243665835667843/ /root/.ansible/tmp/ansible-tmp-1525953412.96-243665835667843/yum.py && sleep 0'"'"''
<192.168.122.64> (0, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mu
x_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_a
live: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit stat
us from master 0\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 -tt 192.168.122.64 '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansib
le-tmp-1525953412.96-243665835667843/yum.py && sleep 0'"'"''
<192.168.122.64> (0, '\r\n{"changed": false, "results": [], "msg": "", "rc": 0, "invocation": {"module_args": {"allow_downgrade": false, "name": ["*"], "validate_certs": true, "list": null, "disable_gpg_check": false, "conf_file": null, "
install_repoquery": true, "state": "latest", "disablerepo": null, "update_cache": true, "disable_plugin": [], "enablerepo": null, "exclude": null, "security": false, "update_only": false, "enable_plugin": [], "installroot": "/", "skip_bro
ken": false}}, "changes": {"updated": [], "installed": []}}\r\n', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebu
g2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive:
 entering\r\ndebug3: mux_client_request_alive: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: 
Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to 192.168.122.64 closed.\r\n')
<192.168.122.64> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.122.64> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o 'IdentityFile="/etc/pki/ovirt-engine/keys/engine_id_rsa"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,ho
stbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -F /var/lib/ovirt-engine/.ssh/config -o ControlPath=/root/.ansible/cp/0948316aa6 192.168.122.64 '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1525
953412.96-243665835667843/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.122.64> (0, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /var/lib/ovirt-engine/.ssh/config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mu
x_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_a
live: done pid = 3358\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit stat
us from master 0\r\n')
ok: [192.168.122.64] => {                                  
    "changed": false,                                      
    "changes": {                                           
        "installed": [],                                   
        "updated": []                                      
    },                                                     
    "invocation": {                                        
        "module_args": {                                   
            "allow_downgrade": false,                      
            "conf_file": null,                             
            "disable_gpg_check": false,                    
            "disable_plugin": [],                          
            "disablerepo": null,                           
            "enable_plugin": [],                           
            "enablerepo": null,                            
            "exclude": null,                               
            "install_repoquery": true,                     
            "installroot": "/",                            
            "list": null,                                  
            "name": [                                      
                "*"                                        
            ],                                             
            "security": false,                             
            "skip_broken": false,                          
            "state": "latest",                             
            "update_cache": true,                          
            "update_only": false,                          
            "validate_certs": true                         
        }                                                  
    },                                                     
    "msg": "",                                             
    "rc": 0,                                               
    "results": []                                          
}                                                          
META: ran handlers                                         
META: ran handlers                                         

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
192.168.122.64             : ok=2    changed=0    unreachable=0    failed=0

Comment 15 Ryan Barry 2018-05-10 12:15:58 UTC
After lowering HostedPackagesUpdateTimeInHours and re-checking, I now see:

['redhat-virtualization-host-image-update']

Huijuan/Ying - can you confirm?

Comment 17 Ryan Barry 2018-05-10 12:52:33 UTC
After re-installing the environment to re-verify, I'm back to the same results as comment#14, even after waiting. Not including the ansible output, because it's identical.

Even in the previous case, no updates were available in Engine.

[root@localhost ~]# subscription-manager register          
Registering to: subscription.rhsm.redhat.com:443/subscription                                                          
Username: rhn-engineering-rbarry                           
Password:                                                  
The system has been registered with ID: b74aef43-34cb-49df-9105-28a9597b9490                                           
The registered system name is: localhost.localdomain       
[root@localhost ~]# subscription-manager attach --auto     
Installed Product Current Status:                          
Product Name: Red Hat Virtualization Host                  
Status:       Subscribed                                   

yum[root@localhost ~]# yum check-update                    
Loaded plugins: imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager              
Obsoleting Packages                                        
redhat-virtualization-host-image-update.noarch                                                                                         4.2-20180508.0.el7_5                                                                          update   
    redhat-virtualization-host-image-update-placeholder.noarch                                                                         4.2-2.1.el7                                                                                   installed
[root@localhost ~]# yum update                             
Loaded plugins: imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager              
Resolving Dependencies                                     
--> Running transaction check                              
---> Package redhat-virtualization-host-image-update.noarch 0:4.2-20180508.0.el7_5 will be obsoleting                  
---> Package redhat-virtualization-host-image-update-placeholder.noarch 0:4.2-2.1.el7 will be obsoleted                
--> Finished Dependency Resolution                         

Dependencies Resolved                                      

==============================================================================================================================================================================================================================================
 Package                                                                        Arch                                          Version                                                     Repository                                     Size 
==============================================================================================================================================================================================================================================
Installing:                                                
 redhat-virtualization-host-image-update                                        noarch                                        4.2-20180508.0.el7_5                                        update                                        613 M 
     replacing  redhat-virtualization-host-image-update-placeholder.noarch 4.2-2.1.el7                                 

Transaction Summary                                        
==============================================================================================================================================================================================================================================
Install  1 Package                                         

Total download size: 613 M                                 
Is this ok [y/d/N]: ^C

Comment 19 Ondra Machacek 2018-05-10 16:15:04 UTC
This is indeed a bug.

The problem is the yum module in Ansible core.

The output of the 'yum check-update' command is following:

 Installing:
 redhat-virtualization-host-image-update                           noarch                           4.2-20180508.0.el7_5                            rhel-7-server-rhvh-4-beta-rpms                           613 M
     replacing  redhat-virtualization-host-image-update-placeholder.noarch 4.2-2.1.el7

The yum.py Ansible module loads it as list with following items:

available_updates = [
        "Loaded plugins: imgbased-persist, package_upload, product-id, search-disabled- repos, subscription-manager", 
        "This system is not registered with an entitlement server. You can use subscription-manager to register.", 
        "Obsoleting Packages", 
        "redhat-virtualization-host-image-update.noarch  4.2-20180508.0.el7_5  rhel-7-server-rhvh-4-beta-rpms redhat-virtualization-host-image-update-placeholder.noarch 4.2-2.1.el7           installed ", 
        ""
]

Then it checks following condition:

if '*' in line or len(line) != 3 or '.' not in line[0]:

where len(line) != 3 is true, so it ignores that.

So my question is: Is such output valid update in RHV-H scenario? If yes we need to fix yum Ansible module.

Comment 20 Ryan Barry 2018-05-10 16:32:33 UTC
This is valid output for any fresh install of RHVH which has never been updated.

It's also valid output for any package on any distro which has Obsoletes: (cockpit-storaged -> udisks2 is a recent example). It's amazing that Ansible doesn't support this.

Can we go back to the old miniyum updating for 4.2 GA for RHVH, and update once it's fixed in Ansible core, or has that code been dropped?

Comment 21 Martin Perina 2018-05-10 19:22:40 UTC
(In reply to Ryan Barry from comment #20)
> This is valid output for any fresh install of RHVH which has never been
> updated.
> 
> It's also valid output for any package on any distro which has Obsoletes:
> (cockpit-storaged -> udisks2 is a recent example). It's amazing that Ansible
> doesn't support this.
> 
> Can we go back to the old miniyum updating for 4.2 GA for RHVH, and update
> once it's fixed in Ansible core, or has that code been dropped?

No, we can't. Ondra already created a github issue, let's try to get that fixed with Ansible 2.5.3.

So we can either wait for Ansible 2.5.3 (and document a workaround using command line yum update invocation) or rebuild engine with the fixed Ansible yum module included within the engine.

Comment 22 Ryan Barry 2018-05-10 19:39:16 UTC
Up to you, Martin.

I would prefer a rebuilt engine (0day/async), but it's your call

Comment 26 Pavol Brilla 2018-06-01 09:43:13 UTC
From :
# imgbase w
You are on rhvh-4.2.3.0-0.20180518.0+1
to :
# imgbase w
You are on rhvh-4.2.3.1-0.20180531.0+1
on: 
Red Hat Virtualization Manager
Software Version:4.2.3.8-0.1.el7

Comment 28 errata-xmlrpc 2018-06-27 10:02:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2071

Comment 29 Franta Kust 2019-05-16 13:07:27 UTC
BZ<2>Jira Resync


Note You need to log in before you can comment on or make changes to this bug.