Bug 1388760

Summary: [3.4] quick install - No error message output when the packages can't be found.
Product: OpenShift Container Platform Reporter: liujia <jiajliu>
Component: InstallerAssignee: Tim Bielawa <tbielawa>
Status: CLOSED ERRATA QA Contact: liujia <jiajliu>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.4.0CC: aos-bugs, jiajliu, jokerman, mmccomas
Target Milestone: ---   
Target Release: 3.4.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-04-25 17:42:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description liujia 2016-10-26 06:37:54 UTC
Description of problem:
No error message when the iptables-service packages can not be found. if we add -v, we can got message "No Package matching 'iptables-services' found available, installed or updated".

<--snip-->
................................................................
Play 9/29 (Set oo_option facts)
.......
Play 10/29 (Configure etcd)
...........................................................................................................
localhost                  : ok=10   changed=9    unreachable=0    failed=0   
openshift-x.x.x.x : ok=49   changed=0    unreachable=0    failed=0   
openshift-x.x.x.x : ok=64   changed=1    unreachable=0    failed=1   
 
Re-run install with "-v", it indicated that from the output the iptables packages failed to install.
#atomic-openshift-installer -v -c ~/.config/openshift/installer.cfg.yml install

TASK [os_firewall : Install iptables packages] *********************************
ok: [openshift-x.x.x.x] => (item=[u'iptables', u'iptables-services']) => {
    "changed": false, 
    "failed": true, 
    "item": [
        "iptables", 
        "iptables-services"
    ], 
    "rc": 0, 
    "results": [
        "iptables-1.4.21-17.el7.x86_64 providing iptables is already installed"
    ]
}

MSG:
No Package matching 'iptables-services' found available, installed or updated


Version-Release number of selected component (if applicable):
ansible-2.2.0.0-0.62.rc1.el7.noarch
atomic-openshift-utils-3.4.12-1.git.0.0b5efd2.el7.noarch


How reproducible:
always

Steps to Reproduce:
1. trigger an quick install with an error repo configured.
atomic-openshift-installer -c ~/.config/openshift/installer.cfg.yml install


Actual results:

the install will fail but no obvious errors output.
Play 10/29 (Configure etcd)
...........................................................................................................
localhost                  : ok=10   changed=9    unreachable=0    failed=0   
openshift-x.x.x.x : ok=49   changed=0    unreachable=0    failed=0   
openshift-x.x.x.x : ok=64   changed=1    unreachable=0    failed=1  

check for more info in ansible.log, there is no fail info either.

Expected results:
It should be output fail info on the screen.


Additional info:

Comment 1 Tim Bielawa 2016-10-28 16:32:56 UTC
This isn't a critical error so I'm pushing it to UpcomingRelease while we focus on blocker bugs.

Comment 2 Tim Bielawa 2017-01-09 20:25:24 UTC
@liujia,

Does this other bug (fixed) change anything you reported in this BZ?

https://bugzilla.redhat.com/show_bug.cgi?id=1377619#c7

Comment 3 liujia 2017-01-22 06:15:27 UTC
Hi Tim
I think this bug is different from the fixed one(BZ1377619). BZ1377619 is to get the whole install playbook stopped once en error happens. But this bug described the problem about no error info printed when quick install failed and stopped. It should output the error msg to help user resolve the problems when install failed even if quick installer used quiet mode now.

I still reproduced it on latest puddle of 3.4. 

Version:
atomic-openshift-utils-3.4.55-1.git.0.9cb1f40.el7.noarch

Step:
1.prepare an error rhel repo on the host to trigger failure happen(for example,a lower version of iptables-services packages in the repo),other repos should be correct.
2.trigger a quick install against the host
#atomic-openshift-installer -c ~/.config/openshift/installer.cfg.yml install

Result:
Play 9/29 (Set oo_option facts)
.......
Play 10/29 (Configure etcd)
..........................................................................................................................
localhost                  : ok=10   changed=0    unreachable=0    failed=0   
openshift-149.lab.eng.nay.redhat.com : ok=59   changed=5    unreachable=0    failed=1   


Installation Complete: Note: Play count is an estimate and some were skipped because your install does not require them


An error was detected. After resolving the problem please relaunch the
installation process.

Comment 4 Scott Dodson 2017-02-10 02:29:03 UTC
Tim,

Can we surface the last error whenever an error occurs?

Comment 5 Tim Bielawa 2017-03-16 17:14:48 UTC
> Can we surface the last error whenever an error occurs?

That should be possible. We already direct ALL ansible output to a log file, so we should be able to check for errors and print those out if we want.

Comment 6 Tim Bielawa 2017-03-17 20:07:33 UTC
I've began scaffolding some code to make this work.

https://github.com/openshift/openshift-ansible/pull/3702

Comment 7 Tim Bielawa 2017-03-20 19:23:13 UTC
I think the code for this is complete now. I've tested it for a few hours and the package errors always bubble up immediately. See [0] for a screenshot and additional words describing the fix.

[0] - https://github.com/openshift/openshift-ansible/pull/3702#issuecomment-287869043

Waiting on review now and then merging and backporting.

Comment 8 Tim Bielawa 2017-03-22 17:29:27 UTC
Merged into master.

Comment 9 Tim Bielawa 2017-03-22 17:32:23 UTC
Just realized this is filed against 3.4. I'll get some backports ready.

Comment 10 Tim Bielawa 2017-03-23 15:39:57 UTC
3.3 and 3.4 backports have been merged

https://github.com/openshift/openshift-ansible/pull/3724
https://github.com/openshift/openshift-ansible/pull/3723

3.5 backport is going through the test systems again now.

Comment 11 Tim Bielawa 2017-03-23 17:15:18 UTC
3.5 has been merged.

Comment 13 liujia 2017-04-20 02:08:58 UTC
Version:
atomic-openshift-utils-3.4.74-1.git.0.6542413.el7.noarch

Step:
1. prepare an rhel extra repo which has lower docker dependency pkg on the host to trigger failure happen,other repos should be correct.
2. trigger a quick install against the host.

Result:
Play 18/29 (Additional master configuration)
......................................................................................................................................................................................................failed: [openshift-129.x.x.x] (item=[u'cockpit-ws', u'cockpit-shell', u'cockpit-bridge', u'cockpit-docker', u'cockpit-kubernetes']) => {"changed": true, "failed": true, "item": ["cockpit-ws", "cockpit-shell", "cockpit-bridge", "cockpit-docker", "cockpit-kubernetes"], "msg": "Error: Package: cockpit-kubernetes-135-4.el7.x86_64 (ose-v34)\n           Requires: cockpit-shell >= 122\n           Installing: cockpit-shell-118-1.el7.noarch (rhel-7-extra)\n               cockpit-shell = 118-1.el7\nError: Package: cockpit-kubernetes-135-4.el7.x86_64 (ose-v34)\n           Requires: cockpit-bridge >= 122\n           Installing: cockpit-bridge-118-1.el7.x86_64 (rhel-7-extra)\n               cockpit-bridge = 118-1.el7\n", "rc": 1, "results": ["Loaded plugins: product-id, search-disabled-repos, subscription-manager\nThis system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.\nResolving Dependencies\n--> Running transaction check\n---> Package cockpit-bridge.x86_64 0:118-1.el7 will be installed\n--> Processing Dependency: libjson-glib-1.0.so.0()(64bit) for package: cockpit-bridge-118-1.el7.x86_64\n---> Package cockpit-docker.x86_64 0:118-1.el7 will be installed\n---> Package cockpit-kubernetes.x86_64 0:135-4.el7 will be installed\n--> Processing Dependency: cockpit-shell >= 122 for package: cockpit-kubernetes-135-4.el7.x86_64\n--> Processing Dependency: cockpit-bridge >= 122 for package: cockpit-kubernetes-135-4.el7.x86_64\n---> Package cockpit-shell.noarch 0:118-1.el7 will be installed\n---> Package cockpit-ws.x86_64 0:118-1.el7 will be installed\n--> Processing Dependency: libssh >= 0.7.1 for package: cockpit-ws-118-1.el7.x86_64\n--> Processing Dependency: libssh.so.4()(64bit) for package: cockpit-ws-118-1.el7.x86_64\n--> Processing Dependency: libssh_threads.so.4()(64bit) for package: cockpit-ws-118-1.el7.x86_64\n--> Running transaction check\n---> Package cockpit-kubernetes.x86_64 0:135-4.el7 will be installed\n--> Processing Dependency: cockpit-shell >= 122 for package: cockpit-kubernetes-135-4.el7.x86_64\n--> Processing Dependency: cockpit-bridge >= 122 for package: cockpit-kubernetes-135-4.el7.x86_64\n---> Package json-glib.x86_64 0:1.0.2-1.el7 will be installed\n---> Package libssh.x86_64 0:0.7.1-2.el7 will be installed\n--> Finished Dependency Resolution\n You could try using --skip-broken to work around the problem\n You could try running: rpm -Va --nofiles --nodigest\n"]}

localhost                  : ok=12   changed=0    unreachable=0    failed=0   
openshift-129.x.x.x : ok=329  changed=100  unreachable=0    failed=1 

Error info printed.

Comment 14 liujia 2017-04-20 04:51:39 UTC
Since no errata puddle, i downloaded packages from advisory RHBA-2017:27716-02 to verify the bug on following version:
atomic-openshift-utils-3.4.79-1.git.0.6faa668.el7.noarch
openshift-ansible-3.4.79-1.git.0.6faa668.el7.noarch

It will output error msg when fail to continue the task.

Play 10/29 (Configure etcd)
...............................................................................................................................Pausing for 10 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
...Pausing for 10 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
..................fatal: [openshift-129.x.x.x]: FAILED! => {"changed": true, "failed": true, "msg": "http://download-node-02.eng.bos.redhat.com/nightly/EXTRAS-RHEL-7.3/EXTRAS-7.3-RHEL-7-20160909.n.0/compose/Server/x86_64/os/rhel-7-server-extras-rpms/Packages/etcd-3.1.3-1.el7.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found\nTrying other mirror.\nTo address this issue please refer to the below knowledge base article \n\nhttps://access.redhat.com/articles/1320623\n\nIf above article doesn't help to resolve this issue please open a ticket with Red Hat Support.\n\n\n\nError downloading packages:\n  etcd-3.1.3-1.el7.x86_64: [Errno 256] No more mirrors to try.\n\n", "rc": 1, "results": ["Loaded plugins: product-id, search-disabled-repos, subscription-manager\nThis system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.\nResolving Dependencies\n--> Running transaction check\n---> Package etcd.x86_64 0:3.1.3-1.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package       Arch            Version              Repository             Size\n================================================================================\nInstalling:\n etcd          x86_64          3.1.3-1.el7          rhel-7-extra          7.3 M\n\nTransaction Summary\n================================================================================\nInstall  1 Package\n\nTotal download size: 7.3 M\nInstalled size: 28 M\nDownloading packages:\n"]}

localhost                  : ok=10   changed=0    unreachable=0    failed=0   
openshift-129.x.x.x : ok=79   changed=14   unreachable=0    failed=1   


Change the status to verify.

Comment 16 errata-xmlrpc 2017-04-25 17:42:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1140