Bug 1389137
| Summary: | No Package matching 'atomic-openshift-3.4.0.16+cc70b72' found available, installed or updated when package is installed | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Mike Fiedler <mifiedle> | ||||||
| Component: | Installer | Assignee: | Tim Bielawa <tbielawa> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Anping Li <anli> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 3.4.0 | CC: | anli, aos-bugs, dgoodwin, ghuang, jeder, jialiu, jokerman, mifiedle, mmccomas, tbielawa, weshi | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | aos-scalability-34 | ||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: |
Cause: openshift_facts was parsing full package versions from 'openshift version'
Consequence: The parsed versions do not match actual yum package versions.
Fix: openshift_facts updated to chop off 'commit offset' strings from parsed versions.
Result: Parsed versions match actual yum package versions.
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2017-01-18 12:46:32 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
The workaround is set "openshift_pkg_version=-3.4.0.16" in inventory host file. Confirming the workaround in comment 1. Thanks Jianlin This is caused by "openshift version" reporting an openshift version with that +COMMIT in it. We suspect this is only for builds not built from a tag, and unlikely to show up in real builds, for 3.3 the output was: $ openshift version openshift v3.3.1.3 kubernetes v1.3.0+52492b4 etcd 2.3.0+git However we've seen this before and we should probably take action to fix it. The openshift_version is read in openshift_facts.py in openshift-ansible, method get_openshift_version. We need to split off after the '+' sign if present here. *** Bug 1389701 has been marked as a duplicate of this bug. *** For upgrade, No help to set "openshift_pkg_version=-3.4.0.16" in inventory host file. Johnny Liu, do you need anything else from me for this? (In reply to Tim Bielawa from comment #7) > Johnny Liu, do you need anything else from me for this? Once your PR is merged, QE need a rpm puddle repo which include this fix to verify this bug. Seem like Scott already did that, so no need more help from you so far. Thanks for your PR. The installation/reinstallation work well, but the upgrade still failed .
openshift-ansible-3.4.16-1.git.0.c846018.el7.noarch
changed: [openshift-180.lab.eng.nay.redhat.com]
TASK [set_fact] ****************************************************************
skipping: [openshift-189.lab.eng.nay.redhat.com]
skipping: [openshift-180.lab.eng.nay.redhat.com]
TASK [Verify containers are available for upgrade] *****************************
fatal: [openshift-180.lab.eng.nay.redhat.com]: FAILED! => {
"changed": false,
"cmd": [
"docker",
"pull",
"openshift3/ose:v3.4.0.19+346a31d"
],
"delta": "0:00:00.043069",
"end": "2016-11-02 23:15:27.799263",
"failed": true,
"rc": 1,
"start": "2016-11-02 23:15:27.756194",
"warnings": []
}
STDERR:
Error parsing reference: "openshift3/ose:v3.4.0.19+346a31d" is not a valid repository/tag
fatal: [openshift-189.lab.eng.nay.redhat.com]: FAILED! => {
"changed": false,
"cmd": [
"docker",
"pull",
"openshift3/ose:v3.4.0.19+346a31d"
],
"delta": "0:00:00.042823",
"end": "2016-11-02 23:15:27.806326",
"failed": true,
"rc": 1,
"start": "2016-11-02 23:15:27.763503",
"warnings": []
}
STDERR:
Error parsing reference: "openshift3/ose:v3.4.0.19+346a31d" is not a valid repository/tag
to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_4/upgrade.retry
PLAY RECAP *********************************************************************
localhost : ok=8 changed=7 unreachable=0 failed=0
openshift-180.lab.eng.nay.redhat.com : ok=49 changed=3 unreachable=0 failed=1
openshift-189.lab.eng.nay.redhat.com : ok=64 changed=5 unreachable=0 failed=1
Also Containerized installation still hit this issue.
Play 17/29 (Configure masters)
....................................................................................................................fatal: [openshift-117.lab.eng.nay.redhat.com]: FAILED! => {"changed": false, "cmd": ["docker", "pull", "openshift3/ose:v3.4.0.19+346a31d"], "delta": "0:00:00.068893", "end": "2016-11-03 06:10:06.307990", "failed": true, "rc": 1, "start": "2016-11-03 06:10:06.239097", "stderr": "Error parsing re
ference: \"openshift3/ose:v3.4.0.19+346a31d\" is not a valid repository/tag", "stdout": "", "stdout_lines": [], "warnings": []}
fatal: [openshift-128.lab.eng.nay.redhat.com]: FAILED! => {"changed": false, "cmd": ["docker", "pull", "openshift3/ose:v3.4.0.19+346a31d"], "delta": "0:00:00.060689", "end": "2016-11-03 06:10:06.299691", "failed": true, "rc": 1, "start": "2016-11-03 06:10:06.239002", "stderr": "Error parsing reference: \"openshift3/ose:v3.4.0.19+346a31d\" is not a valid repository/tag", "stdout": "", "stdout_lines": [], "war
nings": []}
fatal: [openshift-151.lab.eng.nay.redhat.com]: FAILED! => {"changed": false, "cmd": ["docker", "pull", "openshift3/ose:v3.4.0.19+346a31d"], "delta": "0:00:00.061939", "end": "2016-11-03 06:10:05.950320", "failed": true, "rc": 1, "start": "2016-11-03 06:10:05.888381", "stderr": "Error parsing reference: \"openshift3/ose:v3.4.0.19+346a31d\" is not a valid repository/tag", "stdout": "", "stdout_lines": [], "war
nings": []}
From looking at the output you provided in comment #10 and #11 we're not sure what's wrong. Can you please provide: * ansible.log * the inventory file Thanks! Test with today build, not hit this issue with RPM install/upgrade and container install. Since containerized upgrade blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1392701, I will try it later. Still reproduced in containerized installation which didn't specify openshift_image_tag. Log attached in comment 14. Created attachment 1218785 [details]
Containerized upgrade logs
Still reproduced in containerized upgrade which didn't specify openshift_image_tag.
Tim the relevant portion where this is going wrong is probably:
TASK [openshift_version : Set precise containerized version to configure if openshift_release specified] ***
changed: [openshift-181.lab.eng.nay.redhat.com]
- snip -
TASK [openshift_version : debug] ***********************************************
ok: [openshift-181.lab.eng.nay.redhat.com] => {
"openshift_release": "3.4"
}
TASK [openshift_version : debug] ***********************************************
ok: [openshift-181.lab.eng.nay.redhat.com] => {
"openshift_image_tag": "v3.4.0.23+24b1a58"
}
TASK [openshift_version : debug] ***********************************************
ok: [openshift-181.lab.eng.nay.redhat.com] => {
"openshift_pkg_version": "-3.4.0.23+24b1a58"
}
That task pulls down the container image with tag "v3.4" (which is openshift_release) and then attempts to read the openshift version output from that:
The task:
# If we got an openshift_version like "3.2", lookup the latest 3.2 container version
# and use that value instead.
- name: Set precise containerized version to configure if openshift_release specified
command: >
docker run --rm {{ openshift.common.cli_image }}:v{{ openshift_version }} version
register: cli_image_version
when: openshift_version is defined and openshift_version.split('.') | length == 2
- set_fact:
openshift_version: "{{ cli_image_version.stdout_lines[0].split(' ')[1].split('-')[0:2][1:] | join('-') if openshift.common.deployment_type == 'origin' else cli_image_version.stdout_lines[0].split(' ')[1].split('-')[0][1:] }}"
when: openshift_version is defined and openshift_version.split('.') | length == 2
New PR created to fix this for containerized installs available here: https://github.com/openshift/openshift-ansible/pull/2798 Patch merged into master. Let the testing begin! The installation and upgrade works without openshift_image_tag or openshift_pkg_version, so move bug to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0066 |
Created attachment 1214456 [details] Ansible log at -vvv Description of problem: OCP 3.4.0.16 + Ansible 2.2.0.0-0.62.rc1.el7 + openshift-ansible master HEAD Installing an HA environment on OSP9 that's been installed many times in the past. Built a gold image with 3.4.0.16 RPMs pre-installed - a technique we use in every deployment in this environment. When the installer gets to: Install the base package for versioning The install fails for the three master nodes with the error message: No Package matching 'atomic-openshift-3.4.0.16+cc70b72' found available, installed or updated yum list installed atomic-openshift-* on one of the nodes in question shows: Installed Packages atomic-openshift.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-clients.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-clients-redistributable.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-dockerregistry.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-master.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-node.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-pod.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-sdn-ovs.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos atomic-openshift-tests.x86_64 3.4.0.16-1.git.0.cc70b72.el7 @aos Ansible log at level -vvv attached.