Red Hat Bugzilla – Bug 1426070
excluders are not upgraded to corresponding version when upgrade ocp
Last modified: 2017-07-24 10:11 EDT
Description of problem: Upgrade ocp3.4 to ocp3.5 against rpm installed cluster, which installed and enabled atomic-openshift-excluder and atomic-openshift-docker-excluder. After upgrade succesfully, both of excluders are not upgraded together. ==before upgrade openshift-119.x.x.x | SUCCESS | rc=0 >> atomic-openshift-excluder-3.4.1.8-1.git.0.46178be.el7.noarch atomic-openshift-docker-excluder-3.4.1.8-1.git.0.46178be.el7.noarch openshift-129.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.8-1.git.0.46178be.el7.noarch atomic-openshift-excluder-3.4.1.8-1.git.0.46178be.el7.noarch ==after upgrade openshift-119.x.x.x | SUCCESS | rc=0 >> atomic-openshift-excluder-3.4.1.8-1.git.0.46178be.el7.noarch atomic-openshift-docker-excluder-3.4.1.8-1.git.0.46178be.el7.noarch openshift-129.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.8-1.git.0.46178be.el7.noarch atomic-openshift-excluder-3.4.1.8-1.git.0.46178be.el7.noarch openshift v3.5.0.32-1+4f84c83 kubernetes v1.5.2+43a9be4 etcd 3.1.0 Version-Release number of selected component (if applicable): atomic-openshift-utils-3.5.13-1.git.0.562e91d.el7.noarch How reproducible: always Steps to Reproduce: 1. rpm install ocp 3.4 2. install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default) 3. prepare 3.5 ose repo and upgrade ocp3.4 to 3.5 Actual results: Both of excluders are not upgraded together. Expected results: Excluders should be upgraded. Additional info: It seems that there is no "latest" command in related tasks. <--snip--> - name: Update to latest excluder packages package: name: "{{ openshift.common.service_type }}-excluder" when: - "{{ openshift_excluder_installed.installed_versions | default([]) | length > 0 }}" - not openshift.common.is_containerized | bool - name: Update to the latest docker-excluder packages package: name: "{{ openshift.common.service_type }}-docker-excluder" when: - "{{ docker_excluder_installed.installed_versions | default([]) | length > 0 }}" - not openshift.common.is_containerized | bool <--snip-->
I was able to reproduce it. And yes, both tasks are missing "state: latest".
Upstream fix: https://github.com/openshift/openshift-ansible/pull/3502 Fixed. Tested. Rpms after upgrade: atomic-openshift-excluder-3.5.0.34-1.git.0.9bd77cf.el7.noarch atomic-openshift-docker-excluder-3.5.0.34-1.git.0.9bd77cf.el7.noarch
blocked verify by bug1430231
Upstream PR: https://github.com/openshift/openshift-ansible/pull/3529 When the upgrade is run, an excluder is updated to the latest available version there is in available enabled repos. Before the excluders are even use, an excluder version is validated and check it if is not higher than the upgrade target version. So if e.g. the current version of OCP is 3.4 and the upgrade target is 3.5, the upgrade fails if 3.6 or higher version of excluders is available.
The excluder packages still weren't upgraded to v3.5.x. I was using atomic-openshift-utils-3.5.26-1.git.0.d88f597.el7.noarch [root@openshift-182 ~]# rpm -qa|grep atomic atomic-openshift-3.5.0.45-1.git.0.4a06220.el7.x86_64 atomic-openshift-clients-3.5.0.45-1.git.0.4a06220.el7.x86_64 atomic-openshift-node-3.5.0.45-1.git.0.4a06220.el7.x86_64 atomic-openshift-master-3.5.0.45-1.git.0.4a06220.el7.x86_64 atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch tuned-profiles-atomic-openshift-node-3.5.0.45-1.git.0.4a06220.el7.x86_64 atomic-openshift-sdn-ovs-3.5.0.45-1.git.0.4a06220.el7.x86_64 [root@openshift-182 ~]# yum search atomic-openshift-excluder --showdup Loaded plugins: product-id, search-disabled-repos, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. ===================================================================================== N/S matched: atomic-openshift-excluder ====================================================================================== atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch : Exclude openshift packages from updates atomic-openshift-excluder-3.5.0.45-1.git.0.4a06220.el7.noarch : Exclude openshift packages from updates
Fixed in openshift-ansible-3.5.27-1.git.0.08cda99.el7
The excluder was upgraded to correct version with openshift-ansible-3.5.27-1.git.0.08cda99.el7 [root@openshift-214 ~]# rpm -qa|grep atomic- atomic-openshift-docker-excluder-3.5.0.49-1.git.0.c8e072a.el7.noarch atomic-openshift-3.5.0.49-1.git.0.c8e072a.el7.x86_64 atomic-openshift-master-3.5.0.49-1.git.0.c8e072a.el7.x86_64 atomic-openshift-excluder-3.5.0.49-1.git.0.c8e072a.el7.noarch atomic-openshift-clients-3.5.0.49-1.git.0.c8e072a.el7.x86_64 tuned-profiles-atomic-openshift-node-3.5.0.49-1.git.0.c8e072a.el7.x86_64 atomic-openshift-sdn-ovs-3.5.0.49-1.git.0.c8e072a.el7.x86_64 atomic-openshift-node-3.5.0.49-1.git.0.c8e072a.el7.x86_64
Version: atomic-openshift-utils-3.5.28-1.git.0.103513e.el7.noarch Steps: 1. rpm install ocp 3.4 2. install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default) 3. prepare 3.5 ose repo and upgrade ocp3.4 to 3.5 Result: before upgrade: openshift-118.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch openshift-119.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch after upgrade: openshift-118.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.5.0.49-1.git.0.c8e072a.el7.noarch openshift-119.x.x.x | SUCCESS | rc=0 >> atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.5.0.49-1.git.0.c8e072a.el7.noarch Atomic-openshift-docker-excluder is not updated.
Upstream PR: https://github.com/openshift/openshift-ansible/pull/3607
liujia, there is no difference between 3.5.27-1 and 3.5.28-1 wrt. excluders. Can you share your inventory file?
I am not able to reproduce it with the latest commit in master branch with [1] applied. I will run another installation and upgrade scenario later on again once [1] and its dependent PR [2] are merged. [1] https://github.com/openshift/openshift-ansible/pull/3620 [2] https://github.com/openshift/openshift-ansible/pull/3610
additional changes merged into openshift-ansible-3.5.31, ON_QA
Version: atomic-openshift-utils-3.5.32-1.git.0.42cf266.el7.noarch Steps: 1. rpm install ocp 3.4 2. install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default) 3. prepare 3.5 ose repo and upgrade ocp3.4 to 3.5 Result: before upgrade # rpm -qa | grep excluder atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch after upgrade # rpm -qa | grep excluder atomic-openshift-excluder-3.5.0.51-1.git.0.d07622a.el7.noarch atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-docker-excluder is still not upgraded to corresponding version.
Version: atomic-openshift-utils-3.5.39-1.git.0.cb12cdb.el7.noarch Steps: 1. rpm install ocp 3.4 2. install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default) 3. prepare 3.5 ose repo and upgrade ocp3.4 to 3.5 Result: before upgrade # rpm -qa | grep excluder atomic-openshift-docker-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch atomic-openshift-excluder-3.4.1.10-1.git.0.c96aed3.el7.noarch after upgrade # rpm -qa | grep excluder atomic-openshift-excluder-3.5.0.55-1.git.0.a552679.el7.noarch atomic-openshift-docker-excluder-3.5.0.55-1.git.0.a552679.el7.noarch atomic-openshift-docker-excluder is upgraded to corresponding version.