Red Hat Bugzilla – Bug 1421587
etcd-3.0 did not upgrade to etcd-3.1 during upgrade ocp3.4 to ocp3.5
Last modified: 2018-01-12 09:51:44 EST
Description of problem: Run upgrade playbook to upgrade ocp3.4 to ocp3.5, upgrade fail and exit at task [Update etcd RPM] when there is new etcd-3.1.0 package for etcd update(original etcd is etcd-3.0). TASK [Update etcd RPM] ********************************************************* fatal: [openshift-129.x.x.x]: FAILED! => { "changed": false, "failed": true, "rc": 126, "results": [ "No package matching 'etcd-3.0*' found available, installed or updated" ] } MSG: No package matching 'etcd-3.0*' found available, installed or updated Version-Release number of selected component (if applicable): atomic-openshift-utils-3.5.7-1.git.0.5010dec.el7.noarch How reproducible: always Steps to Reproduce: 1. install ocp3.4 with etcd-3.0 2. upgrade ocp3.4 to ocp3.5 3. Actual results: Upgrade fail at task [Update etcd RPM]. Expected results: Upgrade successfully. Additional info: # yum search etcd --show-dup Loaded plugins: product-id, search-disabled-repos, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. ============================================================================= N/S matched: etcd ============================================================================= etcd-3.1.0-1.el7.x86_64 : A highly-available key value store for shared configuration etcd3-3.0.14-3.el7.x86_64 : A highly-available key value store for shared configuration
I believe this is a host with incorrectly configured repos. Do you have normal RHEL 7 extras channels enabled? If so you should see all versions of etcd available to you. For instance, here's my host which has normal rhel 7 extras repo and the nightly rhel 7 extras repos too. [root@ose3-master ~]# yum search etcd --show-dup Loaded plugins: product-id, search-disabled-repos, subscription-manager Repository 'rhel73-extras' is missing name in configuration, using id ================================================= N/S matched: etcd ================================================== etcd-0.4.6-0.12.el7.x86_64 : A highly-available key value store for shared configuration etcd-0.4.6-0.13.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.0.9-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.0.11-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.0.13-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.1.1-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.2.2-5.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.2.5-1.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.2.5-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.3.7-2.el7.x86_64 : A highly-available key value store for shared configuration etcd-2.3.7-4.el7.x86_64 : A highly-available key value store for shared configuration etcd-3.0.15-1.el7.x86_64 : A highly-available key value store for shared configuration etcd-3.1.0-1.el7.x86_64 : A highly-available key value store for shared configuration etcd-3.1.0-1.el7.x86_64 : A highly-available key value store for shared configuration etcd3-3.0.3-1.el7.x86_64 : A highly-available key value store for shared configuration etcd3-3.0.14-2.el7.x86_64 : A highly-available key value store for shared configuration etcd3-3.0.14-3.el7.x86_64 : A highly-available key value store for shared configuration And, installing etcd-3.0* works just fine.
@scott 1) Above test use nightly latest rhel 7 extras repo, no other extra repo. There are only etcd-3.1 available. [ ] etcd-3.1.0-1.el7.x86_64.rpm 06-Feb-2017 19:59 7.3M [ ] etcd3-3.0.14-3.el7.x86_64.rpm 2) I think this issue's key point is "update etcd" means to update current etcd-3.0 to latest version etcd-3.1(if available). But the task just serach "etcd-3.0* " for update. For the second point, I'm not quite sure if we need to update etcd to latest etcd-3.1, or ocp3.4/ocp3.5 just support etcd-3.0, not etcd-3.1? BTW, now for ocp3.5 install, it use latest version of etcd(etcd-3.1) by default. # rpm -qa | grep etcd etcd-3.1.0-1.el7.x86_64 # # openshift version openshift v3.5.0.20+87266c6 kubernetes v1.5.2+43a9be4 etcd 3.1.0
Please test this against a system that's registered to RHSM and has production rhel-7-server-extras-rpms repos enabled. This cannot be tested with only the nightly repo as that repo only has the latest build.
Version: atomic-openshift-utils-3.5.23-1.git.0.1cd0089.el7.noarch Steps: 1. install ocp3.4 with etcd-3.0 2. upgrade ocp3.4 to ocp3.5(with rhel-7-server-extras-rpm repo which included etcd-3.0.15 and etcd-3.1.0) 3. upgrade successfully but etcd did not upgrade to latest etcd-3.1.0. The root cause is the same as original issue that [Update etcd RPM] task just serach "etcd-3.0* " for update. If we need to file a new bug to track it or just change bug title to track the same issue at this bug?
Since the root cause has not fixed, change status back.
Ok, we'll add support for 3.1, but this is not a blocker for 3.5.
liujia, can you verify the issue again? It was implemented as part of #4217 at [1]. [1] https://github.com/openshift/openshift-ansible/pull/4217/files#diff-d468b4ff93b6b8a76d3ae210ea2d31d5R88
(In reply to Jan Chaloupka from comment #7) > liujia, can you verify the issue again? It was implemented as part of #4217 > at [1]. > > [1] > https://github.com/openshift/openshift-ansible/pull/4217/files#diff- > d468b4ff93b6b8a76d3ae210ea2d31d5R88 Jan, it works well on atomic-openshift-utils-3.6.116-1.git.0.e2840e8.el7.noarch. I think it should be fixed.
Version： atomic-openshift-utils-3.6.116-1.git.0.e2840e8.el7.noarch 1. Install ocp3.5 with a host installed etcd-3.0 in advance. 2. Upgrade ocp3.5 to 3.6 Upgrade successfully and etcd updated to 3.1.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716
The errata for the is for 3.6 while the original issue seemed to be with upgrading from 3.4 to 3.5. Can you confirm that this is also fixed in 3.5 and if so can you point to the specific errata?
(In reply to Brendan Mchugh from comment #13) > The errata for the is for 3.6 while the original issue seemed to be with > upgrading from 3.4 to 3.5. > > Can you confirm that this is also fixed in 3.5 and if so can you point to > the specific errata? No, 3.5 playbooks will not upgrade them to etcd 3.1 or 3.2. If they were to upgrade to OCP 3.6 then they'd be upgraded to etcd 3.1 and then 3.2 in a stepwise manner. Unless the customer is encountering a known issue in etcd 3.0 I'd recommend they continue running etcd 3.0 to minimize risk.