Bug 1826781 - Failed to upgrade logging from 4.4 to 4.5 - Could not resolve placeholder 'POD_IP'
Summary: Failed to upgrade logging from 4.4 to 4.5 - Could not resolve placeholder 'PO...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.5
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 4.5.0
Assignee: Jeff Cantrill
QA Contact: Qiaoling Tang
URL:
Whiteboard:
: 1829971 (view as bug list)
Depends On:
Blocks: 1824006 1829971
TreeView+ depends on / blocked
 
Reported: 2020-04-22 14:32 UTC by IgorKarpukhin
Modified: 2020-07-13 17:30 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: EO operator wasn't updating ENV for deployment and stateful set Consequence: EO operator wasn't starting after 4.4 -> 4.5 upgrade Fix: Fixed by explicitly updating ENV variables for the EO operator Result: Bug fixed
Clone Of: 1809498
: 1829064 (view as bug list)
Environment:
Last Closed: 2020-07-13 17:30:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift elasticsearch-operator pull 316 0 None closed Bug 1826781: Added ENV comparison for ES deployment and stateful set 2020-12-11 19:27:51 UTC
Github openshift elasticsearch-operator pull 353 0 None closed Bug 1826781: Add test to support container checks 2020-12-11 19:27:51 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:30:39 UTC

Description IgorKarpukhin 2020-04-22 14:32:53 UTC
+++ This bug was initially created as a clone of Bug #1809498 +++

We are likely still awaiting proper images from ART since this is trying to boot an ES5 node.  ES6 will be used in 4.5.  What images does the EO say it should be using?  What images does the CLO say it should be using?

--- Additional comment from Anping Li on 2020-03-10 03:04:39 UTC ---

Yes, there isn't 4.5 channel in operator-registry redhat-operatrors-art. Waiting art.

--- Additional comment from Jeff Cantrill on 2020-04-14 20:08:20 UTC ---

Please retest now there are available images for 4.5. One suspicion I have is if you upgrade CLO before EO that records will be written to an index with a '-write' suffix and thus the new indexmanagement will be broken

Comment 3 Qiaoling Tang 2020-04-23 01:25:38 UTC
I tested upgrade from 4.4 to 4.5 again, still hit the issue in https://bugzilla.redhat.com/show_bug.cgi?id=1809498#c0. 

For the issue in https://bugzilla.redhat.com/show_bug.cgi?id=1809498#c4, it didn't appear. I checked the logging 4.4, the env var `POD_IP` was added to the elasticsearch when deploying logging 4.4.

I also tested upgrade from 4.3 to 4.5, the issue in https://bugzilla.redhat.com/show_bug.cgi?id=1809498#c4 didn't appear, but I hit the same issue in https://bugzilla.redhat.com/show_bug.cgi?id=1809498#c0. The env var `POD_IP` wasn't added to the elasticsearch when deploying logging 4.3.


It seems the fix PR https://github.com/openshift/elasticsearch-operator/pull/316 only fixes the issue in https://bugzilla.redhat.com/show_bug.cgi?id=1809498#c4. @IgorKarpukhin, is my understanding correct?


4.5 logging images are from 4.5.0-0.ci-2020-04-22-212726.

Comment 4 IgorKarpukhin 2020-04-23 11:23:58 UTC
Yes, that's correct. This fix isn't backported to 4.4 yet. I'm working on it.

Comment 5 IgorKarpukhin 2020-04-23 11:54:32 UTC
Qiaoling Tang could you please post the logs from catalog operator during the upgrade from 4.3 to 4.4?

Ex: oc logs -f catalog-operator-7648d9cff4-rqx9j -n openshift-operator-lifecycle-manager

Comment 6 Qiaoling Tang 2020-04-24 00:36:27 UTC
Catalog operator logs: http://pastebin.test.redhat.com/858655

Cluster version: 4.4.0-0.nightly-2020-04-23-192521

Comment 8 Jeff Cantrill 2020-04-28 18:06:54 UTC
Moving to ON_QA.  Qiaoling if you feel it is reasonable to move this bug to Verified please do so.,

Comment 9 Qiaoling Tang 2020-04-29 00:53:19 UTC
Per the discussions in the above, I move this to VERIFIED.

Feel free to reopen this bz if you have any problems related to this issue.

Comment 10 Jeff Cantrill 2020-05-07 17:13:34 UTC
*** Bug 1829971 has been marked as a duplicate of this bug. ***

Comment 11 IgorKarpukhin 2020-05-19 08:11:45 UTC
This issue has been fixed by this PR: https://github.com/openshift/elasticsearch-operator/pull/316. Not a bug anymore.

Comment 12 Ben Parees 2020-05-19 13:03:11 UTC
moving back to verified.  the release process will close this when appropriate.

Comment 13 errata-xmlrpc 2020-07-13 17:30:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.