Bug 1338965 - necessary tag is not imported, preventing logging deployment
Summary: necessary tag is not imported, preventing logging deployment
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Luke Meyer
QA Contact: chunchen
URL:
Whiteboard:
: 1339060 (view as bug list)
Depends On:
Blocks: 1267746
TreeView+ depends on / blocked
 
Reported: 2016-05-23 18:31 UTC by Luke Meyer
Modified: 2019-11-14 08:07 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Cause: When ImageStreams are created, only a subset of the available tags are imported, and this often excluded the desired tag. Consequence: If the desired tag isn't imported, then the corresponding component never deploys. Workaround (if any): Import each tag manually like so: oc import-image <name>:<version> --from <prefix><name>:<tag> The bug is fixed in 3.3 by not relying on ImageStreams and DC triggers for deployment. Result: Deployment occurs as expected.
Clone Of:
Environment:
Last Closed: 2016-09-27 09:33:03 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1933 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.3 Release Advisory 2016-09-27 13:24:36 UTC

Description Luke Meyer 2016-05-23 18:31:41 UTC
Description of problem:
Refer to issue https://github.com/openshift/origin/issues/8948

The logging deployment sets up ImageStreams and related DeploymentConfigs. Triggers on each DC result in deployment once the necessary tag is imported into the IS. However, it turns out that only a subset of the available tags are imported automatically (ref https://github.com/openshift/origin/issues/8952), and 3.2.0 (the current default) is frequently not among them. Thus deployment never occurs.


Version-Release number of selected component (if applicable):
Noticed most frequently with 3.2.0, however it will probably affect all versions.

How reproducible:
More often than not. It's not clear why sometimes v3.2 is imported versus 3.2.0.


Steps to Reproduce:
1. Follow deployment instructions at https://docs.openshift.com/enterprise/3.2/install_config/aggregate_logging.html

Actual results:
Deployment occurs in some cases, but not others. In the cases where it doesn't, it's because the 3.2.0 tag was not imported, although "latest" and "v3.2" (which have the same image ID) are.

Expected results:
Consistent deployment.


Additional info:
The quick "fix" will be to update docs to indicate the following workaround:

$ oc import-image logging-auth-proxy:3.2.0 \
     --from registry.access.redhat.com/openshift3/logging-auth-proxy:3.2.0 
$ oc import-image logging-kibana:3.2.0 \
     --from registry.access.redhat.com/openshift3/logging-kibana:3.2.0
$ oc import-image logging-elasticsearch:3.2.0 \
     --from registry.access.redhat.com/openshift3/logging-elasticsearch:3.2.0
$ oc import-image logging-fluentd:3.2.0 \
     --from registry.access.redhat.com/openshift3/logging-fluentd:3.2.0

As a more permanent fix, we will dispense with the ImageStreams and triggers, simply deploying the intended images directly.

Comment 1 openshift-github-bot 2016-05-23 20:42:04 UTC
Commit pushed to master at https://github.com/openshift/openshift-docs

https://github.com/openshift/openshift-docs/commit/0e6244897c81dabf0a6634d69a683d0dd2c48bd7
logging: manually import tags for deployment

Docs fix to bug 1338965
https://bugzilla.redhat.com/show_bug.cgi?id=1338965

This is an ugly workaround, but hopefully short-term.

Comment 2 Luke Meyer 2016-05-24 14:11:58 UTC
*** Bug 1339060 has been marked as a duplicate of this bug. ***

Comment 3 Luke Meyer 2016-05-24 14:16:53 UTC
Docs workaround at https://github.com/openshift/openshift-docs/pull/2141

Comment 5 Josep 'Pep' Turro Mauri 2016-06-23 11:10:42 UTC
While there's a docs workaround that helps in the logging deployment (and possibly others), shouldn't this track the problem of import-image?

I.e. https://github.com/openshift/origin/pull/9163

and if so, we should probably move it from Logging to CLI.

Comment 6 Luke Meyer 2016-06-28 20:36:13 UTC
(In reply to Josep 'Pep' Turro Mauri from comment #5)
> While there's a docs workaround that helps in the logging deployment (and
> possibly others), shouldn't this track the problem of import-image?
> 
> I.e. https://github.com/openshift/origin/pull/9163

I wouldn't expect that to get into Enterprise before 3.3, and by that time I fully expect to have eliminated the problem by removing the usage of ImageStreams in logging deployment.

Agree a note in the upgrade doc would help too until then.

Comment 7 Luke Meyer 2016-06-29 21:25:23 UTC
(In reply to Xia Zhao from comment #4)
> FYI, The change is also needed in upgrade doc
> https://docs.openshift.com/enterprise/3.2/install_config/upgrading/
> manual_upgrades.html#manual-upgrading-efk-logging-stack

Could you validate that https://github.com/openshift/openshift-docs/pull/2428 addresses the problem and the upgrade sequence works with this in place?

I think we'll want to keep this bug open to track the ultimate solution to this; docs are a stopgap.

Comment 8 Xia Zhao 2016-07-01 03:29:22 UTC
@lmeyer The modification on upgrade doc and also the sequence mentioned looks good to me.

Comment 9 Luke Meyer 2016-07-01 20:26:50 UTC
Note published in the upgrade docs https://docs.openshift.com/enterprise/latest/install_config/upgrading/manual_upgrades.html#manual-upgrading-efk-logging-stack

Keeping the bug open for tracking removing imagestreams entirely.

Comment 10 Luke Meyer 2016-08-17 18:57:32 UTC
As of 3.3 imagestreams and DC triggers are not a part of the deployment, so this should not be a problem.

Comment 12 Xia Zhao 2016-08-19 02:59:21 UTC
Yes, we never encountered this issue with the 3.3.0 logging deployer, set to verified.

Comment 14 errata-xmlrpc 2016-09-27 09:33:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933


Note You need to log in before you can comment on or make changes to this bug.