Bug 1417526 - openshift_hosted_metrics_deploy does not set IMAGE_PREFIX to "openshift_docker_additional_registries"
Summary: openshift_hosted_metrics_deploy does not set IMAGE_PREFIX to "openshift_docke...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.4.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Matt Wringe
QA Contact: Peng Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-30 05:35 UTC by Peter Larsen
Modified: 2017-07-24 14:11 UTC (History)
8 users (show)

Fixed In Version: 3.5.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-12 19:10:58 UTC
Target Upstream Version:


Attachments (Terms of Use)
Anisble inventory used for install (31.22 KB, text/plain)
2017-01-30 16:00 UTC, Peter Larsen
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0884 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.5 RPM Release Advisory 2017-04-12 22:50:07 UTC

Description Peter Larsen 2017-01-30 05:35:23 UTC
Description of problem:
When enabling metrics deployment, and using an local docker registry for installation base, the IMAGE_PREFIX isn't set to reflect the additional (new) registry. As a result, the pull is attempted from registry.access.redhat.com regardless of parameter.

Error:
======== ERROR ========= 
validate_deployment_artifacts:
Pod hawkular-cassandra-1-z720w from ReplicationController hawkular-cassandra-1 is in a Pending state.
This is most often due to waiting for the container image to pull and should eventually resolve.
  * * * *
Pod hawkular-metrics-3o7p6 from ReplicationController hawkular-metrics is in a Pending state.
This is most often due to waiting for the container image to pull and should eventually resolve.
  * * * *
Pod heapster-rhs8v from ReplicationController heapster specified an image that cannot be pulled.
ERROR: This is most often due to the image name being wrong or the docker registry being unavailable.
Ensure that you used the correct IMAGE_PREFIX and IMAGE_VERSION with the deployment.
There was an event for this pod with the following message:
Failed to pull image "registry.access.redhat.com/openshift3/metrics-heapster:3.4.0": image pull failed for registry.access.redhat.com/openshift3/metrics-heapster:3.4.0, this may be because there are no credentials on this request.  details: (net/http: request canceled)


Version-Release number of selected component (if applicable):
3.4

How reproducible:
Every time

Steps to Reproduce:
1. Install with openshift_hosted_metrics_deploy=true, and with a custom registry
2. Deployment of metrics fails - ansible does not fail, but deployment ends with failure (see above). 

Expected results:
Metrics installation run should set IMAGE_PREFIX to reflect the additional_registry setting. Or there needs to be a parameter to set this value (registry/logging should follow the same setup).


Additional info:

Comment 1 Scott Dodson 2017-01-30 14:35:03 UTC
The templates are updated when a combination of oreg_url and openshift_examples_modify_imagestreams are set. Can you please provide your full inventory?

Comment 2 Peter Larsen 2017-01-30 16:00:07 UTC
Created attachment 1245918 [details]
Anisble inventory used for install

The requested ansible inventory used to install from.

Comment 3 Peter Larsen 2017-01-30 16:01:51 UTC
(In reply to Scott Dodson from comment #1)
> The templates are updated when a combination of oreg_url and
> openshift_examples_modify_imagestreams are set. Can you please provide your
> full inventory?

I've uploaded the inventory file, and here are the settings relevant for this issue:

openshift_docker_additional_registries=satellite.peterlarsen.org:5000
openshift_docker_insecure=satellite.peterlarsen.org
openshift_docker_disable_push_dockerhub=True
oreg_url=peterlarsen_org-ocp_docker_registry-ose-${component}:${version}

Comment 4 Peter Larsen 2017-01-30 16:02:43 UTC
Forgot this line:
openshift_examples_modify_imagestreams=true

Comment 5 Marko Myllynen 2017-02-06 11:15:32 UTC
(In reply to Peter Larsen from comment #3)
> (In reply to Scott Dodson from comment #1)
> 
> openshift_docker_additional_registries=satellite.peterlarsen.org:5000
> openshift_docker_insecure=satellite.peterlarsen.org
> openshift_docker_disable_push_dockerhub=True
> oreg_url=peterlarsen_org-ocp_docker_registry-ose-${component}:${version}

These work for me, I raised an upstream ticket (https://github.com/openshift/openshift-ansible/issues/3263) about the missing openshift_hosted_metrics_deployer_version example from the reference file:

openshift_docker_additional_registries=registry.example.com:5000
openshift_docker_insecure_registries=registry.example.com:5000
openshift_docker_disable_push_dockerhub=True
oreg_url=registry.example.com:5000/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_hosted_metrics_deployer_version=latest
openshift_hosted_logging_deployer_version=latest

But please also note https://bugzilla.redhat.com/show_bug.cgi?id=1419493 and https://bugzilla.redhat.com/show_bug.cgi?id=1419497.

Thanks.

Comment 6 Matt Wringe 2017-02-09 23:30:23 UTC
How is this a metrics issue exactly?

Comment 8 Matt Wringe 2017-02-17 20:10:44 UTC
"Metrics installation run should set IMAGE_PREFIX to reflect the additional_registry setting. Or there needs to be a parameter to set this value (registry/logging should follow the same setup)."

Can you not just set the "openshift_hosted_metrics_deployer_prefix" value to point to your own registry that you want to use?

The default value for this is "registry.access.redhat.com/openshift3/" so you will probably just want to change the 'registry.access.redhat.com' part.

Comment 9 Peter Larsen 2017-02-21 16:41:53 UTC
(In reply to Matt Wringe from comment #8)
> "Metrics installation run should set IMAGE_PREFIX to reflect the
> additional_registry setting. Or there needs to be a parameter to set this
> value (registry/logging should follow the same setup)."
> 
> Can you not just set the "openshift_hosted_metrics_deployer_prefix" value to
> point to your own registry that you want to use?
> 
> The default value for this is "registry.access.redhat.com/openshift3/" so
> you will probably just want to change the 'registry.access.redhat.com' part.

I'm not seeing such an option in the inventory file? I see "openshift_hosted_logging_deployer_prefix" nothing for metrics.

Comment 10 Matt Wringe 2017-02-21 17:01:01 UTC
(In reply to Peter Larsen from comment #9)
> I'm not seeing such an option in the inventory file? I see
> "openshift_hosted_logging_deployer_prefix" nothing for metrics.

If its not set it will use a default. What happens if you set this value?

Is being able to set this value all you are needing here? Or are you expecting that something else should be happening?

Comment 11 Peter Larsen 2017-02-21 17:59:02 UTC
(In reply to Matt Wringe from comment #10)
> (In reply to Peter Larsen from comment #9)
> > I'm not seeing such an option in the inventory file? I see
> > "openshift_hosted_logging_deployer_prefix" nothing for metrics.
> 
> If its not set it will use a default. What happens if you set this value?
> 
> Is being able to set this value all you are needing here? Or are you
> expecting that something else should be happening?

The default is what value? Right now, when I run the setup, logging and everything else is pulled fine from the local repository. Except metrics; 
This bug is about the missing parameter/variable for metrics, just like we have for logging, so we can do an "offline" install that includes metrics. 
Note - I can pass the IMAGE_PREFIX manually to the deployer template of metrics. I just cannot get that done through the ansible installer. Or at least it's not documented how to make it work.

Comment 12 Matt Wringe 2017-02-21 18:37:34 UTC
See https://bugzilla.redhat.com/show_bug.cgi?id=1417526#c8 for the default

Unless specifying that value does not work for you, this issue needs to be closed and another opened against the ansible documentation.

Comment 13 Matt Wringe 2017-02-22 19:12:57 UTC
"Right now, when I run the setup, logging and everything else is pulled fine from the local repository. Except metrics;"

Are you doing anything special with logging? or are you also setting a prefix type value for it?

Comment 14 Peter Larsen 2017-02-22 20:43:17 UTC
(In reply to Matt Wringe from comment #13)
> "Right now, when I run the setup, logging and everything else is pulled fine
> from the local repository. Except metrics;"
> 
> Are you doing anything special with logging? or are you also setting a
> prefix type value for it?

I think we have a "slight" miscommunication here. I'm trying to install metrics. From the ansible installer. From an offline repository. Because I don't have a metrics "prefix" parameter, the installer does NOT set IMAGE_PREFIX when it starts the metrics deployer, and hence the installation fails as it cannot find the required images.

That's as short and precise I can describe the issue.  I'm not sure if this constitute "something special" but it sure is functionality that's not there.

Comment 15 Matt Wringe 2017-02-22 21:47:58 UTC
I am not sure how much more clear I can be here.

I have asked multiple times if you have set the "openshift_hosted_metrics_deployer_prefix"  like what you have to do with logging (https://bugzilla.redhat.com/show_bug.cgi?id=1417526#c8 https://bugzilla.redhat.com/show_bug.cgi?id=1417526#c12)

In 3.5 there are a lot of changes to the ansible install, and the documentation does outline a value 'openshift_metrics_image_prefix' which can be used to specify the prefix (see https://github.com/openshift/openshift-docs/pull/3765)

Comment 16 Peng Li 2017-03-03 06:29:02 UTC
Set to verified since in 3.5.0 user can set the image prefix in the inventory file like

openshift_metrics_image_prefix=

and run the playbook and it works.

Comment 18 errata-xmlrpc 2017-04-12 19:10:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0884


Note You need to log in before you can comment on or make changes to this bug.