Bug 1581876 - [free-int] docker registry hostname not set in master-config
Summary: [free-int] docker registry hostname not set in master-config
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.10.0
Assignee: Scott Dodson
QA Contact: liujia
: 1571608 (view as bug list)
Depends On:
TreeView+ depends on / blocked
Reported: 2018-05-23 20:11 UTC by Justin Pierce
Modified: 2018-07-30 19:16 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Last Closed: 2018-07-30 19:16:18 UTC
Target Upstream Version:

Attachments (Terms of Use)
Pod in ImagePullBackOff (4.53 KB, text/plain)
2018-05-23 20:11 UTC, Justin Pierce
no flags Details

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 None None None 2018-07-30 19:16:38 UTC
Red Hat Bugzilla 1571608 None CLOSED The address for push images should not use ip address of docker-registry after upgrade 2019-05-17 13:17:58 UTC

Internal Links: 1571608

Description Justin Pierce 2018-05-23 20:11:59 UTC
Created attachment 1440749 [details]
Pod in ImagePullBackOff

Description of problem:
Pods began ImagePullBackoff after upgrading to v3.10.0-0.50.0 . Pod was attempting to pull IP based image URL and getting a certificate error. Docker registry hostname was not present in master-config : https://docs.openshift.org/latest/install_config/registry/extended_registry_configuration.html#setting-the-registry-hostname

Version-Release number of selected component (if applicable):

Additional info:
- See attachment for pod listing with error condition. 
- Adding OPENSHIFT_DEFAULT_REGISTRY=docker-registry.default.svc:5000 to master.env allowed pod to start pulling again, but we are speculating this setting should in the master-config & openshift-ansible.

Comment 1 Scott Dodson 2018-05-24 12:46:29 UTC

Where's the most appropriate place to set this?

Previously we've been setting the environment variable in /etc/sysconfig/atomic-openshift-master-{api,controllers} but if a YAML based configuration variable is appropriate it seems easier to maintain it there.

Is this a candidate for devex taking ownership?

Comment 2 Scott Dodson 2018-05-24 13:39:27 UTC
The only reason that this wouldn't have been set is because we didn't sign the certificate with the hostname 'docker-registry.default.svc' prior to 3.7. However in 3.7 and 3.9 upgrades we've been updating the certificate so effectively by the time that they get to 3.10 we should have 100% assurance that the certificate has been signed with hostname so there's no need for conditional logic any longer.

We should default this in the product unless there's a reason not to do so. And if there's a reason not to do so, like `oc cluster up` needs to reconfigure it or something, then we should force it via openshift-ansible.

Comment 4 Ben Parees 2018-05-24 15:51:17 UTC
yeah i think it should be in the master-config and i can't think of a reason not to default it... if we're setting the registry url on the registry DC, we should be setting it in the master configuration also.

As for us taking ownership, sure, but we'll need pointers :)

Comment 5 Scott Dodson 2018-05-24 18:57:27 UTC
Ended up taking care of it, this forces the value during upgrade via master-config.yaml and it makes sure if they've specified openshift_master_image_policy dictionary that we merge in the value we care about.


Comment 6 Vadim Rutkovsky 2018-05-25 09:34:57 UTC
Fix is in openshift-ansible-3.10.0-0.52.0

Comment 7 Scott Dodson 2018-05-25 12:41:32 UTC
*** Bug 1571608 has been marked as a duplicate of this bug. ***

Comment 8 liujia 2018-05-28 11:02:35 UTC
Verified on openshift-ansible-3.10.0-0.53.0.git.0.53fe016.el7.noarch

After upgrade, sti-build still push image with docker registry's DNS address. And docker registry was added into master-config.

# oc get is
NAME                     DOCKER REPO                                                      TAGS      UPDATED
nodejs-mongodb-example   docker-registry.default.svc:5000/mytest/nodejs-mongodb-example             

# cat /etc/origin/master/master-config.yaml|grep -A 1 imagePolicyConfig
  internalRegistryHostname: docker-registry.default.svc:5000

Comment 10 errata-xmlrpc 2018-07-30 19:16:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.