Bug 1516534 - installer oreg_url doesn't serve as universal url - needs to
Summary: installer oreg_url doesn't serve as universal url - needs to
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.10.0
Assignee: Michael Gugino
QA Contact: Johnny Liu
URL:
Whiteboard:
: 1516535 1531949 1558306 1573904 1575994 1576826 1604714 (view as bug list)
Depends On:
Blocks: 1588291 1623602
TreeView+ depends on / blocked
 
Reported: 2017-11-22 20:37 UTC by Erik M Jacobs
Modified: 2018-11-29 12:13 UTC (History)
30 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1588291 1623602 (view as bug list)
Environment:
Last Closed: 2018-08-27 20:52:26 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3538711 None None None 2018-07-22 06:53:30 UTC

Description Erik M Jacobs 2017-11-22 20:37:28 UTC
The installer's hosts/config file takes an argument, oreg_url, that appears like it should set the image format string for all components. It does not.

For example, here's how you have to override/set all of the relevant components, in spite of setting oreg_url:

oreg_url=registry.reg-aws.openshift.com/openshift3/ose-${component}:${version}
openshift_docker_additional_registries=registry.reg-aws.openshift.com
openshift_cockpit_deployer_prefix=registry.reg-aws.openshift.com/openshift3/
# Enable service catalog
openshift_enable_service_catalog=true
openshift_service_catalog_image_prefix=openshift3/ose-
# service broker(s)
# Enable template service broker
template_service_broker_install=true
template_service_broker_prefix=openshift3/ose-
# Enable ansible service broker
ansible_service_broker_install=true
ansible_service_broker_image_prefix=openshift3/ose-

As you can see:

* You have to set "additional registry" for things that you don't give an explicit prefix
* You have to set oreg_url for anything that might happen later that the installer didn't explicitly tell openshift to do
* you have to override many different prefixes either explicitly or partially (to pick up additional registry)

setting oreg_url should automatically override all component installations without needing to additionally dig into component prefixes. It should be optional whether or not to need to override additional prefixes.

Comment 1 Scott Dodson 2018-01-22 17:42:21 UTC
*** Bug 1531949 has been marked as a duplicate of this bug. ***

Comment 2 Nicholas Nachefski 2018-02-10 18:40:31 UTC
In case anyone else needs help with this, here is a comprehensive list of variables that need to be set for disconnected installs to work.  Setting all of these is a work around until the openshift-ansible playbooks get normalized.

These work for 3.7 and 3.9.

openshift_docker_additional_registries=repo.home.nicknach.net:5000
openshift_docker_insecure_registries=repo.home.nicknach.net:5000
openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
oreg_url=repo.home.nicknach.net:5000/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_metrics_image_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_metrics_image_version=v3.9.0
openshift_logging_image_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_logging_image_version=v3.9.0
ansible_service_broker_image_prefix=repo.home.nicknach.net:5000/openshift3/ose-
ansible_service_broker_image_tag=v3.9.0
ansible_service_broker_etcd_image_prefix=repo.home.nicknach.net:5000/rhel7/
ansible_service_broker_etcd_image_tag=latest
openshift_service_catalog_image_prefix=repo.home.nicknach.net:5000/openshift3/ose-
openshift_service_catalog_image_version=v3.9.0
openshift_cockpit_deployer_prefix=repo.home.nicknach.net:5000/openshift3/
template_service_broker_prefix=repo.home.nicknach.net:5000/openshift3/ose-
template_service_broker_version=v3.9.0
openshift_web_console_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_web_console_version=v3.9.0

Comment 3 Johnny Liu 2018-03-20 06:18:15 UTC
Some related github issue:
https://github.com/openshift/openshift-ansible/issues/4415

Comment 4 Johnny Liu 2018-03-20 06:20:02 UTC
*** Bug 1558306 has been marked as a duplicate of this bug. ***

Comment 9 Nicholas Nachefski 2018-03-20 13:45:39 UTC
The playbook is all over the place in it's current state.  In case anyone wants a reference for what works now (in 3.9), here you go...

openshift_docker_additional_registries=repo.home.nicknach.net
openshift_docker_insecure_registries=repo.home.nicknach.net
openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
oreg_url=repo.home.nicknach.net/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_metrics_image_prefix=repo.home.nicknach.net/openshift3/
openshift_metrics_image_version=v3.9.11
openshift_logging_image_prefix=repo.home.nicknach.net/openshift3/
openshift_logging_image_version=v3.9.11
ansible_service_broker_image_prefix=repo.home.nicknach.net/openshift3/ose-
ansible_service_broker_image_tag=v3.9.11
ansible_service_broker_etcd_image_prefix=repo.home.nicknach.net/rhel7/
ansible_service_broker_etcd_image_tag=latest
openshift_service_catalog_image_prefix=repo.home.nicknach.net/openshift3/ose-
openshift_service_catalog_image_version=v3.9.11
openshift_cockpit_deployer_prefix=repo.home.nicknach.net/openshift3/
openshift_web_console_prefix=repo.home.nicknach.net/openshift3/ose-
openshift_web_console_version=v3.9.11
openshift_prometheus_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_image_version=v3.9.11
openshift_prometheus_alertmanager_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_alertmanager_image_version=v3.9.11
openshift_prometheus_alertbuffer_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_alertbuffer_image_version=v3.9.11
openshift_prometheus_node_exporter_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_node_exporter_image_version=v3.9.11
openshift_prometheus_proxy_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_proxy_image_version=v3.9.11
template_service_broker_prefix=repo.home.nicknach.net/openshift3/ose-
template_service_broker_version=v3.9.11
openshift_storage_glusterfs_image=repo.home.nicknach.net/rhgs3/rhgs-server-rhel7
openshift_storage_glusterfs_version=latest
openshift_storage_glusterfs_heketi_image=repo.home.nicknach.net/rhgs3/rhgs-volmanager-rhel7
openshift_storage_glusterfs_heketi_version=latest

Comment 11 Clayton Coleman 2018-03-26 03:31:59 UTC
https://github.com/openshift/openshift-ansible/pull/7645 is a first step towards enforcing a consistent internal pattern based on oreg_url, other work will complete this by publishing images with consistent names.

Comment 12 Scott Dodson 2018-05-02 14:55:14 UTC
*** Bug 1573904 has been marked as a duplicate of this bug. ***

Comment 13 Scott Dodson 2018-05-08 13:56:41 UTC
*** Bug 1575994 has been marked as a duplicate of this bug. ***

Comment 14 Scott Dodson 2018-05-10 14:06:36 UTC
*** Bug 1576826 has been marked as a duplicate of this bug. ***

Comment 19 Johnny Liu 2018-05-29 10:41:44 UTC
Seem like the PR already merged into openshift-ansible-3.10.0-0.53.0, so change target release.

Comment 24 Michael Gugino 2018-05-30 13:56:57 UTC
PR for checking for removed variables: https://github.com/openshift/openshift-ansible/pull/8568

etcd oreg_url support: https://github.com/openshift/openshift-ansible/pull/8564

Comment 25 Nicholas Nachefski 2018-05-31 03:44:23 UTC
As of 3.10.0-0.54.0, i still needed these vars (along with oreg_url) to get a successful install.

openshift_storage_glusterfs_image=repo.home.nicknach.net/rhgs3/rhgs-server-rhel7
openshift_storage_glusterfs_version=v3.10.0
openshift_storage_glusterfs_heketi_image=repo.home.nicknach.net/rhgs3/rhgs-volmanager-rhel7
openshift_storage_glusterfs_heketi_version=v3.10.0
openshift_storage_glusterfs_block_image=repo.home.nicknach.net/rhgs3/rhgs-gluster-block-prov-rhel7
osm_etcd_image=repo.home.nicknach.net/rhel7/etcd

Looks like just etcd and CNS stuff needs fixing.

Comment 26 Scott Dodson 2018-05-31 13:12:49 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1581622 can likely be resolved by addressing comment 25 as well.

Comment 27 Michael Gugino 2018-05-31 17:30:57 UTC
Glusterfs PR: https://github.com/openshift/openshift-ansible/pull/8585

Comment 28 Nicholas Nachefski 2018-05-31 17:32:57 UTC
In addition to my previous comment, the registry web console appears to also not be using oreg_url.

Warning   Failed                        kubelet, node01.ocp.nicknach.net     Failed to pull image "registry.access.redhat.com/openshift3/registry-console:v3.10": rpc error: code = Unknown desc = All endpoints blocked.


I couldnt even get it to work the old way, by setting these:

openshift_web_console_prefix=repo.home.nicknach.net/openshift3/
openshift_web_console_version=v3.10.0

So etcd, CNS, and registry-console seems to not be respecting oreg_url in 3.10.0-0.54.0.

Comment 29 Scott Dodson 2018-05-31 19:06:48 UTC
The fixes for etcd and openshift-web-console weren't in 3.10.0-0.54.0. They'll be in the next build tonight/tomorrow.

The only unmerged fix is GlusterFS right now and that should merge today.

Comment 30 Nicholas Nachefski 2018-06-01 14:51:41 UTC
Looks good now.

I got a clean (disconnected) install of 3.10.0-0.57.0 using only the oreg_url variable.  This will make it alot easier to do these kinds of deployments.  Thank you very much!

-Nick

Comment 31 Scott Dodson 2018-06-01 14:58:10 UTC
In 3.10.0-0.57.0 and later.

Comment 33 Scott Dodson 2018-06-04 12:12:48 UTC
https://github.com/openshift/openshift-ansible/pull/8616 reverts a recent change that should cause the problem you're describing.

Comment 34 Johnny Liu 2018-06-05 07:03:28 UTC
(In reply to Scott Dodson from comment #33)
> https://github.com/openshift/openshift-ansible/pull/8616 reverts a recent
> change that should cause the problem you're describing.

No, they are different issue, PR#8616 would fix BZ#1584966, but in this bug we are tracking oreg_url should be used in registry-console, no need to set any separate options (for registry console, the option name is "openshift_cockpit_deployer_prefix"), PR#8616 would not fix such issue, registry-console would still need specify openshift_cockpit_deployer_prefix to your local registry.

After go through the openshift-ansible code, regsitry-console would be deployed from *registry-console template* installed by openshift-ansible, openshift_cockpit_deployer_prefix is used to overwrite the default registry.access.redhat.com when running "oc new-app --template=registry-console" command, for such template or ImageStream template, installer also have another option - openshift_examples_modify_imagestreams to modify the default registry to oreg_url registry without need openshift_cockpit_deployer_prefix setting. 

So registry-console is not in the scope of this bug.

Comment 35 Johnny Liu 2018-06-05 07:04:29 UTC
Based on comment 32, move this bug to verified.

Comment 38 Scott Dodson 2018-06-21 15:09:38 UTC
*** Bug 1516535 has been marked as a duplicate of this bug. ***

Comment 40 Johnny Liu 2018-07-10 11:32:20 UTC
Copied comments from https://bugzilla.redhat.com/show_bug.cgi?id=1588291#c6
> The following variables need to be kept
> openshift_cockpit_deployer_prefix
> openshift_cockpit_deployer_version
I think I already explained the reason of the two variables should be kept in https://bugzilla.redhat.com/show_bug.cgi?id=1516534#c34.

> openshift_provisioners_image_prefix
> openshift_provisioners_image_version
This is my first time to see this two vars, have some search in openshift-ansible code, seem like they are related to storage component, I am not familiar with that component. @jhou do you know that?

BTW, this bug is tracking too much components and vars, if  openshift_provisioners_image_* could be confirmed to a real issue, it is better to create another bug to track.

Comment 41 Kenjiro Nakayama 2018-07-22 06:53:30 UTC
*** Bug 1604714 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.