Bug 1516534
| Summary: | installer oreg_url doesn't serve as universal url - needs to | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Erik M Jacobs <ejacobs> | |
| Component: | Installer | Assignee: | Michael Gugino <mgugino> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Johnny Liu <jialiu> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | urgent | |||
| Version: | 3.7.0 | CC: | aos-bugs, bbilgin, bkozdemb, bleanhar, ccoleman, cevich, cshereme, dmoessne, dominik.mierzejewski, farandac, jhou, jialiu, jokerman, judd, knakayam, lmeyer, mgugino, mmccomas, mtaru, nick, pdwyer, sauchter, schoudha, sdodson, tcarlin, vigoyal, vlaad, weshi, wmeng, yapei | |
| Target Milestone: | --- | |||
| Target Release: | 3.10.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1588291 1623602 (view as bug list) | Environment: | ||
| Last Closed: | 2018-08-27 20:52:26 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1588291, 1623602 | |||
*** Bug 1531949 has been marked as a duplicate of this bug. *** In case anyone else needs help with this, here is a comprehensive list of variables that need to be set for disconnected installs to work. Setting all of these is a work around until the openshift-ansible playbooks get normalized.
These work for 3.7 and 3.9.
openshift_docker_additional_registries=repo.home.nicknach.net:5000
openshift_docker_insecure_registries=repo.home.nicknach.net:5000
openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
oreg_url=repo.home.nicknach.net:5000/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_metrics_image_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_metrics_image_version=v3.9.0
openshift_logging_image_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_logging_image_version=v3.9.0
ansible_service_broker_image_prefix=repo.home.nicknach.net:5000/openshift3/ose-
ansible_service_broker_image_tag=v3.9.0
ansible_service_broker_etcd_image_prefix=repo.home.nicknach.net:5000/rhel7/
ansible_service_broker_etcd_image_tag=latest
openshift_service_catalog_image_prefix=repo.home.nicknach.net:5000/openshift3/ose-
openshift_service_catalog_image_version=v3.9.0
openshift_cockpit_deployer_prefix=repo.home.nicknach.net:5000/openshift3/
template_service_broker_prefix=repo.home.nicknach.net:5000/openshift3/ose-
template_service_broker_version=v3.9.0
openshift_web_console_prefix=repo.home.nicknach.net:5000/openshift3/
openshift_web_console_version=v3.9.0
Some related github issue: https://github.com/openshift/openshift-ansible/issues/4415 *** Bug 1558306 has been marked as a duplicate of this bug. *** The playbook is all over the place in it's current state. In case anyone wants a reference for what works now (in 3.9), here you go...
openshift_docker_additional_registries=repo.home.nicknach.net
openshift_docker_insecure_registries=repo.home.nicknach.net
openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
oreg_url=repo.home.nicknach.net/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_metrics_image_prefix=repo.home.nicknach.net/openshift3/
openshift_metrics_image_version=v3.9.11
openshift_logging_image_prefix=repo.home.nicknach.net/openshift3/
openshift_logging_image_version=v3.9.11
ansible_service_broker_image_prefix=repo.home.nicknach.net/openshift3/ose-
ansible_service_broker_image_tag=v3.9.11
ansible_service_broker_etcd_image_prefix=repo.home.nicknach.net/rhel7/
ansible_service_broker_etcd_image_tag=latest
openshift_service_catalog_image_prefix=repo.home.nicknach.net/openshift3/ose-
openshift_service_catalog_image_version=v3.9.11
openshift_cockpit_deployer_prefix=repo.home.nicknach.net/openshift3/
openshift_web_console_prefix=repo.home.nicknach.net/openshift3/ose-
openshift_web_console_version=v3.9.11
openshift_prometheus_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_image_version=v3.9.11
openshift_prometheus_alertmanager_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_alertmanager_image_version=v3.9.11
openshift_prometheus_alertbuffer_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_alertbuffer_image_version=v3.9.11
openshift_prometheus_node_exporter_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_node_exporter_image_version=v3.9.11
openshift_prometheus_proxy_image_prefix=repo.home.nicknach.net/openshift3/
openshift_prometheus_proxy_image_version=v3.9.11
template_service_broker_prefix=repo.home.nicknach.net/openshift3/ose-
template_service_broker_version=v3.9.11
openshift_storage_glusterfs_image=repo.home.nicknach.net/rhgs3/rhgs-server-rhel7
openshift_storage_glusterfs_version=latest
openshift_storage_glusterfs_heketi_image=repo.home.nicknach.net/rhgs3/rhgs-volmanager-rhel7
openshift_storage_glusterfs_heketi_version=latest
https://github.com/openshift/openshift-ansible/pull/7645 is a first step towards enforcing a consistent internal pattern based on oreg_url, other work will complete this by publishing images with consistent names. *** Bug 1573904 has been marked as a duplicate of this bug. *** *** Bug 1575994 has been marked as a duplicate of this bug. *** *** Bug 1576826 has been marked as a duplicate of this bug. *** In progress PRs: https://github.com/openshift/openshift-ansible/pull/8479 https://github.com/openshift/openshift-ansible/pull/8489 Seem like the PR already merged into openshift-ansible-3.10.0-0.53.0, so change target release. PR for checking for removed variables: https://github.com/openshift/openshift-ansible/pull/8568 etcd oreg_url support: https://github.com/openshift/openshift-ansible/pull/8564 As of 3.10.0-0.54.0, i still needed these vars (along with oreg_url) to get a successful install. openshift_storage_glusterfs_image=repo.home.nicknach.net/rhgs3/rhgs-server-rhel7 openshift_storage_glusterfs_version=v3.10.0 openshift_storage_glusterfs_heketi_image=repo.home.nicknach.net/rhgs3/rhgs-volmanager-rhel7 openshift_storage_glusterfs_heketi_version=v3.10.0 openshift_storage_glusterfs_block_image=repo.home.nicknach.net/rhgs3/rhgs-gluster-block-prov-rhel7 osm_etcd_image=repo.home.nicknach.net/rhel7/etcd Looks like just etcd and CNS stuff needs fixing. https://bugzilla.redhat.com/show_bug.cgi?id=1581622 can likely be resolved by addressing comment 25 as well. Glusterfs PR: https://github.com/openshift/openshift-ansible/pull/8585 In addition to my previous comment, the registry web console appears to also not be using oreg_url. Warning Failed kubelet, node01.ocp.nicknach.net Failed to pull image "registry.access.redhat.com/openshift3/registry-console:v3.10": rpc error: code = Unknown desc = All endpoints blocked. I couldnt even get it to work the old way, by setting these: openshift_web_console_prefix=repo.home.nicknach.net/openshift3/ openshift_web_console_version=v3.10.0 So etcd, CNS, and registry-console seems to not be respecting oreg_url in 3.10.0-0.54.0. The fixes for etcd and openshift-web-console weren't in 3.10.0-0.54.0. They'll be in the next build tonight/tomorrow. The only unmerged fix is GlusterFS right now and that should merge today. Looks good now. I got a clean (disconnected) install of 3.10.0-0.57.0 using only the oreg_url variable. This will make it alot easier to do these kinds of deployments. Thank you very much! -Nick In 3.10.0-0.57.0 and later. https://github.com/openshift/openshift-ansible/pull/8616 reverts a recent change that should cause the problem you're describing. (In reply to Scott Dodson from comment #33) > https://github.com/openshift/openshift-ansible/pull/8616 reverts a recent > change that should cause the problem you're describing. No, they are different issue, PR#8616 would fix BZ#1584966, but in this bug we are tracking oreg_url should be used in registry-console, no need to set any separate options (for registry console, the option name is "openshift_cockpit_deployer_prefix"), PR#8616 would not fix such issue, registry-console would still need specify openshift_cockpit_deployer_prefix to your local registry. After go through the openshift-ansible code, regsitry-console would be deployed from *registry-console template* installed by openshift-ansible, openshift_cockpit_deployer_prefix is used to overwrite the default registry.access.redhat.com when running "oc new-app --template=registry-console" command, for such template or ImageStream template, installer also have another option - openshift_examples_modify_imagestreams to modify the default registry to oreg_url registry without need openshift_cockpit_deployer_prefix setting. So registry-console is not in the scope of this bug. Based on comment 32, move this bug to verified. *** Bug 1516535 has been marked as a duplicate of this bug. *** Copied comments from https://bugzilla.redhat.com/show_bug.cgi?id=1588291#c6 > The following variables need to be kept > openshift_cockpit_deployer_prefix > openshift_cockpit_deployer_version I think I already explained the reason of the two variables should be kept in https://bugzilla.redhat.com/show_bug.cgi?id=1516534#c34. > openshift_provisioners_image_prefix > openshift_provisioners_image_version This is my first time to see this two vars, have some search in openshift-ansible code, seem like they are related to storage component, I am not familiar with that component. @jhou do you know that? BTW, this bug is tracking too much components and vars, if openshift_provisioners_image_* could be confirmed to a real issue, it is better to create another bug to track. *** Bug 1604714 has been marked as a duplicate of this bug. *** |
The installer's hosts/config file takes an argument, oreg_url, that appears like it should set the image format string for all components. It does not. For example, here's how you have to override/set all of the relevant components, in spite of setting oreg_url: oreg_url=registry.reg-aws.openshift.com/openshift3/ose-${component}:${version} openshift_docker_additional_registries=registry.reg-aws.openshift.com openshift_cockpit_deployer_prefix=registry.reg-aws.openshift.com/openshift3/ # Enable service catalog openshift_enable_service_catalog=true openshift_service_catalog_image_prefix=openshift3/ose- # service broker(s) # Enable template service broker template_service_broker_install=true template_service_broker_prefix=openshift3/ose- # Enable ansible service broker ansible_service_broker_install=true ansible_service_broker_image_prefix=openshift3/ose- As you can see: * You have to set "additional registry" for things that you don't give an explicit prefix * You have to set oreg_url for anything that might happen later that the installer didn't explicitly tell openshift to do * you have to override many different prefixes either explicitly or partially (to pick up additional registry) setting oreg_url should automatically override all component installations without needing to additionally dig into component prefixes. It should be optional whether or not to need to override additional prefixes.