Description of problem: Deploying OSP14 with OSPd https://docs.google.com/document/d/1Z9g9mFViSrmxyA9T7azmt5qnY5POnGCu4wXIQkEAVfE/edit# There is an issue to finish the deployment with OCP 3.11 due an issue with the repositories Your overcloud nodes will require access to the OpenShift repository in order to install OCP packages. For information on how to configure RHSM in your director-based deployment, see https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/advanced_overcloud_customization/ansible-based-registration. To make the OpenShift packages available to your nodes, you will need to add an entry for `rhel-7-server-ose-3.11-rpms` to your `rhsm.yml` file: + ---- resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-7-server-rpms - rhel-7-server-extras-rpms - rhel-7-server-ose-3.11-rpms rhsm_activation_key: "my-openstack" rhsm_org_id: "1234567" rhsm_pool_ids: "1a85f9223e3d5e43013e3d6e8ff506fd" rhsm_method: "portal" ---- or using username/password (undercloud) [stack@undercloud ~]$ cat rhsm.yaml resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-7-server-rpms - rhel-7-server-extras-rpms - rhel-7-server-ose-3.11-rpms rhsm_pool_ids: "8a85f98c63842fef0163949e5f9c4be0" rhsm_method: "portal" rhsm_username: emyemail rhsm_password: password rhsm_autosubscribe: true The error is: CHECK [memory_availability : 172.16.0.43] ************************************** ^[[0;31mfatal: [openshift-worker-0]: FAILED! => {"changed": true, "checks": {"disk_availability": {}, "docker_image_availability": {"changed": true}, "docker_storage": {}, "memory_availability": {"skipped": true, "skipped_reason": "Disabled by user request"}, "package_availability": {"failed": true, "failures": [["OpenShiftCheckException", "Cannot install all of the necessary packages. Unavailable:\n atomic-openshift\n atomic-openshift-node\nYou may need to enable one or more yum repositories to make this content available."]], "invocation": {"module_args": {"packages": ["PyYAML", "atomic-openshift", "atomic-openshift-node", "bind", "ceph-common", "dnsmasq", "docker", "firewalld", "flannel", "glusterfs-fuse", "iptables", "iptables-services", "iscsi-initiator-utils", "libselinux-python", "nfs-utils", "ntp", "openssl", "pyparted", "python-httplib2", "yum-utils"]}}, "last_failed": {"failed": true, "invocation": {"module_args": {"packages": ["PyYAML", "atomic-openshift", "atomic-openshift-node", "bind", "ceph-common", "dnsmasq", "docker", "firewalld", "flannel", "glusterfs-fuse", "iptables", "iptables-services", "iscsi-initiator-utils", "libselinux-python", "nfs-utils", "ntp", "openssl", "pyparted", "python-httplib2", "yum-utils"]}}, "msg": "Cannot install all of the necessary packages. Unavailable:\n atomic-openshift\n atomic-openshift-node\nYou may need to enable one or more yum repositories to make this content available."}, "msg": "Cannot install all of the necessary packages. Unavailable:\n atomic-openshift\n atomic-openshift-node\nYou may need to enable one or more yum repositories to make this content available."}, "package_version": {"failed": true, "failures": [["OpenShiftCheckException", "Not all of the required packages are available at their requested version\natomic-openshift:3.11 \natomic-openshift-master:3.11 \natomic-openshift-node:3.11 \nPlease check your subscriptions and enabled repositories."]], "invocation": {"module_args": {"package_list": [{"check_multi": true, "name": "atomic-openshift", "version": "3.11"}, {"check_multi": true, "name": "atomic-openshift-master", "version": "3.11"}, {"check_multi": true, "name": "atomic-openshift-node", "version": "3.11"}], "package_mgr": "yum"}}, "last_failed": {"failed": true, "invocation": {"module_args": {"package_list": [{"check_multi": true, "name": "atomic-openshift", "version": "3.11"}, {"check_multi": true, "name": "atomic-openshift-master", "version": "3.11"}, {"check_multi": true, "name": "atomic-openshift-node", "version": "3.11"}], "package_mgr": "yum"}}, "msg": "Not all of the required packages are available at their requested version\natomic-openshift:3.11 \natomic-openshift-master:3.11 \natomic-openshift-node:3.11 \nPlease check your subscriptions and enabled repositories."}, "msg": "Not all of the required packages are available at their requested version\natomic-openshift:3.11 \natomic-openshift-master:3.11 \natomic-openshift-node:3.11 \nPlease check your subscriptions and enabled repositories."}}, "msg": "One or more checks failed", "playbook_context": "install"}^[[0m Failure summary: 1. Hosts: openshift-infra-0, openshift-infra-1, openshift-infra-2, openshift-worker-0, openshift-worker-1, openshift-worker-2 Play: OpenShift Health Checks Task: Run health checks (install) - EL Message: ^[[0;31mOne or more checks failed^[[0m Details: ^[[0;31mcheck "package_availability":^[[0m ^[[0;31mCannot install all of the necessary packages. Unavailable:^[[0m ^[[0;31m atomic-openshift^[[0m ^[[0;31m atomic-openshift-node^[[0m ^[[0;31mYou may need to enable one or more yum repositories to make this content available.^[[0m ^[[0;31m^[[0m ^[[0;31mcheck "package_version":^[[0m ^[[0;31mNot all of the required packages are available at their requested version^[[0m ^[[0;31matomic-openshift:3.11 ^[[0m ^[[0;31matomic-openshift-master:3.11 ^[[0m ^[[0;31matomic-openshift-node:3.11 ^[[0m ^[[0;31mPlease check your subscriptions and enabled repositories.^[[0m 2. Hosts: openshift-master-0, openshift-master-1, openshift-master-2 Play: OpenShift Health Checks Task: Run health checks (install) - EL Message: ^[[0;31mOne or more checks failed^[[0m Details: ^[[0;31mcheck "package_availability":^[[0m ^[[0;31mCannot install all of the necessary packages. Unavailable:^[[0m ^[[0;31m atomic-openshift^[[0m ^[[0;31m atomic-openshift-clients^[[0m ^[[0;31m atomic-openshift-hyperkube^[[0m ^[[0;31m atomic-openshift-node^[[0m ^[[0;31mYou may need to enable one or more yum repositories to make this content available.^[[0m ^[[0;31m^[[0m ^[[0;31mcheck "package_version":^[[0m ^[[0;31mNot all of the required packages are available at their requested version^[[0m We can see the repos are configured however it gaves error Nov 27 09:40:00 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-rpms'] Nov 27 09:40:28 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-extras-rpms'] Nov 27 09:40:54 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-ose-3.11-rpms'] Nov 27 09:48:04 openshift-worker-0 ansible-file: Invoked with directory_mode=None force=False remote_src=None _original_basename=None path=/etc/yum.repos.d/openshift_additional.repo owner=None follow=True group=None unsafe_writes=None state=absent content=NOT_LOGGING_PARAMETER serole=None setype=None dest=/etc/yum.repos.d/openshift_additional.repo selevel=None regexp=None src=None seuser=None recurse=False _diff_peek=None delimiter=None mode=None attributes=None backup=None Nov 27 09:39:54 openshift-worker-0 subscription-manager: Removed subscription for product 'Red Hat Gluster Storage Web Administration (for RHEL Server)' Nov 27 09:40:00 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-rpms'] Nov 27 09:40:00 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:40:01 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:40:01,204 [INFO] rhsmd:17131:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:40:01 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:40:12 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:40:12 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:40:12,533 [INFO] rhsmd:17189:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:40:12 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:40:28 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-extras-rpms'] Nov 27 09:40:28 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:40:28 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:40:28,940 [INFO] rhsmd:17265:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:40:28 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:40:39 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:40:39 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:40:39,620 [INFO] rhsmd:17324:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:40:39 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:40:54 openshift-worker-0 ansible-rhsm_repository: Invoked with state=enabled name=['rhel-7-server-ose-3.11-rpms'] Nov 27 09:40:54 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:40:55 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:40:55,002 [INFO] rhsmd:17400:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:40:55 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:41:05 openshift-worker-0 dbus[3095]: [system] Activating service name='com.redhat.SubscriptionManager' (using servicehelper) Nov 27 09:41:05 openshift-worker-0 com.redhat.SubscriptionManager: 2018-11-27 09:41:05,685 [INFO] rhsmd:17458:MainThread @rhsm_d.py:280 - rhsmd started Nov 27 09:41:05 openshift-worker-0 dbus[3095]: [system] Successfully activated service 'com.redhat.SubscriptionManager' Nov 27 09:44:26 openshift-worker-0 ansible-stat: Invoked with checksum_algorithm=sha1 get_checksum=True follow=False path=/var/lib/docker-container-startup-configs.json get_md5=None get_mime=True get_attributes=True Repo rhel-7-server-extras-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Repo rhel-7-server-ose-3.11-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Repo rhel-7-server-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Cleaning repos: rhel-7-server-extras-rpms rhel-7-server-ose-3.11-rpms rhel-7-server-rpms rhelosp-14.0-image-build-override rhelosp-14.0-optools-puddle : rhelosp-14.0-puddle rhelosp-ceph-3.0-mon rhelosp-ceph-3.0-osd rhelosp-ceph-3.0-tools rhelosp-rhel-7.6-extras rhelosp-rhel-7.6-ha : rhelosp-rhel-7.6-image-build-override rhelosp-rhel-7.6-server rhos-release rhos-release-extras Other repos take up 13 k of disk space (use --verbose for details) [root@openshift-worker-0 ~]# grep ose-3.11-rpms /etc/yum.repos.d/redhat.repo -A13 [rhel-7-server-ose-3.11-rpms] metadata_expire = 86400 sslclientcert = /etc/pki/entitlement/1565926590404655490.pem baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/7Server/$basearch/ose/3.11/os ui_repoid_vars = basearch sslverify = 1 name = Red Hat OpenShift Container Platform 3.11 (RPMs) sslclientkey = /etc/pki/entitlement/1565926590404655490-key.pem gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release enabled = 1 sslcacert = /etc/rhsm/ca/chemeredhat-uep.pem gpgcheck = 1 Error that appears is Repo rhel-7-server-extras-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Repo rhel-7-server-ose-3.11-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Repo rhel-7-server-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/chemeredhat-uep.pem Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
The issue is caused by missing the following patch https://review.openstack.org/#/c/617348/ in ansible-role-redhat-subscription. Emilien is working to backport it on downstream for OSP14: After applying it I was able to get the nodes with repos deployed by using: resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-7-server-rpms - rhel-7-server-extras-rpms - rhel-7-server-ose-3.11-rpms rhsm_pool_ids: "8a85f98260c27fc50160c32cc4365ccb" rhsm_method: "portal" rhsm_username: mcornea rhsm_password: password rhsm_autosubscribe: false To apply the patch: curl -4 https://review.openstack.org/changes/617348/revisions/current/patch?download | base64 -d | sudo patch -d /usr/share/ansible/roles/redhat-subscription/ -p1
Deployment works using user and password with Portal method. (undercloud) [stack@undercloud ~]$ cat rhsm.yaml resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-7-server-rpms - rhel-7-server-extras-rpms - rhel-7-server-ose-3.11-rpms rhsm_pool_ids: "8a85f98c63842fef0163949e5f9c4be0" rhsm_method: "portal" rhsm_username: ealcaniz rhsm_password: password rhsm_autosubscribe: true
Deployment doesn't work using Activation key with Portal method (undercloud) [stack@undercloud ~]$ cat rhsmAK.yaml resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_activation_key: "rh-employee-sku" rhsm_org_id: "1979710" rhsm_pool_ids: "8a85f9833e1404a9013e3cddf95a0599" rhsm_method: "portal"
The error using Activation Key (AK) 2. Hosts: openshift-master-0, openshift-master-1, openshift-master-2 Play: OpenShift Health Checks Task: Run health checks (install) - EL Message: One or more checks failed Details: check "package_availability": Cannot install all of the necessary packages. Unavailable: atomic-openshift atomic-openshift-clients atomic-openshift-hyperkube atomic-openshift-node You may need to enable one or more yum repositories to make this content available. check "package_version": Not all of the required packages are available at their requested version atomic-openshift:3.11 atomic-openshift-master:3.11 atomic-openshift-node:3.11 Please check your subscriptions and enabled repositories.
Applying https://review.openstack.org/#/c/620937/1/tasks/portal.yml Deployment works using Activation key with Portal method (undercloud) [stack@undercloud ~]$ cat rhsmAK.yaml resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-7-server-rpms - rhel-7-server-extras-rpms - rhel-7-server-ose-3.11-rpms rhsm_activation_key: "activation-key" rhsm_org_id: "1234567" rhsm_pool_ids: "8a85f9833e1404a6023e4cddf95a0599" rhsm_method: "portal"
(In reply to Edu Alcaniz from comment #10) > Applying https://review.openstack.org/#/c/620937/1/tasks/portal.yml > > Deployment works using Activation key with Portal method > (undercloud) [stack@undercloud ~]$ cat rhsmAK.yaml > resource_registry: > OS::TripleO::Services::Rhsm: > /usr/share/openstack-tripleo-heat-templates/extraconfig/services/rhsm.yaml > parameter_defaults: > RhsmVars: > rhsm_repos: > - rhel-7-server-rpms > - rhel-7-server-extras-rpms > - rhel-7-server-ose-3.11-rpms > rhsm_activation_key: "activation-key" > rhsm_org_id: "1234567" > rhsm_pool_ids: "8a85f9833e1404a6023e4cddf95a0599" > rhsm_method: "portal" I created a new bug to keep track of this issue since this one is already ON_QA - https://bugzilla.redhat.com/show_bug.cgi?id=1654776
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045