Bug 1420292

Summary: RHEL registration fails frequently for compute nodes
Product: Red Hat OpenStack Reporter: Wolfram Richter <wrichter>
Component: openstack-tripleo-heat-templatesAssignee: Emilien Macchi <emacchi>
Status: CLOSED NOTABUG QA Contact: Arik Chernetsky <achernet>
Severity: medium Docs Contact:
Priority: medium    
Version: 10.0 (Newton)CC: aschultz, jcoufal, mburns, rhel-osp-director-maint, vcojot, wrichter
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-10-20 19:50:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Wolfram Richter 2017-02-08 11:31:12 UTC
Description of problem:
RHOSP10 rollout fails randomly due to subscription failing in compute nodes

Version-Release number of selected component (if applicable):
RHOSP10
Satellite 6.2.6

How reproducible:
>50%, affected are only compute nodes. Since I have >1 node, the actual probability to fail is pretty high (but I did have successful rollouts as well).

Steps to Reproduce:
1. Prepare rhel-registration-environment.yaml heat templates to subscribe against satellite using an activation key
2. deploy overcloud with template
3.

Actual results:
heat stack creation fails. This is caused by the step RHELRegistrationDeployment failing for a compute node. On the node, os-collect-config logs show that the katello-agent cannot be installed and a subsequent katello-package-upload fails:

Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/gofer-2.7.6-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/katello-agent-2.5.0-5.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/pulp-rpm-handlers-2.8.7.5-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-gofer-2.7.6-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-gofer-proton-2.7.6-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-isodate-0.5.0-4.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-pulp-agent-lib-2.8.7.3-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-pulp-common-2.8.7.3-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-pulp-rpm-common-2.8.7.5-1.el7sat.noarch.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/python-qpid-proton-0.9-16.el7.x86_64.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: https://satellite.hailstorm1.coe.muc.redhat.com/pulp/repos/Default_Organization/development/CV-RHOSP/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/qpid-proton-c-0.9-16.el7.x86_64.rpm: [Errno 14] curl#56 - "NSS: client certificate not found: /etc/pki/entitlement/8454544229739234265.pem"
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Trying other mirror.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: Error downloading packages:
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-pulp-rpm-common-2.8.7.5-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: gofer-2.7.6-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-gofer-2.7.6-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-pulp-common-2.8.7.3-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: qpid-proton-c-0.9-16.el7.x86_64: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-gofer-proton-2.7.6-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-pulp-agent-lib-2.8.7.3-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: katello-agent-2.5.0-5.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-isodate-0.5.0-4.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: pulp-rpm-handlers-2.8.7.5-1.el7sat.noarch: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: python-qpid-proton-0.9-16.el7.x86_64: [Errno 256] No more mirrors to try.
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: /var/lib/heat-config/heat-config-script/8cf81fcf-b369-44a7-a5a3-308f2c423330: line 130: katello-package-upload: command not found
Feb 07 19:26:01 overcloud-compute-1.localdomain os-collect-config[7457]: [2017-02-07 19:26:01,396] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/8cf81fcf-b369-44a7-a5a3-308f2c423330. [127]




In the /var/log/rhsm/rhsm.log you can see that the certificate in question has been replaced:

2017-02-07 19:25:51,012 [INFO] rhsmcertd-worker:17813:MainThread @connection.py:830 - Connection built: host=satellite.hailstorm1.coe.muc.redhat.com port=443 handler=/rhsm auth=identity_cert ca_dir=/etc/rhsm/ca/ verify=False
2017-02-07 19:25:51,453 [INFO] rhsmcertd-worker:17813:MainThread @entcertlib.py:131 - certs updated:
Total updates: 3
Found (local) serial# [8454544229739234265L]
Expected (UEP) serial# [8058166644368898371, 8394172207414916351]
Added (new)
  [sn:8058166644368898371 (Red Hat Enterprise Virtualization for IBM Power,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Gluster Storage Server for On-premise,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Gluster Storage Management Console (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Storage Console,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux for Power, big endian - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Ceph Storage Calamari,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Gluster Storage Nagios Server,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat OpenStack Beta,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise MRG Messaging,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Beta,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Resilient Storage (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Developer Toolset (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Scalable File System (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Server - Extended Life Cycle Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat OpenStack,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Load Balancer (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Software Test Suite 5 (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Ceph Storage,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Virtualization,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux High Availability (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux for SAP Hana,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat JBoss Core Services,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (JBoss Enterprise Application Platform,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Server - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat CloudForms Beta,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (RHEL for SAP HANA - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux for SAP,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Atomic Host,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Server,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux for Power, big endian,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Resilient Storage (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Storage Console Node,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Software Collections (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Ceph Storage MON,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux High Availability (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Certification (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Scalable File System (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Enterprise Linux Load Balancer (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Software Collections Beta (for RHEL Server),) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat QuickStart Cloud Installer,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat CloudForms,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8058166644368898371 (Red Hat Ceph Storage OSD,) @ /etc/pki/entitlement/8058166644368898371.pem]
  [sn:8394172207414916351 (Oracle Java (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Server - Extended Update Support,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Ceph Storage Calamari,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat OpenStack Beta,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Server,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Atomic Host,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise MRG Messaging,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Oracle Java (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (dotNET on RHEL Beta (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Beta,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Software Collections (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Ceph Storage MON,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (dotNET on RHEL (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux High Availability (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Atomic Host Beta,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Load Balancer (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Software Collections Beta (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat OpenStack,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux Load Balancer (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Ceph Storage,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat CloudForms,) @ /etc/pki/entitlement/8394172207414916351.pem]
  [sn:8394172207414916351 (Red Hat Enterprise Linux High Availability (for RHEL Server),) @ /etc/pki/entitlement/8394172207414916351.pem]
Deleted (rogue):
  [sn:8454544229739234265 (Oracle Java (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Server - Extended Update Support,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Ceph Storage Calamari,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat OpenStack Beta,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Server,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Atomic Host,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise MRG Messaging,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Oracle Java (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (dotNET on RHEL Beta (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Beta,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Software Collections (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Ceph Storage MON,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (dotNET on RHEL (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux High Availability (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Atomic Host Beta,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Load Balancer (for RHEL Server) - Extended Update Support,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Software Collections Beta (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat OpenStack,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux Load Balancer (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Ceph Storage,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat CloudForms,) @ /etc/pki/entitlement/8454544229739234265.pem]
  [sn:8454544229739234265 (Red Hat Enterprise Linux High Availability (for RHEL Server),) @ /etc/pki/entitlement/8454544229739234265.pem]
2017-02-07 19:25:51,678 [INFO] rhsmcertd-worker:17813:MainThread @repolib.py:309 - repos updated: Repo updates


Expected results:
stack creation completes successfully


Additional info:
I can provide access to the systems in question from within the Red Hat VPN

Comment 1 Wolfram Richter 2017-02-08 12:39:56 UTC
The setup is driven by through ansible - with RHOSP9 in the otherwise same environment, the registration process works fine.

Comment 2 Wolfram Richter 2017-02-09 11:00:36 UTC
I was now successfully with three consecutive rollouts with OSP10 based on the following change to the rhea-registration script

[stack@director ~]$ diff -u  /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/scripts/rhel-registration templates/rhel-registration.sh
--- /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/scripts/rhel-registration	2016-11-15 05:30:38.000000000 +0100
+++ templates/rhel-registration.sh	2017-02-09 11:15:58.759332000 +0100
@@ -123,9 +123,22 @@
         if [ "$satellite_version" = "6" ]; then
             repos="$repos --enable ${satellite_repo}"
             curl -L -k -O "$REG_SAT_URL/pub/katello-ca-consumer-latest.noarch.rpm"
+
+            # satellite 6.2.5 needs to be able to resolve the hostname (overcloud-....localdomain)
+            # for subscription to succeed https://bugzilla.redhat.com/show_bug.cgi?id=1405614
+            # however setting the value in /etc/hosts messes up the subsequent steps, hence we remove the entry
+            echo "$(ip addr show dev eth0 | awk '/inet /{ print $2 }' | cut -d'/' -f1) $(hostname) $(hostname -s)" >>/etc/hosts
             rpm -Uvh katello-ca-consumer-latest.noarch.rpm || true
+            sed -i s/^$(ip addr show dev eth0 | awk '/inet /{ print $2 }' | cut -d'/' -f1).*\$// /etc/hosts
+
             subscription-manager register $opts
             subscription-manager $repos
+
+            # this is an attempt to address https://bugzilla.redhat.com/show_bug.cgi?id=1420292
+            # based on https://access.redhat.com/solutions/1364513
+            yum clean all
+            yum repolist all
+
             yum install -y katello-agent || true # needed for errata reporting to satellite6
             katello-package-upload
             subscription-manager repos --disable ${satellite_repo}
[stack@director ~]$

Comment 3 Vincent S. Cojot 2017-02-28 23:05:18 UTC
Seems code to do this was accepted upstream:
openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=3243a1ab275648a17c98f47735a4490b59478d1e

Comment 4 Jaromir Coufal 2017-03-13 06:17:44 UTC
Vincent, I cannot open the link, could you please update and based on when it was merged also update target release downstream? Thanks a lot!

Comment 8 Wolfram Richter 2017-11-01 10:37:33 UTC
So far I haven't been able to reproduce this with Satellite and OSP11