Bug 1558788 - FFU: provide a way to switch repositories during FFU when nodes are registered to a satellite server/CDN
Summary: FFU: provide a way to switch repositories during FFU when nodes are registere...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 13.0 (Queens)
Assignee: Lukas Bezdicka
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-21 01:54 UTC by Marius Cornea
Modified: 2018-09-10 11:42 UTC (History)
11 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.2-17.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:48:15 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2018:2086 None None None 2018-06-27 13:48:48 UTC

Description Marius Cornea 2018-03-21 01:54:00 UTC
Description of problem:
We need to provide a way to switch repositories during FFU when nodes are registered to a satellite server/CDN. Currently for testing with rhos-release we are using a custom script inside an environment file. For the product we probably need to create similar environment file which includes the correct subscription-manager commands and include it by default as a downstream only change.

This is the script that I am currently using for testing

parameter_defaults:
  FastForwardRepoType: custom-script
  FastForwardCustomRepoScriptContent: |
    set -e
    case $1 in
      ocata)
        rhos-release 11
        ;;
      pike)
        rhos-release 12
        ;;
      queens)
        rhos-release 13
        rm -rf /etc/yum.repos.d/rhos-release-ceph*
        ;;
      *)
        echo "unknown release $1" >&2
        exit 1
    esac

Comment 3 Randy Rubins 2018-04-30 12:04:06 UTC
During FFWD testing, I successfully used the following ffu_repos.yaml file with Satellite registered overcloud nodes:

parameter_defaults:
  FastForwardRepoType: custom-script
  FastForwardCustomRepoScriptContent: |
    set -e
    case $1 in
      ocata)
        subscription-manager register --org ACME --activationkey ak_rhosp11 --force
        ;;
      pike)
        subscription-manager register --org ACME --activationkey ak_rhosp12 --force
        ;;
      queens)
        subscription-manager register --org ACME --activationkey ak_rhosp13_ffu --force
        ;;
      *)
        echo "unknown release $1" >&2
        exit 1
    esac

NOTE: I had to use a different activation key for osp13 during FFWD (ak_rhosp13_ffu) due the need for OSP13 + CEPH2 vs. OSP13 + CEPH3 (ak_rhosp13) which would be used later in the process during the Ceph upgrade.

Comment 4 Lukas Bezdicka 2018-05-04 15:27:59 UTC
I think I'll switch default to:

parameter_defaults:
  FastForwardRepoType: custom-script
  FastForwardCustomRepoScriptContent: |
    set -e
    case $1 in
      ocata)
        subscription-manager repos --disable=rhel-7-server-openstack-10-rpms
        subscription-manager repos --enable=rhel-7-server-openstack-11-rpms
        ;;
      pike)
        subscription-manager repos --disable=rhel-7-server-openstack-11-rpms
        subscription-manager repos --enable=rhel-7-server-openstack-12-rpms
        ;;
      queens)
        subscription-manager repos --disable=rhel-7-server-openstack-12-rpms
        subscription-manager repos --enable=rhel-7-server-openstack-13-rpms
        ;;
      *)
        echo "unknown release $1" >&2
        exit 1
    esac

This way ceph shouldn't be switched. Ceph repo should be provided in UpgradeInit command for ceph-upgrade.

Comment 9 Marius Cornea 2018-06-12 01:47:48 UTC
It's not clear what should we do with the environment files used for registration during the initial deployment. Assuming the initial deploy was something like:

openstack overcloud deploy \
[..]
-e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml \
-e /home/stack/virt/environment-rhel-registration.yaml
 
where /home/stack/virt/environment-rhel-registration.yaml:

(undercloud) [stack@undercloud-0 ~]$ cat /home/stack/virt/environment-rhel-registration.yaml
# Note this can be specified either in the call
# to heat stack-create via an additional -e option
# or via the global environment on the seed in
# /etc/heat/environment.d/default.yaml
parameter_defaults:
  rhel_reg_activation_key: "rhosp10"
  rhel_reg_auto_attach: ""
  rhel_reg_base_url: "https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos"
  rhel_reg_environment: ""
  rhel_reg_force: "True"
  rhel_reg_machine_name: ""
  rhel_reg_org: "Default_Organization"
  rhel_reg_password: ""
  rhel_reg_pool_id: ""
  rhel_reg_release: ""
  rhel_reg_repos: "rhel-7-server-rpms,rhel-7-server-extras-rpms,rhel-7-server-rh-common-rpms,rhel-ha-for-rhel-7-server-rpms,rhel-7-server-openstack-10-rpms,rhel-7-server-rhceph-2-osd-rpms,rhel-7-server-rhceph-2-mon-rpms,rhel-7-server-rhceph-2-tools-rpms"
  rhel_reg_sat_url: "https://rhos-compute-node-08.lab.eng.rdu2.redhat.com"
  rhel_reg_server_url: ""
  rhel_reg_service_level: ""
  rhel_reg_user: ""
  rhel_reg_type: ""
  rhel_reg_method: "satellite"
  rhel_reg_sat_repo: "rhel-7-server-satellite-tools-6.2-rpms"
  rhel_reg_http_proxy_host: ""
  rhel_reg_http_proxy_port: ""
  rhel_reg_http_proxy_username: ""
  rhel_reg_http_proxy_password: ""


How should the 'openstack overcloud ffwd-upgrade prepare' command look like? 

1/ Do we keep the registration environment files used during initial deployment? with something like

openstack overcloud ffwd-upgrade prepare \
[..]
-e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml \
-e /home/stack/virt/environment-rhel-registration.yaml \
-e /home/stack/ffu_repos.yaml

or 

2/ Remove the registration environment files used during initial deployment and pass only the environment containing the FastForwardCustomRepoScriptContent used for ffu:

openstack overcloud ffwd-upgrade prepare \
[..]
-e /home/stack/ffu_repos.yaml

where /home/stack/ffu_repos.yaml is something like:


(undercloud) [stack@undercloud-0 ~]$ cat /home/stack/ffu_repos.yaml 
parameter_defaults:
  FastForwardRepoType: custom-script
  FastForwardCustomRepoScriptContent: |
    set -e
    case $1 in
      ocata)
        subscription-manager register --baseurl=https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --force --activationkey=rhosp11 --org=Default_Organization
        ;;
      pike)
        subscription-manager register --baseurl=https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --force --activationkey=rhosp12 --org=Default_Organization
        ;;
      queens)
        subscription-manager register --baseurl=https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --force --activationkey=rhosp13 --org=Default_Organization
        ;;
      *)
        echo "unknown release $1" >&2
        exit 1
    esac

Comment 11 Marius Cornea 2018-06-18 17:14:28 UTC
If I keep the satellite environment files(-e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml, -e /home/stack/virt/environment-rhel-registration.yaml) then openstack overcloud ffwd-upgrade prepare fails with: 


2018-06-18 16:43:37Z [overcloud]: UPDATE_COMPLETE  Stack UPDATE completed successfully

 Stack overcloud UPDATE_COMPLETE 

Started Mistral Workflow tripleo.package_update.v1.get_config. Execution ID: 54c22ed5-7fbf-45c8-bdec-e4113232ceb3
Waiting for messages on queue 'tripleo' with no timeout.
{u'execution': {u'created_at': u'2018-06-18 16:43:43',
                u'id': u'54c22ed5-7fbf-45c8-bdec-e4113232ceb3',
                u'input': {u'container': u'overcloud',
                           u'queue_name': u'tripleo'},
                u'name': u'tripleo.package_update.v1.get_config',
                u'params': {u'namespace': u''},
                u'spec': {u'input': [u'container',
                                     {u'queue_name': u'tripleo'}],
                          u'name': u'get_config',
                          u'tags': [u'tripleo-common-managed'],
                          u'tasks': {u'get_config': {u'action': u'tripleo.config.get_overcloud_config container=<% $.container %>',
                                                     u'name': u'get_config',
                                                     u'on-complete': u'send_message',
                                                     u'publish': {u'message': u'<% task().result %>',
                                                                  u'status': u'SUCCESS'},
                                                     u'publish-on-error': {u'message': u'Init Minor update failed',
                                                                           u'status': u'FAILED'},
                                                     u'type': u'direct',
                                                     u'version': u'2.0'},
                                     u'send_message': {u'action': u'zaqar.queue_post',
                                                       u'input': {u'messages': {u'body': {u'payload': {u'execution': u'<% execution() %>',
                                                                                                       u'message': u"<% $.get('message', '') %>",
                                                                                                       u'status': u"<% $.get('status', 'SUCCESS') %>"},
                                                                                          u'type': u'tripleo.package_update.v1.package_update_plan'}},
                                                                  u'queue_name': u'<% $.queue_name %>'},
                                                       u'name': u'send_message',
                                                       u'on-success': [{u'fail': u'<% $.get(\'status\') = "FAILED" %>'}],
                                                       u'type': u'direct',
                                                       u'version': u'2.0'}},
                          u'version': u'2.0'},
                u'updated_at': u'2018-06-18 16:43:43'},
 u'message': u'Init Minor update failed',
 u'status': u'FAILED'}
 


openstack overcloud ffwd-upgrade prepare worked when removing the satellite environment files used for initial deployment:

openstack overcloud ffwd-upgrade prepare \
--timeout 100 \
--templates /usr/share/openstack-tripleo-heat-templates \
--stack overcloud \
-e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \
-e /home/stack/virt/internal.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/debug.yaml \
-e /home/stack/overcloud_images.yaml \
-e /home/stack/ffu_repos.yaml \
-e /home/stack/cli_opts.yaml

Comment 12 Marius Cornea 2018-06-20 03:07:03 UTC
I was able to successfully upgrade an environment with nodes registered to Satellite with the following notes:

1/ when building the openstack overcloud ffwd-upgrade prepare and subsequent upgrade commands remove the environment files used for initial registration, in my case:

-e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml, -e /home/stack/virt/environment-rhel-registration.yaml

2/ pass an environment file containing the satellite details, e.g:

parameter_defaults:
  FastForwardRepoType: custom-script
  FastForwardCustomRepoScriptContent: |
    set -e
    case $1 in
      ocata)
        subscription-manager register --baseurl=https://$url --force --activationkey=rhosp11 --org=Default_Organization
        ;;
      pike)
        subscription-manager register --baseurl=https://$url --force --activationkey=rhosp12 --org=Default_Organization
        ;;
      queens)
        subscription-manager register --baseurl=https://$url --force --activationkey=rhosp13 --org=Default_Organization
        ;;
      *)
        echo "unknown release $1" >&2
        exit 1
    esac

Note: rhosp13 activation key should *not* provide Red Hat Ceph Storage 3 repositories.

Comment 14 errata-xmlrpc 2018-06-27 13:48:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.