Description of problem:

	
In Progress
	
High
	
  Damien Ciabrini
	
Target to milestone
Also affects project (?) Also affects distribution/package Target to series
Edit
Bug Description

Since I92d4ddf2feeac06ce14468ae928c283f3fd04f45, HA containers <service>_restart_bundle.sh run script pacemaker_restart_bundle.sh to drive the resource restart behaviour based on the value of environment variable TRIPLEO_MINOR_UPDATE. This allows the behaviour to be adapted whether we're running a stack update or a minor update.

However the script doesn't work as expected when TRIPLEO_MINOR_UPDATE is undefined, due to pacemaker_restart_bundle.sh using "set -u". Consequently, any stack update that would trigger that container would error out with e.g.:

 paunch --debug apply --file /var/lib/tripleo-config/hashed-docker-container-startup-config-step_2.json --config-id tripleo_step2 --managed-by tripleo-Controller 2>&1 | tee /tmp/restart
Error running ['docker', 'run', '--name', 'haproxy_restart_bundle', '--label', 'config_id=tripleo_step2', '--label', 'container_name=haproxy_restart_bundle', '--label', 'managed_by=paunch', '--label', 'config_data={"start_order": 2, "image
": "192.168.24.1:8787/rhosp13/openstack-haproxy:2019-10-09.1", "environment": ["TRIPLEO_MINOR_UPDATE", "TRIPLEO_CONFIG_HASH=358ed8a61f99e0575302387a156889ee"], "config_volume": "haproxy", "command": "/pacemaker_restart_bundle.sh haproxy-bu
ndle haproxy", "user": "root", "volumes": ["/etc/hosts:/etc/hosts:ro", "/etc/localtime:/etc/localtime:ro", "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro"
, "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", "/dev/log:/dev/log", "/var/lib/
docker-config-scripts/pacemaker_restart_bundle.sh:/pacemaker_restart_bundle.sh:ro", "/etc/corosync/corosync.conf:/etc/corosync/corosync.conf:ro", "/dev/shm:/dev/shm:rw", "/etc/puppet:/etc/puppet:ro", "/var/lib/config-data/puppet-generated/
haproxy/:/var/lib/kolla/config_files/src:ro"], "net": "host", "detach": false}', '--env=TRIPLEO_MINOR_UPDATE', '--env=TRIPLEO_CONFIG_HASH=358ed8a61f99e0575302387a156889ee', '--net=host', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro',
'--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt
:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/var/lib
/docker-config-scripts/pacemaker_restart_bundle.sh:/pacemaker_restart_bundle.sh:ro', '--volume=/etc/corosync/corosync.conf:/etc/corosync/corosync.conf:ro', '--volume=/dev/shm:/dev/shm:rw', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/
var/lib/config-data/puppet-generated/haproxy/:/var/lib/kolla/config_files/src:ro', '192.168.24.1:8787/rhosp13/openstack-haproxy:2019-10-09.1', '/pacemaker_restart_bundle.sh', 'haproxy-bundle', 'haproxy']. [1]

stdout: Bundle: haproxy-bundle
  Docker: image=192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest network=host options="--user=root --log-driver=journald -e KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" replicas=3 run-command="/bin/bash /usr/local/bin/kolla_start"
  Storage Mapping:
   options=ro source-dir=/var/lib/kolla/config_files/haproxy.json target-dir=/var/lib/kolla/config_files/config.json (haproxy-cfg-files)
   options=ro source-dir=/var/lib/config-data/puppet-generated/haproxy/ target-dir=/var/lib/kolla/config_files/src (haproxy-cfg-data)
   options=ro source-dir=/etc/hosts target-dir=/etc/hosts (haproxy-hosts)
   options=ro source-dir=/etc/localtime target-dir=/etc/localtime (haproxy-localtime)
   options=rw source-dir=/var/lib/haproxy target-dir=/var/lib/haproxy (haproxy-var-lib)
   options=ro source-dir=/etc/pki/ca-trust/extracted target-dir=/etc/pki/ca-trust/extracted (haproxy-pki-extracted)
   options=ro source-dir=/etc/pki/tls/certs/ca-bundle.crt target-dir=/etc/pki/tls/certs/ca-bundle.crt (haproxy-pki-ca-bundle-crt)
   options=ro source-dir=/etc/pki/tls/certs/ca-bundle.trust.crt target-dir=/etc/pki/tls/certs/ca-bundle.trust.crt (haproxy-pki-ca-bundle-trust-crt)
   options=ro source-dir=/etc/pki/tls/cert.pem target-dir=/etc/pki/tls/cert.pem (haproxy-pki-cert)
   options=rw source-dir=/dev/log target-dir=/dev/log (haproxy-dev-log)
   options=ro source-dir=/etc/pki/tls/private/overcloud_endpoint.pem target-dir=/var/lib/kolla/config_files/src-tls/etc/pki/tls/private/overcloud_endpoint.pem (haproxy-cert)

stderr: /pacemaker_restart_bundle.sh: line 11: TRIPLEO_MINOR_UPDATE: unbound variable

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-8.4.1-5.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. deploy an HA overcloud
2. to simulate a stack update that would restart a <svc>_restart_bundle, e.g.:
2.A delete stopped ha_proxy_restart_bundle:
   docker rm haproxy_restart_bundle
2.B let paunch recreate and restart it:
   paunch --debug apply --file /var/lib/tripleo-config/hashed-docker-container-startup-config-step_2.json --config-id tripleo_step2 --managed-by tripleo-Controller

Actual results:
haproxy_restart_bundle will exit in error due to TRIPLEO_MINOR_UPDATE being undefined

Expected results:
haproxy_restart_bundle should finish successfully

Additional info: