Bug 1470016

Summary: OSP11 -> OSP12 upgrade: when using extra overcloud services the user needs to switch the environment file from environments/services/ to environments/services-docker/
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: openstack-tripleo-heat-templatesAssignee: Dan Prince <dprince>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: urgent Docs Contact:
Priority: high    
Version: 12.0 (Pike)CC: dbecker, jschluet, jstransk, ltoscano, mandreou, mbracho, mbultel, mburns, morazi, rhel-osp-director-maint, sclewis, shardy, tvignaud
Target Milestone: betaKeywords: Triaged
Target Release: 12.0 (Pike)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-7.0.3-6.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-13 21:40:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1399762    

Description Marius Cornea 2017-07-12 09:30:36 UTC
Description of problem:
OSP11 -> OSP12 upgrade: Sahara services are still running under systemd after upgrading to docker architecture:

During upgrade, at step overcloud-AllNodesDeploySteps-74yhsp77to5a-AllNodesPostUpgradeSteps-tl7b4sbb2xft-ControllerDeployment_Step3-o2ybgewtnni3
we can see on the overcloud node:

[root@controller-0 heat-admin]# systemctl status openstack-sahara-api.service
● openstack-sahara-api.service - Sahara API Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-sahara-api.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Tue 2017-07-11 22:43:28 UTC; 10h ago
 Main PID: 84013 (code=exited, status=0/SUCCESS)

Jul 11 18:01:31 controller-0 systemd[1]: Started Sahara API Server.
Jul 11 18:01:31 controller-0 systemd[1]: Starting Sahara API Server...
Jul 11 22:43:28 controller-0 systemd[1]: Stopping Sahara API Server...
Jul 11 22:43:28 controller-0 systemd[1]: Stopped Sahara API Server.
[root@controller-0 heat-admin]# systemctl status openstack-sahara-engine.service
● openstack-sahara-engine.service - Sahara Engine Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-sahara-engine.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-07-11 23:28:25 UTC; 9h ago
 Main PID: 307754 (sahara-engine)
   Memory: 226.7M
   CGroup: /system.slice/openstack-sahara-engine.service
           ├─307754 /usr/bin/python2 /usr/bin/sahara-engine --config-file /etc/sahara/sahara.conf
           └─307824 /usr/bin/python2 /usr/bin/sahara-engine --config-file /etc/sahara/sahara.conf

Jul 11 23:28:25 controller-0 systemd[1]: Started Sahara Engine Server.
Jul 11 23:28:25 controller-0 systemd[1]: Starting Sahara Engine Server...

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch

How reproducible:
100%
Steps to Reproduce:
1. Deploy OSP11
2. Upgrade to OSP12 with containerized architecture

Actual results:
Sahara services are running or enabled under systemd.

Expected results:
Sahara services should be disabled in systemd and running only inside containers.

Additional info:

Comment 1 Marius Cornea 2017-07-12 09:48:55 UTC
This is the overcloud deploy command used for upgrade:

(undercloud) [stack@undercloud-0 ~]$ cat composable_docker_upgrade.sh 
#!/bin/bash

openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
--environment-file /usr/share/openstack-tripleo-heat-templates/environments/services/sahara.yaml \
--environment-file /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/debug.yaml \
-e /home/stack/virt/nodes_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps-docker.yaml \
-e /home/stack/docker-osp12.yaml \

Note that in OSP11 we used /usr/share/openstack-tripleo-heat-templates/environments/services/sahara.yaml but for the Docker services we need switch it to /usr/share/openstack-tripleo-heat-templates/environments/services-docker/sahara.yaml

Comment 2 Marius Cornea 2017-07-12 15:32:18 UTC
I am changing the title to make this report more generic. When upgrading to OSP12 and moving to containers we need to manually replace the environment file(in OSP11 they are located in environments/services/) which activate the extra feature with its correspondent in environments/services-docker. This is an error prone operation which can lead to unexpected upgrade results in case switching the environment files gets missed by the user. The transition to containerized services should be as much as possible transparent for the user.

Comment 3 Jiri Stransky 2017-07-25 15:42:59 UTC
This ties into a greater problem of dockerized deployment not being the default. E.g. we also have to use docker.yaml, docker-ha.yaml, major-upgrade-composable-steps-docker.yaml. (We had to go this way to allow for Puppet variant to keep working, and build the Docker variant in parallel.)

Technically it should be possible to change all the defaults to Docker and make Puppet be the special case instead, but the cost of this could be fairly high as the change is an inherent chicken-and-egg issue with regards to CI and people's expectations.

We'd have to first copy all the defaults to Puppet-specific files, then make CI/people use those new files when they want to deploy Puppet variant, then we can switch the defaults to Docker, then make CI/people use the defaults when they want to deploy Docker variant (instead of Docker-specific files), and then get rid of the (now deprecated) Docker-specific files from repos.

So it's a multi-step change and we should weigh the costs/benefits here to assess feasibility.

Comment 12 errata-xmlrpc 2017-12-13 21:40:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462