Bug 1619819 - [OSP 13] overcloud deployment breaks at ControllerDeployment_Step3 when manila and tls everywhere are deployed together
Summary: [OSP 13] overcloud deployment breaks at ControllerDeployment_Step3 when manil...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: z3
: 13.0 (Queens)
Assignee: Goutham Pacha Ravi
QA Contact: Jason Grosso
URL:
Whiteboard:
Depends On: 1644747
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-08-21 19:59 UTC by Matt Flusche
Modified: 2022-03-13 16:10 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.7-2.el7ost
Doc Type: Bug Fix
Doc Text:
The Docker manifests for the manila-api did not contain a necessary bind mount to provide SSL options. As a result, Overcloud deployments failed when deploying with manila and TLS-Everywhere. With this release, the bind mounts for the bootstrap containers are fixed. Overcloud deployments succeed with manila and TLS-Everywhere.
Clone Of:
Environment:
Last Closed: 2018-12-20 11:44:43 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1788337 0 None None None 2018-08-22 04:57:37 UTC
OpenStack gerrit 594801 0 'None' 'MERGED' 'Fix bind-mount to manila''s bootstrap container' 2019-11-20 17:52:32 UTC
OpenStack gerrit 595014 0 'None' 'MERGED' 'Fix bind-mount to manila''s bootstrap container' 2019-11-20 17:52:32 UTC
Red Hat Issue Tracker OSP-13716 0 None None None 2022-03-13 16:10:06 UTC

Description Matt Flusche 2018-08-21 19:59:28 UTC
Description of problem:
When manila and tls everywhere are deployed together the deployment fails at ControllerDeployment_Step3

manila_api_db_sync fails as it appear to use a non-ssl client connection to mariadb; however, ssl is required by the tls everywhere config.

I'll attach more detailed errors to the case.

Deployment error:

overcloud.AllNodesDeploySteps.ControllerDeployment_Step3.0:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: c1773ea6-9eff-499d-936f-8894b948d8a2
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |

[...]

    TASK [Debug output for task which failed: Run puppet host configuration for step 3] ***
    ok: [localhost] => {
        "failed_when_result": false, 

[...]

            "Error running ['docker', 'run', '--name', 'manila_api_db_sync', '--label', 'config_id=tripleo_step3', '--label', 'container_name=manila_api_db_sync', '--label', 'managed_by=paunc
h', '--label', 'config_data={\"command\": \"/usr/bin/bootstrap_host_exec manila_api su manila -s /bin/bash -c \\'/usr/bin/manila-manage db sync\\'\", \"user\": \"root\", \"volumes\": [\"/etc/
hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.cr
t:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/ipa/ca.crt:/etc
/ipa/ca.crt:ro\", \"/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/manila/etc/manila/:/etc/manila/:ro\", \"/var/log/containers/
manila:/var/log/manila\", \"/var/log/containers/httpd/manila-api:/var/log/httpd\"], \"image\": \"hostname:5000/osp13_containers-manila-api:13.0-47\", \"detach\": false,
 \"net\": \"host\"}', '--net=host', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/ex
tracted:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--v
olume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/ipa/ca.crt:/etc/ipa/ca.crt:ro', '--volume=/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts
:ro', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/lib/config-data/manila/etc/manila/:/etc/manila/:ro', '--volume=/var/log/containers/manila:/var/log/manila', '--volume=/var/log/cont
ainers/httpd/manila-api:/var/log/httpd', 'hostname:5000osp13_containers-manila-api:13.0-47', '/usr/bin/bootstrap_host_exec', 'manila_api', 'su', 'manila', '-s', '/bin/
bash', '-c', \"'/usr/bin/manila-manage\", 'db', \"sync'\"]. [1]", 

Error from /var/log/containers/manila/manila-manage.log on controller 0:

2018-08-16 23:32:43.351 11 ERROR manila OperationalError: (pymysql.err.OperationalError) (1045, u"Access denied for user 'manila'@'X.X.X.X' (using password: YES)") (Background on this error at: http://sqlalche.me/e/e3q8)


Version-Release number of selected component (if applicable):
OSP 13 current

How reproducible:
100%

Steps to Reproduce:
1. Deploy OSP 13 with manila and tls everywhere
2.
3.


Additional info:

Will attach links to additional logs

Comment 15 Jason Grosso 2018-12-19 19:11:00 UTC
verified on OSP 13 z4 install with TLS anywhere was successful verified once installed could mount a manila share, write and read a file with TLS enabled  

jenkins build https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OSPD-Customized-Deployment-virt/8306/artifact/.sh/08-ir-tripleo-overcloud-deploy.log

Comment 16 Lon Hohberger 2018-12-20 11:44:43 UTC
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.0.7-4.el7ost.  This build is available now.


Note You need to log in before you can comment on or make changes to this bug.