Bug 1719691

+++ This bug was initially created as a clone of Bug #1719667 +++

Description of problem:

[1] switched to run nova-manage discovery as non root user.

                command: "/usr/bin/bootstrap_host_exec nova_compute su nova -s /bin/bash -c '/container-config-scripts/pyshim.sh /container-config-scripts/nova_cell_v2_discover_hosts.py'"

In case of updates where we already can have a nova-manage log owned by root make the overcloud deploy to fail with:

2019-06-12 05:30:08,789 p=13529 u=mistral | fatal: [compute-0]: FAILED! => {
    "failed_when_result": true,
    "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [
        "Error running ['docker', 'run', '--name', 'nova_cell_v2_discover_hosts', '--label', 'config_id=tripleo_step5', '--label', 'container_name=nova_cell_v2_discover_hosts', '--label', 'managed_by=paunch', '--label', 'config_data={\"start_order\": 0, \"image\": \"192.168.24.1:87
87/rhosp14/openstack-nova-compute:2019-04-23.1\", \"environment\": [\"TRIPLEO_DEPLOY_IDENTIFIER=1560329712\"], \"command\": \"su nova -s /bin/bash -c \\'/docker-config-scripts/nova_cell_v2_discover_hosts.py\\'\", \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/et
c/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.
crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/nova_libvirt/etc/my.cnf.d/:/etc/my.cnf.d/:ro\
", \"/var/lib/config-data/nova_libvirt/etc/nova/:/etc/nova/:ro\", \"/var/log/containers/nova:/var/log/nova\", \"/var/lib/docker-config-scripts/:/docker-config-scripts/\"], \"net\": \"host\", \"detach\": false}', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1560329712', '--net=host', '--user=ro
ot', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:
/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro
', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/lib/config-data/nova_libvirt/etc/my.cnf.d/:/etc/my.cnf.d/:ro', '--volume=/var/lib/config-data/nova_libvirt/etc/nova/:/etc/nova/:ro', '--volume=/var/log/containers/nova:/var/log/nova', '--volume=/var/lib/docker-config-scripts/
:/docker-config-scripts/', '192.168.24.1:8787/rhosp14/openstack-nova-compute:2019-04-23.1', 'su', 'nova', '-s', '/bin/bash', '-c', \"'/docker-config-scripts/nova_cell_v2_discover_hosts.py'\"]. [1]",
        "",
        "stdout: ERROR:nova_cell_v2_discover_hosts:Cell v2 discovery failed with exit code 1, retrying",
        "ERROR:nova_cell_v2_discover_hosts:Cell v2 discovery failed with exit code 1, retrying",
        "stderr: Traceback (most recent call last):",
        " File \"/usr/bin/nova-manage\", line 10, in <module>",
        " sys.exit(main())",
        " File \"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\", line 2325, in main",
        " logging.setup(CONF, \"nova\")",
        " File \"/usr/lib/python2.7/site-packages/oslo_log/log.py\", line 264, in setup",
        " _setup_logging_from_conf(conf, product_name, version)",
        " File \"/usr/lib/python2.7/site-packages/oslo_log/log.py\", line 353, in _setup_logging_from_conf",
        " filelog = file_handler(logpath)",
        " File \"/usr/lib64/python2.7/logging/handlers.py\", line 392, in __init__",
        " logging.FileHandler.__init__(self, filename, mode, encoding, delay)",
        " File \"/usr/lib64/python2.7/logging/__init__.py\", line 902, in __init__",
        " StreamHandler.__init__(self, self._open())",
        " File \"/usr/lib64/python2.7/logging/__init__.py\", line 925, in _open",
        " stream = open(self.baseFilename, self.mode)",
        "IOError: [Errno 13] Permission denied: '/var/log/nova/nova-manage.log'",
        "Traceback (most recent call last):"
    ]
}

We need to fix log owner ship on every deploy run on the computes, like we have on the
controller.

[1] https://review.opendev.org/#/c/652039/13/deployment/nova/nova-compute-container-puppet.yaml

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1. deploy environment with openstack-tripleo-heat-templates-9.3.1-0.20190314162760.d0a6cb1.el7ost
2. update to openstack-tripleo-heat-templates-9.3.1-0.20190314162762.d0a6cb1
3. run overcloud deploy again

Actual results:
deploy fails on compute with  IOError: [Errno 13] Permission denied: '/var/log/nova/nova-manage.log'

Expected results:
deploy is successful

Additional info:

--- Additional comment from Martin Schuppert on 2019-06-12 10:27:57 UTC ---

This could be worked around deleting /var/log/containers/nova/nova-manage.log or chown it to nova:nova.