Bug 1856835 - TripleO overcloud deployments with amphora log offloading enable (THT parameter "OctaviaLogOffload") fail
Summary: TripleO overcloud deployments with amphora log offloading enable (THT paramet...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: tripleo-ansible
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z2
: 16.1 (Train on RHEL 8.2)
Assignee: Carlos Goncalves
QA Contact: Omer Schwartz
URL:
Whiteboard:
: 1858588 (view as bug list)
Depends On: 1883536
Blocks: 1623977 1860658 1881362
TreeView+ depends on / blocked
 
Reported: 2020-07-14 14:29 UTC by Omer Schwartz
Modified: 2020-10-28 15:38 UTC (History)
7 users (show)

Fixed In Version: tripleo-ansible-0.5.1-0.20200715213413.04c6fff.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-28 15:38:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1887516 0 None None None 2020-07-15 14:34:44 UTC
OpenStack gerrit 740944 0 None MERGED Fix Octavia amphora log offloading template path 2020-12-24 18:26:15 UTC
Red Hat Product Errata RHEA-2020:4284 0 None None None 2020-10-28 15:38:54 UTC

Description Omer Schwartz 2020-07-14 14:29:57 UTC
Description of problem:

TripleO overcloud deployments with amphora log offloading enable (THT parameter "OctaviaLogOffload") fail with:

"fatal: [controller-2]: FAILED! => {\"changed\": false, \"msg\": \"Could not 
find or access 'templates/10-octavia.conf.j2'\\nSearched in:\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/templates/templates/10-octavia.conf.j2
\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/tasks/templates/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/tasks/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/tripleo-playbooks/templates/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/tripleo-playbooks/templates/10-octavia.conf.j2 on the Ansible Controller.\\nIf you are using a module and expect the file to exist on the remote, see the remote_src option\

Version-Release number of selected component (if applicable):
(undercloud) [stack@undercloud-0 ~]$ cat /var/lib/rhos-release/latest-installed

16.1  -p RHOS-16.1-RHEL-8-20200701.n.0

How reproducible:

100%, I managed to reproduce it on my enviroment.

Steps to Reproduce:
1. Deploy OSP 16.1 in HA
2. Change the flag of the LogOffload default to true on /usr/share/openstack-tripleo-heat-templates/deployment/octavia/octavia-deployment-config.j2.yaml
3. Run overcloud_deploy.sh

Actual results:

Overcloud deployed failed:

"fatal: [controller-2]: FAILED! => {\"changed\": false, \"msg\": \"Could not find or access 'templates/10-octavia.conf.j2'\\nSearched in:\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/templates/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/tasks/templates/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/roles/octavia-controller-post-config/tasks/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/tripleo-playbooks/templates/templates/10-octavia.conf.j2\\n\\t/usr/share/ansible/tripleo-playbooks/templates/10-octavia.conf.j2 on the Ansible Controller.\\n

In file /var/lib/mistral/overcloud/ansible.log

Expected results:

Overcloud deployment was supposed to pass.

Additional info:

Launchpad bug: https://bugs.launchpad.net/tripleo/+bug/1887516

Comment 3 Bruna Bonguardo 2020-07-19 10:48:37 UTC
*** Bug 1858588 has been marked as a duplicate of this bug. ***

Comment 7 Omer Schwartz 2020-09-15 11:21:35 UTC
Hi,

The package tripleo-ansible-0.5.1-0.20200715213413.04c6fff.el8 is not in the passed_phase2 puddle of osp16.1.

The current passed_phase2 puddle RHOS-16.1-RHEL-8-20200903.n.0 [1] has the tripleo-ansible-0.5.1-0.20200611113659.34b8fcc.el8ost.noarch.rpm package. [2]



[1] http://download.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/16.1-RHEL-8/RHOS-16.1-RHEL-8-20200903.n.0/

[2] http://download.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/16.1-RHEL-8/passed_phase2/compose/OpenStack/x86_64/os/Packages/

Comment 8 Omer Schwartz 2020-09-24 09:16:29 UTC
Hi,

I tried to verify this bug and after I ran the steps to reproduce, I saw that the bug still occurs:

2020-09-24 05:04:00,324 p=62323 u=mistral n=ansible | fatal: [undercloud]: FAILED! => {"changed": true, "cmd": "ANSIBLE_CONFIG=\"/var/lib/mistral/overcloud/ansible.cfg\" ansible-playbook -i \"/var/lib/mistral/overcloud/octavia-ansible/inventory.yaml\" --extra-vars @/var/lib/mistral/overcloud/octavia-ansible/group_vars/octavia_vars.yaml --private-key /var/lib/mistral/overcloud/ssh_private_key /usr/share/ansible/tripleo-playbooks/octavia-files.yaml", "delta": "0:00:32.725300", "end": "2020-09-24 05:04:00.281225", "msg": "non-zero return code", "rc": 2, "start": "2020-09-24 05:03:27.555925", "stderr": "[WARNING]: Failure using method (v2_runner_on_start) in callback plugin\n(<ansible.plugins.callback.tripleo.CallbackModule object at 0x7fac73bb7240>):\n'show_per_host_start'\nERROR! the role 'octavia-controller-check' was not found in /usr/share/ansible/tripleo-playbooks/roles:/var/lib/mistral/overcloud/roles:/root/.ansible/roles:/usr/share/ansible/tripleo-roles:/usr/share/ansible/roles:/etc/ansible/roles:/usr/share/openstack-tripleo-validations/roles:/var/lib/mistral/overcloud:/usr/share/ansible/tripleo-playbooks\n\nThe error appears to be in '/usr/share/ansible/tripleo-playbooks/octavia-files.yaml': line 40, column 15, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n    - include_role:\n        name: octavia-controller-check\n              ^ here", "stderr_lines": ["[WARNING]: Failure using method (v2_runner_on_start) in callback plugin", "(<ansible.plugins.callback.tripleo.CallbackModule object at 0x7fac73bb7240>):", "'show_per_host_start'", "ERROR! the role 'octavia-controller-check' was not found in /usr/share/ansible/tripleo-playbooks/roles:/var/lib/mistral/overcloud/roles:/root/.ansible/roles:/usr/share/ansible/tripleo-roles:/usr/share/ansible/roles:/etc/ansible/roles:/usr/share/openstack-tripleo-validations/roles:/var/lib/mistral/overcloud:/usr/share/ansible/tripleo-playbooks", "", "The error appears to be in '/usr/share/ansible/tripleo-playbooks/octavia-files.yaml': line 40, column 15, but may", "be elsewhere in the file depending on the exact syntax problem.", "", "The offending line appears to be:", "", "    - include_role:", "        name: octavia-controller-check", "              ^ here"], "stdout": "\nPLAY [Undercloud[0]]



Which leads to the failure of the Amphora Log Offloading deployment:
2020-09-24 05:04:00,326 p=62323 u=mistral n=ansible | NO MORE HOSTS LEFT *************************************************************
2020-09-24 05:04:00,327 p=62323 u=mistral n=ansible | PLAY RECAP *********************************************************************
2020-09-24 05:04:00,327 p=62323 u=mistral n=ansible | compute-0                  : ok=257  changed=103  unreachable=0    failed=0    skipped=302  rescued=0    ignored=0 
2020-09-24 05:04:00,327 p=62323 u=mistral n=ansible | compute-1                  : ok=254  changed=103  unreachable=0    failed=0    skipped=302  rescued=0    ignored=0 
2020-09-24 05:04:00,327 p=62323 u=mistral n=ansible | controller-0               : ok=307  changed=134  unreachable=0    failed=0    skipped=291  rescued=0    ignored=0 
2020-09-24 05:04:00,327 p=62323 u=mistral n=ansible | controller-1               : ok=293  changed=134  unreachable=0    failed=0    skipped=289  rescued=0    ignored=0 
2020-09-24 05:04:00,328 p=62323 u=mistral n=ansible | controller-2               : ok=293  changed=134  unreachable=0    failed=0    skipped=289  rescued=0    ignored=0 
2020-09-24 05:04:00,328 p=62323 u=mistral n=ansible | undercloud                 : ok=96   changed=26   unreachable=0    failed=1    skipped=9    rescued=0    ignored=0


Thanks

Comment 9 Carlos Goncalves 2020-10-01 06:38:14 UTC
Error in comment #8 is a separate bug that blocks verification of this BZ, and is being tracked in BZ #1883536.

Comment 11 Bruna Bonguardo 2020-10-01 11:43:02 UTC

*** This bug has been marked as a duplicate of bug 1883536 ***

Comment 12 Omer Schwartz 2020-10-01 16:11:37 UTC
Yes, I had a misunderstanding about this bug, I agree with Carlos in Comment #9,

It should be ON_QA.

Comment 13 Omer Schwartz 2020-10-06 10:11:25 UTC
After verification process that involved these steps I:

1. Deployed OSP 16.1 in HA.
2. Added to the environment file /home/stack/virt/extra_templates.yaml the following flag:
    parameter_defaults:
        OctaviaLogOffload: true
   And added it to the overcloud_deploy.sh.
3. Ran overcloud_deploy.sh


Actual results:

Ansible passed.
Overcloud configuration completed.
Overcloud Endpoint: http://10.0.0.119:5000
Overcloud Horizon Dashboard URL: http://10.0.0.119:80/dashboard
Overcloud rc file: /home/stack/overcloudrc
Overcloud Deployed without error


(overcloud) [stack@undercloud-0 ~]$ cat /var/lib/rhos-release/latest-installed
16.1  -p RHOS-16.1-RHEL-8-20201005.n.0


Looks good to me, moving this BZ to MODIFIED.

Comment 18 errata-xmlrpc 2020-10-28 15:38:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4284


Note You need to log in before you can comment on or make changes to this bug.