Bug 1697507 - Forward logging to haproxy.log file task failing in dry-run mode
Summary: Forward logging to haproxy.log file task failing in dry-run mode
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z3
: 14.0 (Rocky)
Assignee: Cédric Jeanneret
QA Contact: Victor Voronkov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-08 14:29 UTC by Victor Voronkov
Modified: 2019-07-02 20:08 UTC (History)
9 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.3.1-0.20190513171729.9995be9.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-02 20:08:27 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1823838 0 None None None 2019-04-09 06:14:25 UTC
OpenStack gerrit 652621 0 None None None 2019-04-15 11:46:27 UTC
OpenStack gerrit 652623 0 None None None 2019-04-15 11:46:27 UTC
Red Hat Product Errata RHBA-2019:1672 0 None None None 2019-07-02 20:08:35 UTC

Description Victor Voronkov 2019-04-08 14:29:27 UTC
Description of problem:
Running deployment in dry run mode (only validating config files without actually deploying, performed by running ansible playbooks in check mode) fails while trying to run Forward logging to haproxy.log file task, here the log:

TASK [Forward logging to haproxy.log file] *************************************
fatal: [controller-2]: FAILED! => {"msg": "The conditional check 'rsyslog_config.rc == 0' failed. The error was: error while evaluating conditional (rsyslog_config.rc == 0): 'dict object' has no attribute 'rc'\n\nThe error appears to have been in '/home/stack/config-download/Controller/host_prep_tasks.yaml': line 479, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n  - blockinfile:\n    ^ here\n"}

Version-Release number of selected component (if applicable):


How reproducible:
Run overcloud deployment with dry-run flag

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Cédric Jeanneret 2019-04-09 06:13:57 UTC
Hello Victor,

It probably don't hit just Rocky, Master should also get it. A patch is provided for Master right now, and will be backported to Rocky once merged.

I'll do some more tests on my own in parallel.

Cheers,

C.

Comment 4 Marius Cornea 2019-04-12 17:31:08 UTC
Issue is still present:

puddle id - 2019-04-12.1
openstack-tripleo-heat-templates.noarch 0:9.3.1-0.20190314162756.d0a6cb1.el7ost 

https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/df/view/rfe/job/DFG-df-rfe-14-virsh-3cont_2comp_3ceph-dry-run-verification/19/artifact/.sh/verification-run.log

Comment 5 Victor Voronkov 2019-04-13 03:36:55 UTC
Solution that worked for sure for previous bug [1] was to disable the whole block where variable was in use [2]

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1684864
[2] https://review.openstack.org/#/c/639402/1/roles/tripleo-ssh-known-hosts/tasks/main.yml

Comment 6 Cédric Jeanneret 2019-04-15 11:40:38 UTC
Hello,

This is the "right" solution (at least, it's working): https://review.openstack.org/#/c/652621/

I've tested it with a simple playbook:
- hosts: localhost
  tasks:
    - name: check command
      shell: systemctl is-active rsyslog
      register: rsyslog_config
    - name: push output
      when:
        - rsyslog_config is changed
        - rsyslog_config.rc == 0
      debug:
        var: rsyslog_config

When we run it with --check:
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [check command] **************************************************************************************************************************************************************************************************************************
skipping: [localhost]

TASK [push output] ****************************************************************************************************************************************************************************************************************************
skipping: [localhost]

PLAY RECAP ************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0

And with standard:
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [check command] **************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [push output] ****************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "rsyslog_config": {
        "changed": true, 
        "cmd": "systemctl is-active rsyslog", 
        "delta": "0:00:00.006942", 
        "end": "2019-04-15 13:31:58.830151", 
        "failed": false, 
        "rc": 0, 
        "start": "2019-04-15 13:31:58.823209", 
        "stderr": "", 
        "stderr_lines": [], 
        "stdout": "active", 
        "stdout_lines": [
            "active"
        ]
    }
}

PLAY RECAP ************************************************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0

Have to make it merge, then backport it into rocky, and finally downstream.

Comment 8 Cédric Jeanneret 2019-04-16 13:05:09 UTC
New package should sort the issue. Sorry for the delay :(

Comment 12 Victor Voronkov 2019-06-04 13:25:13 UTC
"TASK [Forward logging to haproxy.log file] *************************************", "fatal: [controller-2]: FAILED! => {\"msg\": \"The conditional check 'rsyslog_config.rc == 0' failed. The error was: error while evaluating conditional (rsyslog_config.rc == 0): 'dict object' has no attribute 'rc'\\n\\nThe error appears to have been in '/home/stack/config-download/Controller/host_prep_tasks.yaml': line 479, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n- block:\\n  - blockinfile:\\n    ^ here\\n\"}", "fatal: [controller-1]: FAILED! => {\"msg\": \"The conditional check 'rsyslog_config.rc == 0' failed. The error was: error while evaluating conditional (rsyslog_config.rc == 0): 'dict object' has no attribute 'rc'\\n\\nThe error appears to have been in '/home/stack/config-download/Controller/host_prep_tasks.yaml': line 479, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n- block:\\n  - blockinfile:\\n    ^ here\\n\"}", "fatal: [controller-0]: FAILED! => {\"msg\": \"The conditional check 'rsyslog_config.rc == 0' failed. The error was: error while evaluating conditional (rsyslog_config.rc == 0): 'dict object' has no attribute 'rc'\\n\\nThe error appears to have been in '/home/stack/config-download/Controller/host_prep_tasks.yaml': line 479, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n- block:\\n  - blockinfile:\\n    ^ here\\n\"}",


core_puddle 2019-05-31.1

openstack-tripleo-heat-templates-9.3.1-0.20190513171725.9995be9.el7ost.noarch

The fix should be in the compose, but it's missing:

- name: Check if rsyslog exists
  register: rsyslog_config
  shell: systemctl is-active rsyslog
- block:
  - blockinfile:
      content: 'if $syslogfacility-text == ''{{facility}}'' and $programname == ''haproxy''
        then -/var/log/containers/haproxy/haproxy.log

        & stop

        '
      create: true
      path: /etc/rsyslog.d/openstack-haproxy.conf
    name: Forward logging to haproxy.log file
    register: logconfig
    vars:
      facility: local0
  - name: restart rsyslog service after logging conf change
    service:
      name: rsyslog
      state: restarted
    when: logconfig is changed
  when: rsyslog_config.rc == 0
- file:

Comment 13 Cédric Jeanneret 2019-06-07 19:40:00 UTC
Hello,

so the backport for pacemaker wasn't done for rocky - the stein one took a long time and got merged only last week - since I was off it wasn't taken down to rocky.

I've just pushed https://review.opendev.org/#/c/664046/ - hopefully it will be merged next week upstream, and I'll backport it now downstream.

Sorry for all those delays.

Comment 14 Cédric Jeanneret 2019-06-07 19:43:56 UTC
(added downstream backport for osp-14)

Comment 16 Victor Voronkov 2019-06-12 15:55:55 UTC
Verified on compose 2019-06-10.3

Comment 19 errata-xmlrpc 2019-07-02 20:08:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1672


Note You need to log in before you can comment on or make changes to this bug.