Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1693560

Summary: Initial validations fail on Set fluentd_base_packages_available fact
Product: [oVirt] ovirt-engine-metrics Reporter: Shirly Radco <sradco>
Component: GenericAssignee: Shirly Radco <sradco>
Status: CLOSED CURRENTRELEASE QA Contact: Nikolai Sednev <nsednev>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 1.2.1.2CC: bugs, cshao, dfediuck, lleistne, nsednev, sbonazzo
Target Milestone: ovirt-4.3.3Keywords: Triaged
Target Release: 1.2.1.3Flags: nsednev: needinfo+
dfediuck: ovirt-4.3+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-metrics-1.2.1.3 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-16 13:58:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Metrics RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1670492, 1671912    
Attachments:
Description Flags
sosreport from alma04 none

Description Shirly Radco 2019-03-28 07:51:42 UTC
Description of problem:

Failure in initial validations 

TASK [oVirt.initial-validations : Check Fluentd packages are available] **********************************************************************************************************************
skipping: [localhost] => (item=fluentdxxx) 
skipping: [localhost] => (item=rubygem-fluent-plugin-collectd-nest) 
skipping: [localhost] => (item=rubygem-fluent-plugin-rewrite-tag-filter) 
skipping: [localhost] => (item=rubygem-fluent-plugin-secure-forward) 
skipping: [localhost] => (item=rubygem-fluent-plugin-viaq_data_model) 
skipping: [localhost] => (item=rubygem-fluent-plugin-elasticsearch) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=fluentdxxx) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=rubygem-fluent-plugin-collectd-nest) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=rubygem-fluent-plugin-rewrite-tag-filter) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=rubygem-fluent-plugin-secure-forward) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=rubygem-fluent-plugin-viaq_data_model) 
skipping: [hera06.eng.lab.tlv.redhat.com] => (item=rubygem-fluent-plugin-elasticsearch) 

TASK [oVirt.initial-validations : Set fluentd_base_packages_available fact] ******************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The conditional check 'item.results == []' failed. The error was: error while evaluating conditional (item.results == []): 'dict object' has no attribute 'results'\n\nThe error appears to have been in '/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.initial-validations/tasks/check_logging_collectors.yml': line 64, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set fluentd_base_packages_available fact\n  ^ here\n"}
fatal: [hera06.eng.lab.tlv.redhat.com]: FAILED! => {"msg": "The conditional check 'item.results == []' failed. The error was: error while evaluating conditional (item.results == []): 'dict object' has no attribute 'results'\n\nThe error appears to have been in '/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.initial-validations/tasks/check_logging_collectors.yml': line 64, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set fluentd_base_packages_available fact\n  ^ here\n"}

Version-Release number of selected component (if applicable):
1.2.1.2

How reproducible:
100%

Steps to Reproduce:
1. Run configure_ovirt_machines_for_metrics.sh from engine machine
2.
3.

Actual results:
The above failure

Expected results:
No failure and correct logging_collector is set based on the packages availability

Additional info:

Comment 1 Nikolai Sednev 2019-04-02 16:33:53 UTC
I tried to deploy HE using rhvm-appliance-4.3-20190328.0.el7.x86_64.rhevm.ova and deployment failed with the same metrics errors:

2019-04-02 18:04:22,141 p=3416 u=ovirt |  TASK [oVirt.metrics/roles/oVirt.initial-validations : Set fluentd_base_packages_available fact] ***
2019-04-02 18:04:22,193 p=3416 u=ovirt |  fatal: [alma04.qa.lab.tlv.redhat.com]: FAILED! => {}

MSG:

The conditional check 'item.results == []' failed. The error was: error while evaluating conditional (item.results == []): 'dict object' has no attribute 'results'

The error appears to have been in '/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.initial-validations/tasks/check_logging_collectors.yml': line 60, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

Tested on:
ovirt-hosted-engine-setup-2.3.7-1.el7ev.noarch
ovirt-hosted-engine-ha-2.3.1-1.el7ev.noarch
ansible-2.7.9-1.el7ae.noarch
ovirt-ansible-hosted-engine-setup-1.0.14-1.el7ev.noarch
ovirt-ansible-engine-setup-1.1.9-1.el7ev.noarch
ovirt-ansible-repositories-1.1.5-1.el7ev.noarch
Linux 3.10.0-957.10.1.el7.x86_64 #1 SMP Thu Feb 7 07:12:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.6 (Maipo)

Forth to our conversation with Simone, moving back to assigned.

Comment 2 Nikolai Sednev 2019-04-02 16:34:43 UTC
Created attachment 1551081 [details]
sosreport from alma04

Comment 3 Shirly Radco 2019-04-02 18:22:41 UTC
Please provide the full ovirt metrics log, this snippet is not enough to figure out the issue.
Also please provide more details on when this failed? On adding additional host?
What is the the ovirt engine-metrics rpm version in your installation?

Comment 4 Shirly Radco 2019-04-02 18:40:43 UTC
Sorry, I found ovirt-host-deploy-ansible-20190402180241-alma04.qa.lab.tlv.redhat.com-3122a5f9.log
that includes the log.
From it I see that this is not the correct ovirt-engine-metrics 1.2.1.3 rpm.
Can you please check again?

Comment 5 Nikolai Sednev 2019-04-03 08:07:11 UTC
(In reply to Shirly Radco from comment #3)
> Please provide the full ovirt metrics log, this snippet is not enough to
> figure out the issue.
> Also please provide more details on when this failed? On adding additional
> host?
> What is the the ovirt engine-metrics rpm version in your installation?

It failed on adding the first and the only ha-host, during HE deployment.
The metrics rpm is what we're getting from rhvm-appliance-4.3-20190328.0.el7.x86_64.rhevm.ova, 
I can retry using later OVAs if you wish, like rhvm-appliance-4.3-20190328.1.el7.x86_64.rhevm.ova, its the latest available for QA.

Comment 6 Shirly Radco 2019-04-03 09:42:22 UTC
Sandro, Is the ovirt-engine-metrics rpm for the latest rhvm-appliance-4.3 is ovirt-engine-metrics-1.2.1.3-1.el7ev ?

Comment 7 Nikolai Sednev 2019-04-04 09:13:59 UTC
In latest engine ovirt-engine-setup-4.3.3.1-0.1.el7.noarch available to QA we have:
ovirt-engine-metrics-1.2.1.3-1.el7ev.noarch
hystrix-metrics-event-stream-1.4.21-6.el7ev.noarch

Comment 8 Nikolai Sednev 2019-04-04 12:44:24 UTC
Deployment was now successful from OVA image and also ./configure_ovirt_machines_for_metrics.sh ran without issues.
PLAY RECAP ***********************************************************************************************************
alma04.qa.lab.tlv.redhat.com : ok=14   changed=2    unreachable=0    failed=0   
localhost                  : ok=20   changed=4    unreachable=0    failed=0   

Details of the script run can be found here:
http://pastebin.test.redhat.com/749691

Tested on these components:
ovirt-engine-setup-4.3.3.1-0.1.el7.noarch
rhvm-appliance-4.3-20190328.1.el7.x86_64.rhevm.ova
ovirt-ansible-hosted-engine-setup-1.0.15-1.el7ev.noarch
ovirt-ansible-engine-setup-1.1.9-1.el7ev.noarch
ovirt-ansible-repositories-1.1.5-1.el7ev.noarch
ansible-2.8.0-1.201904040040git.9776037abe.el7.ans.noarch
ovirt-hosted-engine-setup-2.3.7-1.el7ev.noarch
ovirt-hosted-engine-ha-2.3.1-1.el7ev.noarch
Linux 3.10.0-957.10.1.el7.x86_64 #1 SMP Thu Feb 7 07:12:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.6 (Maipo)

Moving to verified.

Comment 9 Sandro Bonazzola 2019-04-16 13:58:22 UTC
This bugzilla is included in oVirt 4.3.3 release, published on April 16th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.3 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.