1910508 – [OSP 16.2] No such file or directory: '/var/log/validations

Bug 1910508 - [OSP 16.2] No such file or directory: '/var/log/validations

Summary: [OSP 16.2] No such file or directory: '/var/log/validations

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	validations-common
Sub Component:
Version:	16.2 (Train)
Hardware:	x86_64
OS:	Linux
Priority:	low
Severity:	low
Target Milestone:	rc
Target Release:	16.2 (Train on RHEL 8.4)
Assignee:	mathieu bultel
QA Contact:	nlevinki
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2020-12-24 09:09 UTC by Pradipta Kumar Sahoo
Modified:	2021-10-18 08:55 UTC (History)
CC List:	8 users (show)
Fixed In Version:	validations-common-1.1.2-2.20210316005404.4c29579.el8ost python-validations-libs-1.0.5-2.20210313005027.6c1b8a5.el8ost
Doc Type:	Bug Fix
Doc Text:	Before this update, validation results were not logged and validation artifacts were not collected because the permissions required to access the requested logging directory were not granted. This update resolves the issue, and validation results are successfully logged and validation artifacts are collected.
Clone Of:
Environment:
Last Closed:	2021-09-15 07:11:01 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Launchpad	1904781	None	None	None	2021-03-06 12:36:17 UTC
OpenStack gerrit	778168	None	MERGED	Make callback write to default if /var/log/validations does not exist	2021-03-31 07:08:02 UTC
OpenStack gerrit	778171	None	MERGED	Default to current user dir for log creation and management	2021-03-31 07:08:08 UTC
Red Hat Issue Tracker	VALFRWK-393	None	None	None	2021-08-13 05:48:41 UTC
Red Hat Product Errata	RHEA-2021:3483	None	None	None	2021-09-15 07:11:21 UTC

Description Pradipta Kumar Sahoo 2020-12-24 09:09:23 UTC

Description of problem:
During the deployment, the overcloud stack failed with below error.

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(<ansible.plugins.callback.validation_json.CallbackModule object at
0x7fb0bae71e80>): [Errno 2] No such file or directory: '/var/log/validations/52
5400f8-462c-4ece-1fd9-000000000009_deploy_steps_playbook_2020-12-23T17:12:41.01
5043Z.json'


Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform release 16.2.0 Beta (Train)

How reproducible: 100% reproduced in nfv perf lab


Steps to Reproduce:
1. Deployed Undercloud successfully with the following puddle: RHOS-16.2-RHEL-8-20201215.n.1
2. NFV templates are validated in 16.1 and reuse the same template for osp16.2.
3. The overcloud stack has failed with below status:


2020-12-23 17:23:28.609106 | 525400f8-462c-4ece-1fd9-00000000014b |    SKIPPED | Restart services | nfv-controller-0 | item=rsyslog
2020-12-23 17:23:28.609966 | 525400f8-462c-4ece-1fd9-00000000014b |    SKIPPED | Restart services | nfv-controller-0 | item=crond
2020-12-23 17:23:28.610922 | 525400f8-462c-4ece-1fd9-00000000014b |     TIMING | Restart services | nfv-controller-0 | 0:10:47.596016 | 0.06s

PLAY RECAP *********************************************************************
nfv-compute-offload-0      : ok=112  changed=68   unreachable=0    failed=1    skipped=122  rescued=0    ignored=0   
nfv-controller-0           : ok=170  changed=108  unreachable=0    failed=0    skipped=126  rescued=0    ignored=0   
undercloud                 : ok=10   changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

2020-12-23 17:23:28.901873 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.902029 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 343        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.902147 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:10:47.887248 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.902311 |                                 UUID |       Info |       Host |   Task Name |   Run Time
2020-12-23 17:23:28.902459 | 525400f8-462c-4ece-1fd9-000000000972 |    SUMMARY | nfv-compute-offload-0 | tripleo-kernel : Reboot after kernel args update | 241.62s
2020-12-23 17:23:28.902583 | 525400f8-462c-4ece-1fd9-0000000001a9 |    SUMMARY | nfv-compute-offload-0 | Gathering Facts | 93.04s
2020-12-23 17:23:28.902702 | 525400f8-462c-4ece-1fd9-0000000001cd |    SUMMARY | nfv-compute-offload-0 | Gathering Facts | 91.52s
2020-12-23 17:23:28.902817 | 525400f8-462c-4ece-1fd9-00000000004a |    SUMMARY | nfv-compute-offload-0 | tripleo-network-config : Run NetworkConfig script | 76.07s
2020-12-23 17:23:28.902929 | 525400f8-462c-4ece-1fd9-000000000090 |    SUMMARY | nfv-controller-0 | Run puppet on the host to apply IPtables rules | 24.42s
2020-12-23 17:23:28.903057 | 525400f8-462c-4ece-1fd9-00000000004a |    SUMMARY | nfv-controller-0 | tripleo-network-config : Run NetworkConfig script | 13.83s
2020-12-23 17:23:28.903166 | 525400f8-462c-4ece-1fd9-000000000aaf |    SUMMARY | nfv-controller-0 | tripleo-hieradata : Render hieradata from template | 4.97s
2020-12-23 17:23:28.903286 | 525400f8-462c-4ece-1fd9-000000000aaf |    SUMMARY | nfv-compute-offload-0 | tripleo-hieradata : Render hieradata from template | 4.55s
2020-12-23 17:23:28.903401 | 525400f8-462c-4ece-1fd9-000000000d23 |    SUMMARY | nfv-compute-offload-0 | tripleo-kernel : Set extra sysctl options | 3.05s
2020-12-23 17:23:28.903517 | 525400f8-462c-4ece-1fd9-000000000d23 |    SUMMARY | nfv-controller-0 | tripleo-kernel : Set extra sysctl options | 2.95s
2020-12-23 17:23:28.903613 | 525400f8-462c-4ece-1fd9-0000000001a9 |    SUMMARY | undercloud | Gathering Facts | 2.67s
2020-12-23 17:23:28.903740 | 525400f8-462c-4ece-1fd9-0000000001a9 |    SUMMARY | nfv-controller-0 | Gathering Facts | 2.53s
2020-12-23 17:23:28.903842 | 525400f8-462c-4ece-1fd9-0000000001cd |    SUMMARY | nfv-controller-0 | Gathering Facts | 1.93s
2020-12-23 17:23:28.903942 | 525400f8-462c-4ece-1fd9-0000000001bd |    SUMMARY | undercloud | Gathering Facts | 1.89s
2020-12-23 17:23:28.904051 | 525400f8-462c-4ece-1fd9-0000000000ea |    SUMMARY | nfv-controller-0 | Populate service facts (chrony) | 1.89s
2020-12-23 17:23:28.904159 | 525400f8-462c-4ece-1fd9-000000000d1e |    SUMMARY | nfv-compute-offload-0 | tripleo-kernel : Remove dracut-config-generic | 1.78s
2020-12-23 17:23:28.904285 | 525400f8-462c-4ece-1fd9-0000000000f1 |    SUMMARY | nfv-controller-0 | Populate service facts | 1.69s
2020-12-23 17:23:28.904399 | 525400f8-462c-4ece-1fd9-00000000010d |    SUMMARY | nfv-compute-offload-0 | enable virt_sandbox_use_netlink for healtcheck | 1.67s
2020-12-23 17:23:28.904528 | 525400f8-462c-4ece-1fd9-000000000d1e |    SUMMARY | nfv-controller-0 | tripleo-kernel : Remove dracut-config-generic | 1.62s
2020-12-23 17:23:28.904640 | 525400f8-462c-4ece-1fd9-00000000006c |    SUMMARY | nfv-controller-0 | enable virt_sandbox_use_netlink for healthcheck | 1.55s
2020-12-23 17:23:28.904756 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.904874 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.904991 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~
2020-12-23 17:23:28.905085 |  The following node(s) had failures: nfv-compute-offload-0
2020-12-23 17:23:28.905208 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(<ansible.plugins.callback.validation_json.CallbackModule object at
0x7fb0bae71e80>): [Errno 2] No such file or directory: '/var/log/validations/52
5400f8-462c-4ece-1fd9-000000000009_deploy_steps_playbook_2020-12-23T17:12:41.01
5043Z.json'
Host 192.168.24.8 not found in /home/stack/.ssh/known_hosts
Overcloud configuration failed.


Actual results: Overcloud stack failed


Additional info: The same issue has reported in upstream, kindly share us the downstream patch if it available: 
https://bugs.launchpad.net/tripleo/+bug/1904781

BR,
Pradipta

Comment 2 Alex Schultz 2020-12-25 21:52:12 UTC

The warning is not a fatal error and does not affect the deployment.  We'll use this bz to track this warning and get it resolved.  I think it might be fixed in packaging in the upstream.  You'll need to look back in the logs to see what failed on nfv-compute-offload-0 and open a separate issue if necessary.

nfv-compute-offload-0      : ok=112  changed=68   unreachable=0    failed=1    skipped=122  rescued=0    ignored=0

Comment 3 Jiri Podivin 2021-03-31 07:38:02 UTC

The patch was merged and included in build of both validations libs:
https://pkgs.devel.redhat.com/cgit/rpms/python-validations-libs/commit/?h=rhos-16.2-rhel-8-trunk&id=83342579498646ce42f1973d4397ef77d17546e1

and validations-common:
https://pkgs.devel.redhat.com/cgit/rpms/validations-common/commit/?h=rhos-16.2-rhel-8-trunk&id=aa74906508aa98476d0ecd0e064027d806d4c610

Comment 7 David Rosenfeld 2021-06-07 19:59:53 UTC

The No such file or directory error no longer appears during deploy:

more overcloud_install.log | grep validations

In addition the file: 5254007a-5179-1d59-299f-000000000007_deploy_steps_playbook_2021-06-04T15:28:52.923574Z.json is created in the /home/stack directory.

Comment 9 errata-xmlrpc 2021-09-15 07:11:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:3483

Note You need to log in before you can comment on or make changes to this bug.