Bug 1895045 - Validations in FFU 13->16.1: validations before 'openstack undercloud upgrade' failed with:Permission denied: '/var/log/validations'
Summary: Validations in FFU 13->16.1: validations before 'openstack undercloud upgrade...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: validations-common
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z6
: 16.1 (Train on RHEL 8.2)
Assignee: Jiri Podivin
QA Contact: nlevinki
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-11-05 15:59 UTC by Jose Luis Franco
Modified: 2022-09-05 13:16 UTC (History)
6 users (show)

Fixed In Version: python-validations-libs-1.0.5-1.20210326153612.4a660ec.el8ost validations-common-1.1.2-1.20210326153615.7f3c565.el8ost
Doc Type: Bug Fix
Doc Text:
This update fixes a bug that caused failure of validations before `openstack undercloud upgrade` in some cases. Before this upgrade, a lack of permissions needed to access the requested logging directory sometimes resulted in the following failures: ** Failure to log validation results ** Failure of the validation run ** Failure of artifacts collection from validation. + This update adds a fallback logging directory. Validation results are logged and artifacts collected.
Clone Of:
Environment:
Last Closed: 2021-05-26 13:49:37 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 778168 0 None MERGED Make callback write to default if /var/log/validations does not exist 2021-03-24 16:37:29 UTC
OpenStack gerrit 778171 0 None MERGED Default to current user dir for log creation and management 2021-03-24 16:37:27 UTC
Red Hat Issue Tracker OSP-790 0 None None None 2022-09-05 13:16:56 UTC
Red Hat Product Errata RHBA-2021:2097 0 None None None 2021-05-26 13:50:06 UTC

Description Jose Luis Franco 2020-11-05 15:59:45 UTC
Description of problem:
After applying in my local environment the patch https://review.opendev.org/760351 which fixes https://bugzilla.redhat.com/show_bug.cgi?id=1892323  I got the following error when trying to run any validation/group:

(undercloud) [stack@undercloud-0 ~]$ openstack tripleo validator run --static-inventory inventory.yaml --group pre-upgrade              
Error while creating Ansible artifacts log file.Please check the access rights for {}                                                
Traceback (most recent call last):                                                                                                                                   
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 45, in create_artifacts_dir                             
    os.makedirs(log_dir)                                                                                                                            
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs                                                                            
    makedirs(head, mode, exist_ok)                                                                                                                                   
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs                                                                        
    makedirs(head, mode, exist_ok)                                                                                                                   
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs                                                                                      
    mkdir(name, mode)                                                                                                                 
PermissionError: [Errno 13] Permission denied: '/var/log/validations'                                                                        
Exception occured while running the command                                                                                           
Traceback (most recent call last):                                                                                                                  
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 45, in create_artifacts_dir                                   
    os.makedirs(log_dir)                                                                                                                     
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs                                                                           
    makedirs(head, mode, exist_ok)                                                                                                    
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs                                                                              
    makedirs(head, mode, exist_ok)                                                                                                
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs                                                                           
    mkdir(name, mode)                                                                                                                             
PermissionError: [Errno 13] Permission denied: '/var/log/validations'            
During handling of the above exception, another exception occurred:                                                                   
                                                                                                                                     
Traceback (most recent call last):                                                                                                                                   
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run                                                     
    super(Command, self).run(parsed_args)                                                                                            
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run                                                 
    return super(Command, self).run(parsed_args)                                                                                                    
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_validator.py", line 398, in take_action
    self._run_validator_run(parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_validator.py", line 369, in _run_validator_run
    quiet=parsed_args.quiet)
  File "/usr/lib/python3.6/site-packages/validations_libs/validation_actions.py", line 100, in run_validations
    prefix=os.path.basename(playbook))
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 48, in create_artifacts_dir
    LOG.exception("Error while creating Ansible artifacts log file."
AttributeError: 'NoneType' object has no attribute 'format'
'NoneType' object has no attribute 'format'

(undercloud) [stack@undercloud-0 ~]$ openstack tripleo validator run --static-inventory inventory.yaml --validation 512e
Error while creating Ansible artifacts log file.Please check the access rights for {}
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 45, in create_artifacts_dir
    os.makedirs(log_dir)
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/log/validations'
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 45, in create_artifacts_dir
    os.makedirs(log_dir)
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/log/validations'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_validator.py", line 398, in take_action
    self._run_validator_run(parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_validator.py", line 369, in _run_validator_run
    quiet=parsed_args.quiet)
  File "/usr/lib/python3.6/site-packages/validations_libs/validation_actions.py", line 100, in run_validations
    prefix=os.path.basename(playbook))
  File "/usr/lib/python3.6/site-packages/validations_libs/utils.py", line 48, in create_artifacts_dir
    LOG.exception("Error while creating Ansible artifacts log file."
AttributeError: 'NoneType' object has no attribute 'format'
'NoneType' object has no attribute 'format'

(undercloud) [stack@undercloud-0 ~]$ openstack tripleo validator show history
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/display.py", line 116, in run
    column_names, data = self.take_action(parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_validator.py", line 460, in take_action
    return actions.show_history(parsed_args.validation)
  File "/usr/lib/python3.6/site-packages/validations_libs/validation_actions.py", line 164, in show_history
    if validation_id else vlogs.get_all_logfiles(extension))
  File "/usr/lib/python3.6/site-packages/validations_libs/validation_logs.py", line 209, in get_all_logfiles
    return [join(self.logs_path, f) for f in os.listdir(self.logs_path) if
FileNotFoundError: [Errno 2] No such file or directory: '/var/log/validations/'
[Errno 2] No such file or directory: '/var/log/validations/'

When checking the system, the /var/log/validations directory wasn't created yet.



Version-Release number of selected component (if applicable):

(undercloud) [stack@undercloud-0 ~]$ sudo rpm -qa | egrep 'validations|ansible'
openstack-tripleo-validations-11.3.2-1.20200914170825.el8ost.noarch
ansible-tripleo-ipsec-9.2.1-0.20200311073016.0c8693c.el8ost.noarch
tripleo-ansible-0.5.1-1.20200914163925.el8ost.noarch
python3-heat-agent-ansible-1.10.1-0.20200311091123.96b819c.el8ost.noarch
validations-common-1.1.2-1.20200914180304.7f3c565.el8ost.noarch
ansible-config_template-1.1.2-1.20200818183400.ea07ed0.el8ost.noarch
ceph-ansible-4.0.31-1.el8cp.noarch
ansible-role-thales-hsm-0.2.1-0.20200311074756.2803c6c.el8ost.noarch
ansible-freeipa-0.1.8-3.el8.noarch
python3-ansible-runner-1.3.4-2.el8ar.noarch
ansible-role-chrony-1.0.2-0.20200311064420.03e7fbe.el8ost.noarch
ansible-pacemaker-1.0.4-1.20200907073409.accaf26.el8ost.noarch
ansible-role-tripleo-modify-image-1.2.1-0.20200804085623.1dffa21.el8ost.noarch
ansible-role-container-registry-1.1.1-0.20200311065947.7eca2dd.el8ost.noarch
python3-validations-libs-1.0.5-1.20200914174429.4a660ec.el8ost.noarch
ansible-role-openstack-operations-0.0.1-0.20200311080930.274739e.el8ost.noarch
ansible-role-atos-hsm-0.1.1-0.20200318203421.1269408.el8ost.noarch
ansible-tripleo-ipa-0.2.1-1.20200813093411.3bb3c53.el8ost.noarch
ansible-2.9.15-1.el8ae.noarch
ansible-role-redhat-subscription-1.1.1-0.20200716163409.6d26afa.el8ost.noarch


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Jiri Podivin 2021-03-11 08:58:18 UTC
I deployed the 16.1 undecloud and attempted to replicate. 
The directory /var/log/validations was present immediately after deployment and I was unable to replicate.
Requesting more information.

Comment 2 Jiri Podivin 2021-03-26 07:49:19 UTC
Both patches are now backported to 16.2. Backport to 16.1 pending...

Comment 3 Jiri Podivin 2021-03-26 15:19:21 UTC
16.1 trunk patches merged.

Comment 16 errata-xmlrpc 2021-05-26 13:49:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.6 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2097


Note You need to log in before you can comment on or make changes to this bug.