Bug 1835874

Summary: openstack tripleo validator run --group pre-introspection fails with Error: no container with name or ID keystone_cron found: no such container
Product: Red Hat OpenStack Reporter: Jose Kaz C R <jcasimir>
Component: openstack-tripleo-validationsAssignee: Gaël Chamoulaud <gchamoul>
Status: CLOSED ERRATA QA Contact: nlevinki <nlevinki>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.0 (Train)CC: cjeanner, gchamoul, jhajyahy, jjoyce, jschluet, slinaber, tvignaud
Target Milestone: betaKeywords: Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-tripleo-validations-11.3.2-0.20200318113434.3fd14c9.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-29 07:52:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jose Kaz C R 2020-05-14 16:35:09 UTC 
Description of problem:

openstack tripleo validator run --group pre-introspection fails with Error: no container with name or ID keystone_cron found: no such container. This happens with the default installation of the undercloud which provides fernet tokens. (Not sure, but could be related to https://bugzilla.redhat.com/show_bug.cgi?id=1600781)

Version-Release number of selected component (if applicable):

$ rpm -qa "openstack-tripleo-validations*"
openstack-tripleo-validations-11.3.2-0.20200206065551.1a9b92a.el8ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. Install an undercloud.
2. Follow the documentation up to the step "openstack tripleo validator run --group pre-introspection" (https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html-single/director_installation_and_usage/index#validating-the-introspection-requirements)
3. Receive the error message.

Actual results:
(undercloud) [stack@nj51labosp16 ~]$ openstack tripleo validator run --group pre-introspection                                                                                                                                              
[SUCCESS] - check-network-gateway.yaml
    b'Using /tmp/check-network-gateway.yaml3gl7tg1mansible.cfg as config file'                                                                                                                                                              
    b'Success! The validation passed for all hosts:'                                                                                                                                                                                        
    b'* undercloud'                                                                                                                                                                                                                         

[SUCCESS] - ctlplane-ip-range.yaml
    b'Using /tmp/ctlplane-ip-range.yamlynv4_3h5ansible.cfg as config file'
    b'Success! The validation passed for all hosts:'
    b'* undercloud'

[SUCCESS] - dhcp-introspection.yaml
    b'Using /tmp/dhcp-introspection.yamld0sh9xouansible.cfg as config file'                                                                                                                                                                 
    b'Success! The validation passed for all hosts:'
    b'* undercloud'                                                                                                                                                                                                                         

[SUCCESS] - undercloud-cpu.yaml                                                                                                                                                                                                             
    b'Using /tmp/undercloud-cpu.yaml3k389jogansible.cfg as config file'                                                                                                                                                                     
    b'Success! The validation passed for all hosts:'                                                                                                                                                                                        
    b'* undercloud'

[SUCCESS] - undercloud-ram.yaml
    b'Using /tmp/undercloud-ram.yamlv2zyk77pansible.cfg as config file'
    b'Success! The validation passed for all hosts:'
    b'* undercloud'

[SUCCESS] - undercloud-selinux-mode.yaml
    b'Using /tmp/undercloud-selinux-mode.yamlcarvsswjansible.cfg as config file'
    b'Success! The validation passed for all hosts:'
    b'* undercloud'

[SUCCESS] - undercloud-disk-space.yaml
    b'Using /tmp/undercloud-disk-space.yamlq5ijq4g8ansible.cfg as config file'
    b'Success! The validation passed for all hosts:'
    b'* undercloud'                                                                                                                                                                                                                         

[FAILED] - undercloud-tokenflush.yaml                                                                                                                                                                                                       
    b'Using /tmp/undercloud-tokenflush.yamlwvmri9v_ansible.cfg as config file'
    b"Task 'undercloud-tokenflush : Get keystone crontab' failed:"                                                                                                                                                                          
    b'Host: undercloud'
    b'Message: Command `set -o pipefail'
    b"podman exec keystone_cron crontab -l -u keystone |grep -v '^#'"
    b'` exited with code: 1: non-zero return code'
    b'stderr:'
    b'    Error: no container with name or ID keystone_cron found: no such container'
    b'Failure! The validation failed for all hosts:'
    b'* undercloud'

[FAILED] - undercloud-neutron-sanity-check.yaml
    b'Using /tmp/undercloud-neutron-sanity-check.yaml6ood_pzgansible.cfg as config file'
    b"Task 'neutron-sanity-check : Run neutron-sanity-check' failed:"
    b'Host: undercloud'
    b'Message: Command `podman exec -u root neutron_ovs_agent /bin/bash -c neutron-sanity-check --config-file /etc/neutron/fwaas_driver.ini` exited with code: 1: non-zero return code'                                                     
    b'stderr:'
    b'    2020-05-13 22:55:58.491 475628 INFO neutron.common.config [-] Logging enabled!\x1b[00m'
    b'    2020-05-13 22:55:58.492 475628 INFO neutron.common.config [-] /usr/bin/neutron-sanity-check version 15.0.2.dev34\x1b[00m'                                                                                                         
    b'    2020-05-13 22:55:58.496 475628 CRITICAL neutron [-] Unhandled error: oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/neutron/fwaas_driver.ini'                                                   
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron Traceback (most recent call last):'
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/bin/neutron-sanity-check", line 10, in <module>'
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron     sys.exit(main())'
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/neutron/cmd/sanity_check.py", line 424, in main'                                                                                            
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron     config.init(sys.argv[1:], default_config_files=[])'
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/neutron/common/config.py", line 89, in init'                                                                                                
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron     **kwargs)'
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/oslo_config/cfg.py", line 2134, in __call__'                                                                                                
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron     raise ConfigFilesNotFoundError(self._namespace._files_not_found)'                                                                                                                
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/neutron/fwaas_driver.ini'                                                                           
    b'    2020-05-13 22:55:58.496 475628 ERROR neutron \x1b[00m'
    b'    Error: non zero exit code: 1: OCI runtime error'
    b"Task 'neutron-sanity-check : Fail' failed:"
    b'Host: undercloud'
    b'Message: 2020-05-13 22:55:58.496 475628 ERROR neutron Traceback (most recent call last):'
    b'2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/bin/neutron-sanity-check", line 10, in <module>'
    b'2020-05-13 22:55:58.496 475628 ERROR neutron     sys.exit(main())'
    b'2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/neutron/cmd/sanity_check.py", line 424, in main'                                                                                                
    b'2020-05-13 22:55:58.496 475628 ERROR neutron     config.init(sys.argv[1:], default_config_files=[])'
    b'2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/neutron/common/config.py", line 89, in init'                                                                                                    
    b'2020-05-13 22:55:58.496 475628 ERROR neutron     **kwargs)'
    b'2020-05-13 22:55:58.496 475628 ERROR neutron   File "/usr/lib/python3.6/site-packages/oslo_config/cfg.py", line 2134, in __call__'                                                                                                    
    b'2020-05-13 22:55:58.496 475628 ERROR neutron     raise ConfigFilesNotFoundError(self._namespace._files_not_found)'                                                                                                                    
    b'2020-05-13 22:55:58.496 475628 ERROR neutron oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/neutron/fwaas_driver.ini'                                                                               
    b'2020-05-13 22:55:58.496 475628 ERROR neutron \x1b[00m'
    b'Failure! The validation failed for all hosts:'
    b'* undercloud'

One or more validations have failed!



Expected results:

See all the validator tasks to be successful

Additional info:

N/A
Comment 1 Gaël Chamoulaud 2020-05-15 08:41:32 UTC 
About the undercloud-tokenflush validation, this commit [1] has to be backported downstream asap in order to fix this issue.

[1] https://opendev.org/openstack/tripleo-validations/commit/3fd14c996068da955ee3842a9771869d51a4ecea
Comment 2 Gaël Chamoulaud 2020-05-15 12:08:19 UTC 
The fix for your issue is part of this following build openstack-tripleo-validations-11.3.2-0.20200318113434.3fd14c9.el8ost.
Comment 7 Jad Haj Yahya 2020-07-23 08:22:29 UTC 
undercloud-tokenflush and undercloud-neutron-sanity-check validations passed
Comment 9 errata-xmlrpc 2020-07-29 07:52:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3148