Description of problem: rhcert-ci shows up this KeyError message several times in the logs. (...) Subtest: Configuration - Check tempest configuration Reusing the existing tempest.conf under /etc/redhat-certification-openstack as it already exists! Error: Exception raised in test openstack/neutron: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/rhcert/client/harness.py", line 1055, in runTest rv = test.run() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/neutron.py", line 238, in run description="Check tempest configuration")) File "/usr/lib/python2.7/site-packages/rhcert/test.py", line 363, in runSubTest result = subtestFunction() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/neutron.py", line 68, in pre_run self.map_apis_to_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 316, in map_apis_to_tests self.tempest_test_list = tempest_test.get_tempest_mapped_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 42, in get_tempest_mapped_tests self.policy_test_list = self.get_policy_test_list() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 37, in get_policy_test_list self.policy_test_regex.append(self.cert_type[api]) KeyError: u'volumes' copying attachments... (...) Subtest: Configuration - Check tempest configuration Reusing the existing tempest.conf under /etc/redhat-certification-openstack as it already exists! Error: Exception raised in test openstack/sahara: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/rhcert/client/harness.py", line 1055, in runTest rv = test.run() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 473, in run description="Check tempest configuration")) File "/usr/lib/python2.7/site-packages/rhcert/test.py", line 363, in runSubTest result = subtestFunction() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 306, in pre_run self.map_apis_to_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 316, in map_apis_to_tests self.tempest_test_list = tempest_test.get_tempest_mapped_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 42, in get_tempest_mapped_tests self.policy_test_list = self.get_policy_test_list() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 37, in get_policy_test_list self.policy_test_regex.append(self.cert_type[api]) KeyError: u'volumes' copying attachments... checking directory /var/log/rhcert/runs/1/openstack/sahara Skipping output.log (...) Version-Release number of selected component (if applicable): How reproducible: [stack@director ~]$ cat /etc/redhat-certification-openstack/test_config.json { "keystone_auth_url" : "http://192.168.190.37:5000/v2.0", "keystone_region_name": "regionOne", "admin_username": "admin", "admin_password": "foo", "admin_tenant_name": "admin", "plugin_type": "blockstorage", "supported_apis_and_extensions": ["volumes","snapshots","volume_types","qos","quota-set-extension","multiple-backends","availability","extensions","volume_transfers"], "lbaas_api_version": "v2", "undercloud_stackrc": "/home/stack/stackrc" } [stack@director ~]$ sudo rhcert-ci run --key-file /home/stack/key.file --debug high Steps to Reproduce: 1. prepare the configuration 2. run the rhcert-ci command Actual results: Error messages in the logs. Expected results: No error in the logs.
Hi Goneri, I think you would probably want to use something like, $rhcert-ci run --test cinder --key-file /home/stack/key.file --debug high with the test_config.json mentioned above, that should work fine. Error is happening because you are basically running every test which includes neutron cert and volumes api doesn't exist there.
I thought it was the purpose of the plugin_type key. I still believe this is a bug because the content of the supported_apis_and_extensions section will never match what all the tests can run and the user will always get these KeyError messages. --test should be a mandatory parameter. Could we have a configuration key to pass the name of the test to run?
The --test parameter is not enough to avoid all the errors: [stack@director ~]$ sudo rhcert-ci run --test cinder --key-file /home/stack/key.file --debug high --mode auto (...) setting testserver to IP 127.0.0.1 Test Parameters: outputfile=/var/log/rhcert/runs/1/openstack/cinder/output.log testserver=127.0.0.1 Checking installed rpms: openstack-tempest-13.0.0-9.bafe630git.el7ost.noarch sos-3.3-5.el7_3.noarch python-neutron-tests-9.1.1-2.el7ost.noarch python-neutron-lbaas-tests-9.1.0-2.el7ost.noarch python-sahara-tests-5.0.0-4.el7ost.noarch python-tempest-lib-1.0.0-1.el7ost.noarch All required packages installed Subtest: Configuration - Check tempest configuration Configuring tempest Error: Exception raised in test openstack/cinder: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/rhcert/client/harness.py", line 1055, in runTest rv = test.run() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 473, in run description="Check tempest configuration")) File "/usr/lib/python2.7/site-packages/rhcert/test.py", line 363, in runSubTest result = subtestFunction() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 306, in pre_run self.map_apis_to_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/common.py", line 316, in map_apis_to_tests self.tempest_test_list = tempest_test.get_tempest_mapped_tests() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 42, in get_tempest_mapped_tests self.policy_test_list = self.get_policy_test_list() File "/usr/lib/python2.7/site-packages/rhcert/suites/openstack_cert/tempest_tests.py", line 37, in get_policy_test_list self.policy_test_regex.append(self.cert_type[api]) KeyError: u'vpnaas' copying attachments... checking directory /var/log/rhcert/runs/1/openstack/cinder Skipping output.log Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 128: ordinal not in range(128) Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 128: ordinal not in range(128) Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 128: ordinal not in range(128) Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 128: ordinal not in range(128) Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 130: ordinal not in range(128) Warning: Unicode decode error in /var/log/messages 'ascii' codec can't decode byte 0xe2 in position 130: ordinal not in range(128) Error: Could not find log messages in /var/log/messages! saveOutput: /var/log/rhcert/runs/1/openstack/cinder/output.log Warning: could not merge output XML, reading as text file. no element found: line 33, column 0 Return value was False saved to /var/rhcert/results.xml
My bad, the supported_apis_and_extensions was not correct.