Created attachment 1423927 [details] rhcert_cli log Description of problem: when I run manila certification test, I found the output.log is incorrect: [root@osp10unityctl0 openstack]# cat manila_shares/output.log ... Following tests have been executed: +--------------------------------------------------+----------+----------+----------+----------+ |Test |Total |Passed |Failed |Skipped | +--------------------------------------------------+----------+----------+----------+----------+ |test_microversions |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_share_networks |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_extensions |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_shares_actions_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_share_instances |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_rules_negative |5 |0 |0 |5 | +--------------------------------------------------+----------+----------+----------+----------+ |test_shares |3 |0 |0 |3 | +--------------------------------------------------+----------+----------+----------+----------+ |test_shares_actions |1 |0 |1 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_metadata |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_shares_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_availability_zones |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_quotas_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_availability_zones_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_security_services_mapping_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_quotas |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_share_networks_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_metadata_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_security_services |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_limits |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_security_services_negative |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_security_services_mapping |0 |0 |0 |0 | +--------------------------------------------------+----------+----------+----------+----------+ |test_rules |5 |0 |0 |5 | +--------------------------------------------------+----------+----------+----------+----------+ ... From the validation report the result is correct: failures": 1, "tests": 306, "errors": 0, "success": 267, "skipped": 38 [root@osp10unityctl0 manila_shares]# less manila_shares-validation_report.json {"total": {"failures": 1, "tests": 306, "errors": 0, "success": 267, "skipped": 38}, "test_cases": {"manila_tempest_tests.tests.api.test_rules_negative.ShareIpRulesForNFSNegativeTest.test_create_duplicate_of_ip_rule_2_2_9[api_with_backend,negative]": {"status": "OK", "output": "manila_tempest_tests.tests.api.test_rules_negative.ShareIpRulesForNFSNegativeTest.test_create_duplicate_of_ip_rule_2_2_9[api_with_backend,negative]", "name": "manila_tempest_tests.tests.api.test_rules_negative.ShareIpRulesForNFSNegativeTest.test_create_duplicate_of_ip_rule_2_2_9[api_with_backend,negative]"}, "manila_tempest_tests.tests.api.test_shares_actions_negative.SharesActionsNegativeTest.test_share_extend_with_invalid_share_state[api_with_backend,negative]": {"status": "OK", "output": "manila_tempest_tests.tests.api.test_shares_actions_negative.SharesActionsNegativeTest.test_share_extend_with_invalid_share_state[api_with_backend,negative]", "name": "manila_tempest_tests.tests.api.test_shares_actions_negative.SharesActionsNegativeTest.test_share_extend_with_invalid_share_state[api_with_backend,negative]"}, "manila_tempest_tests.tests.api.test_shares.SharesNFSTest.test_create_get_delete_share[backend,positive]": {"status": "OK", "output": "manila_tempest_tests.tests.api.test_shares.SharesNFSTest.test_create Version-Release number of selected component (if applicable): redhat-certification-backend-5.10-20180330.2.el7.noarch redhat-certification-openstack-5.10-20180330.2.el7.noarch redhat-certification-5.10-20180330.2.el7.noarch How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Certification team will try to fix this in upcoming releases of rhcert, this looks like a minor parsing issue and should not block any cert as the information is also available in validation_report Thanks, Anshul Behl
This issue will be fixed in redhat-certification-openstack-5.12 version. The release is scheduled in next week. Thanks, Pooja
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1590
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days