Description of problem: "instance-ha" job is failing with an ansible timeout error in "udercloud-0/var/log/containers/mistral/mistral-db-manage.log": 2019-07-22 21:10:39.111 13 ERROR mistral.actions.openstack.action_generator.base [-] Failed to create action: nova.certs_convert_into_with_meta: AttributeError: 'Client' object has no attribute 'certs' Additional info: Jenkins Job: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/pidone/view/instance-ha/job/DFG-pidone-instance-ha-15_director-rhel-virthost-3cont_2comp_3ceph-ipv4-geneve-instance-ha-test-suite/9/console
(In reply to Cesar Fernandez from comment #0) > Description of problem: > > "instance-ha" job is failing with an ansible timeout error in > "udercloud-0/var/log/containers/mistral/mistral-db-manage.log": > > 2019-07-22 21:10:39.111 13 ERROR > mistral.actions.openstack.action_generator.base [-] Failed to create action: > nova.certs_convert_into_with_meta: AttributeError: 'Client' object has no > attribute 'certs' > > Additional info: > > Jenkins Job: > https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/pidone/ > view/instance-ha/job/DFG-pidone-instance-ha-15_director-rhel-virthost- > 3cont_2comp_3ceph-ipv4-geneve-instance-ha-test-suite/9/console that ERROR message is due to https://bugzilla.redhat.com/show_bug.cgi?id=1709566. As for the actual failure I am going to have a look :)
here the step where it hanged/failed: TASK [configure tmpwatch on the host] ****************************************** Monday 22 July 2019 22:09:01 +0000 (0:00:00.552) 0:12:38.772 *********** skipping: [controller-0] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [controller-1] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [controller-2] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [overcloud-novacomputeiha-0] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [overcloud-novacomputeiha-1] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [tripleo-container-tag : Pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1 image] *** Monday 22 July 2019 22:09:02 +0000 (0:00:00.353) 0:12:39.125 *********** skipping: [overcloud-novacomputeiha-0] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [overcloud-novacomputeiha-1] => {"changed": false, "skip_reason": "Conditional result was False"} changed: [controller-0] => {"changed": true, "cmd": "podman pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1", "delta": "0:00:06.547186", "end": "2019-07-22 22:09:08.906103", "rc": 0, "start": "2019-07-22 22:09:02.358917", "stderr": "Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1...Getting image source signatures\nCopying blob sha256:3fd1d6bddf52f5739ebe7a52906714c0142e430b2305e8e9e663ee8d19c5d5fd\nCopying blob sha256:3b3d402417deac332eda774ac4b9fbc0e7f4da02558b0b3a0ba54dc283203047\nCopying blob sha256:6bc8d2c3e35d7762fdc45ab9ac14c152b308d183680ac0b5bb85f17bd0962dbf\nCopying blob sha256:0db9c662d67b89727a5a3a6aebbf04c67b1d711e6ac133f34634d10da67822e4\nCopying blob sha256:3dc708edb94b8390cd03c6e55a89cca0483cec8d9be8db5b2d08f1eb95d1012e\nCopying blob sha256:3506f1c05051545bc85d863feb4c990abe6735fb09fea7280854d4d92c9c881c\nCopying config sha256:134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1\nWriting manifest to image destination\nStoring signatures", "stderr_lines": ["Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1...Getting image source signatures", "Copying blob sha256:3fd1d6bddf52f5739ebe7a52906714c0142e430b2305e8e9e663ee8d19c5d5fd", "Copying blob sha256:3b3d402417deac332eda774ac4b9fbc0e7f4da02558b0b3a0ba54dc283203047", "Copying blob sha256:6bc8d2c3e35d7762fdc45ab9ac14c152b308d183680ac0b5bb85f17bd0962dbf", "Copying blob sha256:0db9c662d67b89727a5a3a6aebbf04c67b1d711e6ac133f34634d10da67822e4", "Copying blob sha256:3dc708edb94b8390cd03c6e55a89cca0483cec8d9be8db5b2d08f1eb95d1012e", "Copying blob sha256:3506f1c05051545bc85d863feb4c990abe6735fb09fea7280854d4d92c9c881c", "Copying config sha256:134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1", "Writing manifest to image destination", "Storing signatures"], "stdout": "134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1", "stdout_lines": ["134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1"]} changed: [controller-1] => {"changed": true, "cmd": "podman pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1", "delta": "0:00:06.514072", "end": "2019-07-22 22:09:08.945439", "rc": 0, "start": "2019-07-22 22:09:02.431367", "stderr": "Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1...Getting image source signatures\nCopying blob sha256:3fd1d6bddf52f5739ebe7a52906714c0142e430b2305e8e9e663ee8d19c5d5fd\nCopying blob sha256:3b3d402417deac332eda774ac4b9fbc0e7f4da02558b0b3a0ba54dc283203047\nCopying blob sha256:6bc8d2c3e35d7762fdc45ab9ac14c152b308d183680ac0b5bb85f17bd0962dbf\nCopying blob sha256:0db9c662d67b89727a5a3a6aebbf04c67b1d711e6ac133f34634d10da67822e4\nCopying blob sha256:3dc708edb94b8390cd03c6e55a89cca0483cec8d9be8db5b2d08f1eb95d1012e\nCopying blob sha256:3506f1c05051545bc85d863feb4c990abe6735fb09fea7280854d4d92c9c881c\nCopying config sha256:134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1\nWriting manifest to image destination\nStoring signatures", "stderr_lines": ["Trying to pull 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190715.1...Getting image source signatures", "Copying blob sha256:3fd1d6bddf52f5739ebe7a52906714c0142e430b2305e8e9e663ee8d19c5d5fd", "Copying blob sha256:3b3d402417deac332eda774ac4b9fbc0e7f4da02558b0b3a0ba54dc283203047", "Copying blob sha256:6bc8d2c3e35d7762fdc45ab9ac14c152b308d183680ac0b5bb85f17bd0962dbf", "Copying blob sha256:0db9c662d67b89727a5a3a6aebbf04c67b1d711e6ac133f34634d10da67822e4", "Copying blob sha256:3dc708edb94b8390cd03c6e55a89cca0483cec8d9be8db5b2d08f1eb95d1012e", "Copying blob sha256:3506f1c05051545bc85d863feb4c990abe6735fb09fea7280854d4d92c9c881c", "Copying config sha256:134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1", "Writing manifest to image destination", "Storing signatures"], "stdout": "134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1", "stdout_lines": ["134cbb7d77dc9efab004099fb8432c85fc44baaf3cc5f47599499faafed85cf1"]} Overcloud configuration failed. Ansible timed out at 5259 seconds. looks like something happened on controller-2, unfortunately infrared did not grab anything from that vm. Can you try rerunning this job? Thanks Luca