Bug 1705730
| Summary: | [OSP15] TASK [Run tripleo-container-image-prepare logged to /var/log/tripleo-container-image-prepare.log] failed with Exception occured while running the command and without explanation | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Artem Hrechanychenko <ahrechan> |
| Component: | rhosp-director | Assignee: | Emilien Macchi <emacchi> |
| Status: | CLOSED DUPLICATE | QA Contact: | Sasha Smolyak <ssmolyak> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 15.0 (Stein) | CC: | aschultz, dbecker, emacchi, mburns, mcornea, morazi |
| Target Milestone: | beta | Keywords: | Reopened, Triaged |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-05-03 22:02:10 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
You need to use ContainerImagePrepareDebug: True I don't understand where the failure is, I checked the logs and see: ######################################################## Deployment successful! ######################################################## in home/stack/install-undercloud.log. Please run the job again with: - undercloud.conf with undercloud_debug=true - overcloud environment with ContainerImagePrepareDebug: True Overcloud deployment failed, not Undercloud. Ok, I'll try to re-run job with ContainerImagePrepareDebug: True for overcloud Adding more details from such environment. It looks like the overcloud deploy command failed for some reason but deployment is still ongoing:
(undercloud) [stack@undercloud-0 ~]$ cat overcloud_deployment_91.log (--log-file overcloud_deployment_91.log)
2019-05-03 16:05:29.807 112185 WARNING tripleoclient.plugin [ admin] Waiting for messages on queue 'tripleo' with no timeout.
2019-05-03 16:32:57.250 112185 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [ admin] Exception occured while running the command
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 30, in run
super(Command, self).run(parsed_args)
File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
return_code = self.take_action(parsed_args) or 0
File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 958, in take_action
verbosity=self.app_args.verbose_level)
File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 327, in config_download
raise exceptions.DeploymentError("Overcloud configuration failed.")
tripleoclient.exceptions.DeploymentError: Overcloud configuration failed.
2019-05-03 16:32:57.257 112185 ERROR openstack [ admin] Overcloud configuration failed.
*but*
[root@undercloud-0 stack]# tail -f /var/lib/mistral/overcloud/ansible.log
2019-05-03 16:46:19,003 p=384 u=mistral | TASK [Calculate container_puppet_tasks for controller-0 step 3] ****************
2019-05-03 16:46:19,003 p=384 u=mistral | Friday 03 May 2019 16:46:19 -0400 (0:00:00.477) 0:39:56.214 ************
2019-05-03 16:46:19,088 p=384 u=mistral | ok: [controller-0] => (item={'config_image': '192.168.24.1:8787/rhosp15/openstack-keystone:20190423.1', 'config_volume': 'keystone_init_tasks', 'puppet_tags': 'keystone_config,keystone_domain_config,keystone_endpoint,keystone_identity_provider,keystone_paste_ini,keystone_role,keystone_service,keystone_tenant,keystone_user,keystone_user_role,keystone_domain', 'service_name': 'keystone', 'step_config': 'include ::tripleo::profile::base::keystone'}) => {"ansible_facts": {"host_container_puppet_tasks": [{"config_image": "192.168.24.1:8787/rhosp15/openstack-keystone:20190423.1", "config_volume": "keystone_init_tasks", "puppet_tags": "keystone_config,keystone_domain_config,keystone_endpoint,keystone_identity_provider,keystone_paste_ini,keystone_role,keystone_service,keystone_tenant,keystone_user,keystone_user_role,keystone_domain", "service_name": "keystone", "step_config": "include ::tripleo::profile::base::keystone"}]}, "ansible_loop_var": "item", "changed": false, "item": {"config_image": "192.168.24.1:8787/rhosp15/openstack-keystone:20190423.1", "config_volume": "keystone_init_tasks", "puppet_tags": "keystone_config,keystone_domain_config,keystone_endpoint,keystone_identity_provider,keystone_paste_ini,keystone_role,keystone_service,keystone_tenant,keystone_user,keystone_user_role,keystone_domain", "service_name": "keystone", "step_config": "include ::tripleo::profile::base::keystone"}}
2019-05-03 16:46:19,209 p=384 u=mistral | TASK [Write container-puppet-tasks json file for controller-0 step 3] **********
2019-05-03 16:46:19,209 p=384 u=mistral | Friday 03 May 2019 16:46:19 -0400 (0:00:00.206) 0:39:56.420 ************
2019-05-03 16:46:19,329 p=384 u=mistral | skipping: [compute-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
2019-05-03 16:46:20,254 p=384 u=mistral | changed: [controller-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
2019-05-03 16:46:20,353 p=384 u=mistral | TASK [Run container-puppet tasks (bootstrap tasks) for step 3] *****************
2019-05-03 16:46:20,353 p=384 u=mistral | Friday 03 May 2019 16:46:20 -0400 (0:00:01.143) 0:39:57.564 ************
2019-05-03 16:46:20,474 p=384 u=mistral | skipping: [compute-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
and deployment is ongoing
*** This bug has been marked as a duplicate of bug 1700044 *** |
Description of problem: Overcloud deployment failed: ... TASK [Run tripleo-container-image-prepare logged to /var/log/tripleo-container-image-prepare.log] *** Thursday 02 May 2019 18:09:58 +0000 (0:00:00.646) 0:06:02.272 ********** changed: [undercloud] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 30, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 958, in take_action verbosity=self.app_args.verbose_level) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 327, in config_download raise exceptions.DeploymentError("Overcloud configuration failed.") tripleoclient.exceptions.DeploymentError: Overcloud configuration failed. Overcloud configuration failed. no other explanation /var/log/tripleo-container-image-prepare.log doesn't contains any error messages Version-Release number of selected component (if applicable): core_puddle: RHOS_TRUNK-15.0-RHEL-8-20190423.n.1 How reproducible: always Steps to Reproduce: 1. Deploy OSP15 with 3controller and 3 ceph nodes Actual results: TASK [Run tripleo-container-image-prepare logged to /var/log/tripleo-container-image-prepare.log] failed without any explanation of failure Expected results: pass Additional info: all tarballs for undercloud and other nodes located here - https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DF%20Current%20release/job/DFG-df-deployment-15-virthost-3cont_1comp_3ceph-yes_UC_SSL-no_OC_SSL-scalecompute-ceph-ipv4-geneve-RHELOSP-31842/3/artifact/