Bug 1303321
| Summary: | ospd-8 poodle 2016-01-29.4 deployment fails w/ Node failed to get image for deploy | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | wes hayutin <whayutin> |
| Component: | openstack-ironic | Assignee: | Lucas Alvares Gomes <lmartins> |
| Status: | CLOSED NOTABUG | QA Contact: | Toure Dunnon <tdunnon> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 8.0 (Liberty) | CC: | dtantsur, mburns, rhel-osp-director-maint, yeylon |
| Target Milestone: | --- | Keywords: | Automation, AutomationBlocker |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-02-02 13:51:29 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Also see the following error in the ironic-api log 2015-12-08 22:20:10.034 27416 DEBUG keystonemiddleware.auth_token [-] Storing token in cache store /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_cache.py:170 2015-12-08 22:20:10.035 27416 DEBUG keystonemiddleware.auth_token [-] Received request from user: user_id 274ec72cac704533a101b1d4d01d8b4f, project_id 66f8d9954f024550a6c6dd192f51e77c, roles admin,_member_ process_request /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py:759 2015-12-08 22:20:10.055 27416 CRITICAL ironic [-] IOError: [Errno 32] Broken pipe 2015-12-08 22:20:10.055 27416 ERROR ironic Traceback (most recent call last): 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/bin/ironic-api", line 10, in <module> 2015-12-08 22:20:10.055 27416 ERROR ironic sys.exit(main()) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/ironic/cmd/api.py", line 64, in main 2015-12-08 22:20:10.055 27416 ERROR ironic wsgi.serve_forever() 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib64/python2.7/SocketServer.py", line 236, in serve_forever 2015-12-08 22:20:10.055 27416 ERROR ironic poll_interval) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib64/python2.7/SocketServer.py", line 155, in _eintr_retry 2015-12-08 22:20:10.055 27416 ERROR ironic return func(*args) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/eventlet/green/select.py", line 83, in select 2015-12-08 22:20:10.055 27416 ERROR ironic return hub.switch() 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2015-12-08 22:20:10.055 27416 ERROR ironic return self.greenlet.switch() 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in run 2015-12-08 22:20:10.055 27416 ERROR ironic self.fire_timers(self.clock()) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 461, in fire_timers 2015-12-08 22:20:10.055 27416 ERROR ironic self.squelch_timer_exception(timer, sys.exc_info()) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 385, in squelch_timer_exception 2015-12-08 22:20:10.055 27416 ERROR ironic traceback.print_exception(*exc_info) 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib64/python2.7/traceback.py", line 124, in print_exception 2015-12-08 22:20:10.055 27416 ERROR ironic _print(file, 'Traceback (most recent call last):') 2015-12-08 22:20:10.055 27416 ERROR ironic File "/usr/lib64/python2.7/traceback.py", line 13, in _print 2015-12-08 22:20:10.055 27416 ERROR ironic file.write(str+terminator) 2015-12-08 22:20:10.055 27416 ERROR ironic IOError: [Errno 32] Broken pipe 2015-12-08 22:20:10.055 27416 ERROR ironic Was not able to reproduce with my images, will try yours. P.S. The ironic-api message is worrying, but I don't think it's related. We've found the root cause of the issue to be a bug in the rhos-release rpm. The invocation of `rhos-release -r 7.2 -t /home/stack/DIB -d -P 8-director` is not setting the repos in the DIB dir to be poodles, the repos installed in /etc/yum.repos.d/ *are* set to poodles. This caused the confusion. |
Description of problem: ospd-8 poodle build candidate fails to deploy properly. Error appears to be the following. an 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor [-] Asynchronous exception for node 77ad67b6-e7e1-47d7-ab45-78aa2bbcce86: Node failed to get image for deploy. exception: Failed to deploy instance: Failed to start the iSCSI target to deploy the node 77ad67b6-e7e1-47d7-ab45-78aa2bbcce86. Error: {u'message': u'[Errno 2] No such file or directory', u'code': 500, u'type': u'OSError', u'details': u''} Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last): Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/agent_base_vendor.py", line 345, in heartbeat Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor self.continue_deploy(task, **kwargs) Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 134, in wrapper Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor return f(*args, **kwargs) Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 873, in continue_deploy Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor uuid_dict_returned = do_agent_iscsi_deploy(task, self._client) Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 443, in do_agent_iscsi_deploy Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor raise exception.InstanceDeployFailure(reason=msg) Jan 30 13:09:51 instack.localdomain ironic-conductor[7259]: 2016-01-30 13:09:51.251 7259 ERROR ironic.drivers.modules.agent_base_vendor InstanceDeployFailure: Failed to deploy instance: Failed to start the iSCSI target to deploy the node 77ad67b6-e7e1-47d7-ab45-78aa2bbcce86. Error: {u'message': u'[Errno 2] No such file or directory', u'code': 500, u'type': u'OSError', u'details': u''}