Bug 1370942

Summary: ironic-api crashes with broken pipe while deploying the overcloud in Liberty
Product: [Community] RDO Reporter: David Hill <dhill>
Component: openstack-ironicAssignee: Dmitry Tantsur <dtantsur>
Status: CLOSED CURRENTRELEASE QA Contact: Toure Dunnon <tdunnon>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: LibertyCC: dhill
Target Milestone: ---   
Target Release: Mitaka   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-03 15:53:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description David Hill 2016-08-28 18:21:20 UTC 
Description of problem:

ironic-api crashes with broken pipe while deploying the overcloud in Liberty
2016-08-28 12:32:19.062 9902 CRITICAL ironic [-] IOError: [Errno 32] Broken pipe
2016-08-28 12:32:19.062 9902 ERROR ironic Traceback (most recent call last):
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/bin/ironic-api", line 10, in <module>
2016-08-28 12:32:19.062 9902 ERROR ironic     sys.exit(main())
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/ironic/cmd/api.py", line 64, in main
2016-08-28 12:32:19.062 9902 ERROR ironic     wsgi.serve_forever()
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib64/python2.7/SocketServer.py", line 236, in serve_forever
2016-08-28 12:32:19.062 9902 ERROR ironic     poll_interval)
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib64/python2.7/SocketServer.py", line 155, in _eintr_retry
2016-08-28 12:32:19.062 9902 ERROR ironic     return func(*args)
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/eventlet/green/select.py", line 83, in select
2016-08-28 12:32:19.062 9902 ERROR ironic     return hub.switch()
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-08-28 12:32:19.062 9902 ERROR ironic     return self.greenlet.switch()
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in run
2016-08-28 12:32:19.062 9902 ERROR ironic     self.fire_timers(self.clock())
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 461, in fire_timers
2016-08-28 12:32:19.062 9902 ERROR ironic     self.squelch_timer_exception(timer, sys.exc_info())
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 385, in squelch_timer_exception
2016-08-28 12:32:19.062 9902 ERROR ironic     traceback.print_exception(*exc_info)
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib64/python2.7/traceback.py", line 124, in print_exception
2016-08-28 12:32:19.062 9902 ERROR ironic     _print(file, 'Traceback (most recent call last):')
2016-08-28 12:32:19.062 9902 ERROR ironic   File "/usr/lib64/python2.7/traceback.py", line 13, in _print
2016-08-28 12:32:19.062 9902 ERROR ironic     file.write(str+terminator)
2016-08-28 12:32:19.062 9902 ERROR ironic IOError: [Errno 32] Broken pipe

Version-Release number of selected component (if applicable):


How reproducible:
Once in a while

Steps to Reproduce:
1. Try deploying the overcloud and wait
2.
3.

Actual results:
Sometimes ironic-api will exit , sometimes it will complete

Expected results:
No crash

Additional info:
Comment 1 David Hill 2016-08-28 18:22:37 UTC 
This appears to be solved in https://bugzilla.redhat.com/show_bug.cgi?id=1285470 but not in RDO.
Comment 2 Dmitry Tantsur 2016-10-03 15:53:47 UTC 
Presumably fixed in Mitaka with move to oslo.service.