Bug 806713

Summary:	REST API throws 500 when DNS service is unavailable due to outage
Product:	OKD	Reporter:	Clayton Coleman <ccoleman>
Component:	Pod	Assignee:	Abhishek Gupta <abhgupta>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	medium
Version:	1.x	CC:	abhgupta, bmeng, lnader, mpatel
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-06-25 18:26:23 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Clayton Coleman 2012-03-26 04:10:57 UTC

When attempting an operation that requires DNS, and DNS is down, the service throws a 500 exception and returns invalid output.  It should instead return an appropriate status code (service unavailable, 503?) and appropriate error message, since in production DYN has proven to be unreliable.  In general, when a distributed system is required but unavailable, and it can't be reached to complete an operation we should indicate that to the user.  

Broker exception trace was:
StickShift::DNSException (Error communicating with DNS system.  If the problem persists please contact Red Hat support.):
  lib/express/broker/dns_service.rb:152:in `raise_dns_exception'
  lib/express/broker/dns_service.rb:128:in `dyn_login'
  lib/express/broker/dns_service.rb:167:in `dyn_do'
  lib/express/broker/dns_service.rb:112:in `dyn_login'
  lib/express/broker/dns_service.rb:84:in `login'
  lib/express/broker/dns_service.rb:32:in `initialize'

Rest returned 500, HTML and an error page (a rails error page)

Possible error message in this case might be

"Your domain cannot be created because of a temporary outage.  Please check the OpenShift status page for information about when this outage will be resolved."

Returning a specific exit code would also be useful.

Comment 1 Abhishek Gupta 2012-06-18 23:34:37 UTC

Fixed in https://github.com/openshift/crankcase/pull/142
Waiting for merge

Comment 2 Abhishek Gupta 2012-06-19 00:43:46 UTC

We are now returning 503 instead of 500 for dns failures during application creation/deletion and domain creation/update/deletion.

Comment 3 Meng Bo 2012-06-19 06:58:38 UTC

Checked on devenv_1857, new errno. returned.

===development.log===

[REQ_ID=4fad49e95e6e477e8d885d3c06069e3f] ACTION=LEGACY_BROKER StickShift::DNSException for /broker/cartridge: Error communicating with DNS system.  If the problem persists please contact Red Hat support.
Completed 503 Service Unavailable in 30278ms (Views: 2.0ms)