When attempting an operation that requires DNS, and DNS is down, the service throws a 500 exception and returns invalid output. It should instead return an appropriate status code (service unavailable, 503?) and appropriate error message, since in production DYN has proven to be unreliable. In general, when a distributed system is required but unavailable, and it can't be reached to complete an operation we should indicate that to the user. Broker exception trace was: StickShift::DNSException (Error communicating with DNS system. If the problem persists please contact Red Hat support.): lib/express/broker/dns_service.rb:152:in `raise_dns_exception' lib/express/broker/dns_service.rb:128:in `dyn_login' lib/express/broker/dns_service.rb:167:in `dyn_do' lib/express/broker/dns_service.rb:112:in `dyn_login' lib/express/broker/dns_service.rb:84:in `login' lib/express/broker/dns_service.rb:32:in `initialize' Rest returned 500, HTML and an error page (a rails error page) Possible error message in this case might be "Your domain cannot be created because of a temporary outage. Please check the OpenShift status page for information about when this outage will be resolved." Returning a specific exit code would also be useful.
Fixed in https://github.com/openshift/crankcase/pull/142 Waiting for merge
We are now returning 503 instead of 500 for dns failures during application creation/deletion and domain creation/update/deletion.
Checked on devenv_1857, new errno. returned. ===development.log=== [REQ_ID=4fad49e95e6e477e8d885d3c06069e3f] ACTION=LEGACY_BROKER StickShift::DNSException for /broker/cartridge: Error communicating with DNS system. If the problem persists please contact Red Hat support. Completed 503 Service Unavailable in 30278ms (Views: 2.0ms)