Description of problem: Very often (but not always), the REST service refuses to delete a domain with applications in it. Version-Release number of selected component (if applicable): "1.0" How reproducible: often, not always Steps to Reproduce: 1. curl -k --user "<USER>:<PASSWORD>" -H "Accept: application/xml" https://openshift.redhat.com/broker/rest/domains/ -X POST --form id=bingobongo 2. curl -k --user "<USER>:<PASSWORD>" -H "Accept: application/xml" https://openshift.redhat.com/broker/rest/domains/bingobongo/applications/ -X POST -v --form name=as --form cartridge=jbossas-7 3. curl -k --user "<USER>:<PASSWORD>" -H "Accept: application/xml" https://openshift.redhat.com/broker/rest/domains/bingobongo -X DELETE -v --form force=true Actual results: < HTTP/1.1 100 Continue < HTTP/1.1 500 < Cache-Control: no-cache < Content-Type: application/xml; charset=utf-8 < Date: Wed, 27 Feb 2013 19:52:26 GMT < ProxyTime: D=10857264 < Server: Apache/2.2.15 (Red Hat) < Status: 500 < Strict-Transport-Security: max-age=15768000, includeSubDomains < Vary: Accept-Encoding,User-Agent < X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.17 < X-Rack-Cache: invalidate, pass < X-Request-Id: d80f674c6be0c26f879ed66afd52c295 < X-Runtime: 10.727944 < X-UA-Compatible: IE=Edge,chrome=1 < Content-Length: 1045 < Connection: keep-alive < <?xml version="1.0" encoding="UTF-8"?> <response> <status>internal_server_error</status> <type nil="true"></type> <data> <datum nil="true"></datum> </data> <messages> <message> <severity>error</severity> <text> Problem: Cannot delete Domain because of dependent 'applications'. Summary: When defining 'applications' with a :dependent => :restrict, Mongoid will raise an error when attempting to delete the Domain when the child 'applications' still has documents in it. Resolution: Don't attempt to delete the parent Domain when it has children, or change the dependent option on the relation.</text> <exit-code>1</exit-code> <field nil="true"></field> </message> </messages> <version>1.3</version> <supported-api-versions> <supported-api-version>1.0</supported-api-version> <supported-api-version>1.1</supported-api-version> <supported-api-version>1.2</supported-api-version> <supported-api-version>1.3</supported-api-version> </supported-api-versions> </response> Expected results: Should be able to kill the domain with applications within it, since we're using "force=true" Additional info:
It looks like as if the domain was killed even if the above error is returned: curl -k --user "<USER>:<PASSWORD>" -H "Accept: application/xml" https://openshift.redhat.com/broker/rest/domains/ -X GET -v returns < HTTP/1.1 200 < Cache-Control: must-revalidate, private, max-age=0 < Content-Type: application/xml; charset=utf-8 < Date: Wed, 27 Feb 2013 20:13:42 GMT < ETag: "537df1f0ea204c9209cd13d47b8b766e" < ProxyTime: D=642329 < Server: Apache/2.2.15 (Red Hat) < Status: 200 < Strict-Transport-Security: max-age=15768000, includeSubDomains < Vary: Accept-Encoding,User-Agent < X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.17 < X-Rack-Cache: miss < X-Request-Id: af33f94ccae88547a785d1296d4b290f < X-Runtime: 0.635460 < X-UA-Compatible: IE=Edge,chrome=1 < Content-Length: 431 < Connection: keep-alive < <?xml version="1.0" encoding="UTF-8"?> <response> <status>ok</status> <type>domains</type> <data/> <messages/> <version>1.3</version> <supported-api-versions> <supported-api-version>1.0</supported-api-version> <supported-api-version>1.1</supported-api-version> <supported-api-version>1.2</supported-api-version> <supported-api-version>1.3</supported-api-version> </supported-api-versions> </response>
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/ed2a45b2b9a90785a3c064f0d2a3214c4eb86e10 Fix for bug 916323 - making sure that we delete all applications and re-verify before force-deleting a domain
This bug fixed in devenv_2872 1.Create domain and applicaton 2.Force delete domain [qgong@dhcp-10-92 dev]$ curl -k -X DELETE -H 'Accept: application/xml' --user qgong:111111 https://ec2-23-22-17-223.compute-1.amazonaws.com/broker/rest/domains/qgong2 -d force=true 3. No application and domain exist. [qgong@dhcp-10-92 dev]$ rhc domain show In order to deploy applications, you must create a domain with 'rhc setup' or 'rhc domain create' [qgong@dhcp-10-92 dev]$ curl -k -X GET -H 'Accept: application/xml' --user qgong:111111 https://ec2-23-22-17-223.compute-1.amazonaws.com/broker/rest/domains/ -v * About to connect() to ec2-23-22-17-223.compute-1.amazonaws.com port 443 (#0) * Trying 23.22.17.223... connected * Connected to ec2-23-22-17-223.compute-1.amazonaws.com (23.22.17.223) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * warning: ignoring value of ssl.verifyhost * skipping SSL peer certificate verification * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: E=root@ip-10-204-37-235,CN=ip-10-204-37-235,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=-- * start date: Feb 26 18:41:26 2013 GMT * expire date: Feb 26 18:41:26 2014 GMT * common name: ip-10-204-37-235 * issuer: E=root@ip-10-204-37-235,CN=ip-10-204-37-235,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=-- * Server auth using Basic with user 'qgong' > GET /broker/rest/domains/ HTTP/1.1 > Authorization: Basic cWdvbmdAcmVkaGF0LmNvbToxMTExMTE= > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > Host: ec2-23-22-17-223.compute-1.amazonaws.com > Accept: application/xml > < HTTP/1.1 200 < Date: Thu, 28 Feb 2013 06:05:12 GMT < Server: Apache/2.2.15 (Red Hat) < X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.17 < X-OpenShift-Identity: qgong < X-OAuth-Scopes: session < X-UA-Compatible: IE=Edge < ETag: "4ce487b0701df282341c9254c60ade17" < Cache-Control: must-revalidate, private, max-age=0 < X-Request-Id: e8b064521359fa829a9918437851391c < X-Runtime: 0.662419 < X-Rack-Cache: miss < Status: 200 < Content-Length: 486 < Content-Type: application/xml; charset=utf-8 < Vary: Accept-Encoding,User-Agent < Strict-Transport-Security: max-age=15768000 < ProxyTime: D=665347 < <?xml version="1.0" encoding="UTF-8"?> <response> <status>ok</status> <type>domains</type> <data/> <messages/> <version>1.4</version> <supported-api-versions> <supported-api-version>1.0</supported-api-version> <supported-api-version>1.1</supported-api-version> <supported-api-version>1.2</supported-api-version> <supported-api-version>1.3</supported-api-version> <supported-api-version>1.4</supported-api-version> </supported-api-versions> </response> * Connection #0 to host ec2-23-22-17-223.compute-1.amazonaws.com left intact * Closing connection #0
Should be fixed in devenv_2873 Refer https://github.com/openshift/origin-server/pull/1481 Note to QE: To test this bug fix, you should try to force delete a domain with 2-3 applications while simultaneously trying to add an application. The request to add the application should be made a few seconds (3-5) after the rest api call to force delete the domain is made.
Verified on devenv_2878 1.Create 2 applications in domain "qgong2" 2.Force delete this domain in one terminal [qgong@dhcp-10-92 ~]$ curl -k -X DELETE -H 'Accept: application/xml' --user qgong+1:111111 https://ec2-184-73-41-123.compute-1.amazonaws.com/broker/rest/domains/qgong2 -d force=true 3.Wait 3s, then create another applicaion in this domain in another terminal 4.Check result, first this application could created successfully, then the domain could force delete. too. 5.Check info: [qgong@dhcp-10-92 ~]$ curl -k -X GET -H 'Accept: application/xml' --user qgong+1:111111 https://ec2-184-73-41-123.compute-1.amazonaws.com/broker/rest/domains/ -v * About to connect() to ec2-184-73-41-123.compute-1.amazonaws.com port 443 (#0) * Trying 184.73.41.123... connected * Connected to ec2-184-73-41-123.compute-1.amazonaws.com (184.73.41.123) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * warning: ignoring value of ssl.verifyhost * skipping SSL peer certificate verification * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: E=root@ip-10-82-254-165,CN=ip-10-82-254-165,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=-- * start date: Feb 28 06:10:19 2013 GMT * expire date: Feb 28 06:10:19 2014 GMT * common name: ip-10-82-254-165 * issuer: E=root@ip-10-82-254-165,CN=ip-10-82-254-165,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=-- * Server auth using Basic with user 'qgong+1' > GET /broker/rest/domains/ HTTP/1.1 > Authorization: Basic cWdvbmcrMUByZWRoYXQuY29tOjExMTExMQ== > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > Host: ec2-184-73-41-123.compute-1.amazonaws.com > Accept: application/xml > < HTTP/1.1 200 < Date: Fri, 01 Mar 2013 09:24:57 GMT < Server: Apache/2.2.15 (Red Hat) < X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.17 < X-OpenShift-Identity: qgong+1 < X-OAuth-Scopes: session < X-UA-Compatible: IE=Edge < ETag: "537df1f0ea204c9209cd13d47b8b766e" < Cache-Control: must-revalidate, private, max-age=0 < X-Request-Id: 7335f077049c7ad43f75aa9f5ca67cdd < X-Runtime: 0.032176 < X-Rack-Cache: miss < Status: 200 < Content-Length: 431 < Content-Type: application/xml; charset=utf-8 < Vary: Accept-Encoding,User-Agent < Strict-Transport-Security: max-age=15768000 < ProxyTime: D=46984 < <?xml version="1.0" encoding="UTF-8"?> <response> <status>ok</status> <type>domains</type> <data/> <messages/> <version>1.3</version> <supported-api-versions> <supported-api-version>1.0</supported-api-version> <supported-api-version>1.1</supported-api-version> <supported-api-version>1.2</supported-api-version> <supported-api-version>1.3</supported-api-version> </supported-api-versions> </response> * Connection #0 to host ec2-184-73-41-123.compute-1.amazonaws.com left intact * Closing connection #0