Bug 1357296

Summary: v3 API | Failed to delete a specified storage domain in the system
Product: [oVirt] ovirt-engine Reporter: Raz Tamir <ratamir>
Component: RestAPIAssignee: Juan Hernández <juan.hernandez>
Status: CLOSED CURRENTRELEASE QA Contact: Raz Tamir <ratamir>
Severity: urgent Docs Contact:
Priority: high    
Version: 4.0.1.1CC: bugs, juan.hernandez, lsvaty, mgoldboi, oliel, oourfali, ratamir, sbonazzo
Target Milestone: ovirt-4.0.2Keywords: Automation, AutomationBlocker, Regression
Target Release: 4.0.2.1Flags: rule-engine: ovirt-4.0.z+
rule-engine: blocker+
mgoldboi: planning_ack+
oourfali: devel_ack+
lsvaty: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-12 14:30:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
server log none

Description Raz Tamir 2016-07-17 21:53:05 UTC
Description of problem:
* Similar issue was fixed in bug #1340164 *

When trying to delete a storage domain using the REST API, the return code is 500 Internal Server Error and the response body is:
<html>
    <head>
        <title>Error</title>
    </head>
    <body>Internal Server Error</body>
</html>

The call is - DELETE ovirt-engine/api/storagedomains/3c0a32b7-0ec1-4dd5-a1e4-54156a4753b4 body:
<storage_domain>
    <host id="724a18e4-bd69-42d9-a547-9f3aece0d0f4"/>
    <format>false</format>
</storage_domain>

In rsdl, the parameter set is:
<parameters_set>
 <parameter required="false" type="xs:boolean" context="matrix">
  <name>force</name>
  <value>true|false</value>
 </parameter>
 <parameter required="true" type="xs:string" context="matrix">
  <name>destroy</name>
  <value>the id or name of the host</value>
 </parameter>
 <parameter required="false" type="xs:boolean" context="matrix">
  <name>async</name>
  <value>true|false</value>
 </parameter>
</parameters_set>

So it seems there is a problem here as well - the parameter name is 'destroy' instead of host

No logs attached because the request don't even get's to the engine 


Version-Release number of selected component (if applicable):
rhevm-4.0.1.1-0.1.el7ev.noarch


How reproducible:
100%

Steps to Reproduce:
- Have a detached storage domain
1. Send DELETE request to ovirt-engine/api/storagedomains/<sd_id> with body:
<storage_domain>
    <host id="724a18e4-bd69-42d9-a547-9f3aece0d0f4"/>
    <format>false</format>
</storage_domain>
2.
3.

Actual results:
500 Internal Server Error and the response body is:
<html>
    <head>
        <title>Error</title>
    </head>
    <body>Internal Server Error</body>
</html>

Expected results:


Additional info:

Comment 1 Red Hat Bugzilla Rules Engine 2016-07-18 05:24:11 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 2 Ori Liel 2016-07-18 07:26:12 UTC
Is the call made with version 3 or version 4 of the API?

Comment 3 Raz Tamir 2016-07-18 08:08:53 UTC
v3, look at the header of this bug

Comment 4 Ori Liel 2016-07-18 12:03:45 UTC
I can't seem to reproduce this, 

I try: 

DELETE    http://localhost:8080/ovirt-engine/api/v3/storagedomains/072fbaa1-08f3-4a40-9f34-a5ca22dd1d74

Content-Type: application/xml

Body: 
<storage_domain>
     <host id="17b817e1-2cce-4c29-a275-0174c53ea211"/>
     <format>false</format>
</storage_domain>

And the request reaches the engine just fine. 

- Could it be that the fix for issue 1340164 that you mentioned, solves this too? the code change seems to be very relevant
- An error resulting in 500: Internal Server Error should probably leave a trace in server.log, can you please attach it? 

Thanks

Comment 5 Raz Tamir 2016-07-20 14:04:42 UTC
Ori,
This bug introduced after bug #1340164 was fixed.
I am able to reproduce this on 3 different environments with the following packages:

ovirt-vmconsole-1.0.4-1.el7ev.noarch
ovirt-engine-dashboard-1.0.0-20160705git3e1e994.el7ev.x86_64
ovirt-engine-setup-plugin-ovirt-engine-4.0.1.1-0.1.el7ev.noarch
ovirt-host-deploy-java-1.5.1-1.el7ev.noarch
ovirt-engine-dwh-setup-4.0.1-1.el7ev.noarch
ovirt-host-deploy-1.5.1-1.el7ev.noarch
ovirt-engine-restapi-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-extension-aaa-ldap-setup-1.2.1-1.el7ev.noarch
python-ovirt-engine-sdk4-4.0.0-0.1.a3.el7ev.x86_64
ovirt-engine-setup-base-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-websocket-proxy-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-vmconsole-proxy-helper-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-sdk-python-3.6.7.0-1.el7ev.noarch
ovirt-engine-extension-aaa-jdbc-1.1.0-1.el7ev.noarch
ovirt-engine-lib-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-setup-plugin-websocket-proxy-4.0.1.1-0.1.el7ev.noarch
ovirt-vmconsole-proxy-1.0.4-1.el7ev.noarch
ovirt-engine-backend-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-webadmin-portal-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-extension-aaa-misc-1.0.1-2.el7ev.noarch
ovirt-iso-uploader-4.0.0-1.el7ev.noarch
ovirt-engine-cli-3.6.7.0-1.el7ev.noarch
ovirt-engine-dwh-4.0.1-1.el7ev.noarch
ovirt-engine-dbscripts-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-userportal-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-sdk-java-3.6.7.0-1.el7ev.noarch
ovirt-log-collector-4.0.0-1.el7ev.noarch
ovirt-engine-tools-backup-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-extension-aaa-ldap-1.2.1-1.el7ev.noarch
ovirt-setup-lib-1.0.2-1.el7ev.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-setup-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-tools-4.0.1.1-0.1.el7ev.noarch
ovirt-engine-extensions-api-impl-4.0.1.1-0.1.el7ev.noarch
ovirt-image-uploader-4.0.0-1.el7ev.noarch

Comment 6 Oved Ourfali 2016-07-21 08:16:46 UTC
Raz, can you attach the logs as Ori requested?

Comment 7 Raz Tamir 2016-07-21 08:50:17 UTC
Created attachment 1182403 [details]
server log

server.log attached

Comment 8 Juan Hernández 2016-07-21 09:00:00 UTC
This is a real and important issue. The relevant error message is the following:

Caused by: java.lang.UnsupportedOperationException
        at org.jboss.resteasy.specimpl.UnmodifiableMultivaluedMap.putSingle(UnmodifiableMultivaluedMap.java:29) [resteasy-jaxrs.jar:3.0.17.Final-redhat-1]
        at org.ovirt.engine.api.v3.servers.V3StorageDomainServer.remove(V3StorageDomainServer.java:81) [restapi-jaxrs.jar:]

In the V3 compatibility layer we modify the map of input parameters. For example, we take the "<format>true</format>" parameter from the body of the V3 response and convert into a "?format=true" query parameter. The version of JBoss that we used to try to reproduce works correctly, but the version used by Raz doesn't work:

  eap7-wildfly-7.0.1-4.GA_redhat_2.1.ep7.el7.noarch

That is probably because a change in the implementation of Resteasy. I'm checking it.

Comment 9 Juan Hernández 2016-07-21 09:18:00 UTC
This error is a side effect of the following change in Resteasy:

  UriInfo.getQueryParameters() should return an immutable map
  https://issues.jboss.org/browse/RESTEASY-1331

This was introduced in version 3.0.17.Final of Resteasy, which is part of JBoss EAP 7.0.1.

We need to modify our code so that it doesn't try to modify these now immutable maps.

Comment 10 Oved Ourfali 2016-07-21 11:42:46 UTC
4.0.2 backport is needed.

Comment 11 Raz Tamir 2016-07-31 11:26:40 UTC
Verified on ovirt-engine-4.0.2.2-0.1.el7ev.noarch.
The storage domain deleted successfully from REST API v3