Bug 1264807

Summary: API error responses inconsistent between Foreman and Katello
Product: Red Hat Satellite Reporter: Peter Vreman <peter.vreman>
Component: API - ContentAssignee: Amit Karsale <akarsale>
Status: CLOSED NOTABUG QA Contact: Katello QA List <katello-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.1.2CC: bbuckingham, brubisch, jomitsch, omaciel
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-23 13:47:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1122832    

Description Peter Vreman 2015-09-21 09:14:50 UTC
Description of problem:
The delete repository is an background action. The API user is not informed which task id is executed or alternatively that the delete action was successful.

This makes it for automation impossible to know that the delete was successful or wait for the delete action to complete.

# curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/katello/api/v2/repositories/200
{}



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create Product TestProduct1
2. Create yum Repo TestRepo1
3. API call to Delete Repo TestRepo1


Actual results:
Empty response

Expected results:
Response with task id
Alternative is to have a response with confirmation that the destroy succeeded.

Additional info:

Comment 1 Peter Vreman 2015-09-21 09:17:06 UTC
The patch below makes the delete explicitly an async action and returns the task id.

--- repositories_controller.rb.old      2015-09-21 08:11:46.392825699 +0000
+++ repositories_controller.rb  2015-09-21 08:16:04.847305624 +0000
@@ -150,8 +150,8 @@
     api :DELETE, "/repositories/:id", N_("Destroy a custom repository")
     param :id, :identifier, :required => true
     def destroy
-      sync_task(::Actions::Katello::Repository::Destroy, @repository)
-      respond_for_destroy
+      task = async_task(::Actions::Katello::Repository::Destroy, @repository)
+      respond_for_async :resource => task
     end

     api :POST, "/repositories/sync_complete"


The response is not usable and contains:

# curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/katello/api/v2/repositories/621
{"id":"d1fa0f8f-e184-4150-b025-e221fff06971","label":"Actions::Katello::Repository::Destroy","pending":true,"username":"hoici","started_at":"2015-09-21T09:14:03Z","ended_at":null,"state":"running","result":"pending","progress":0.0,"input":{"repository":{"id":621,"name":"TestRepo1","label":"TestRepo1"},"product":{"id":39,"name":"Testproduct","label":"Testproduct","cp_id":"1442822962548"},"provider":{"id":3,"name":"Anonymous"},"organization":{"id":3,"name":"Hilti","label":"Hilti"},"services_checked":["pulp","pulp_auth","candlepin","candlepin_auth","elasticsearch"],"user_id":4,"planned_destroy":false,"remote_user":"hoici-59c70e2b","remote_cp_user":"hoici","locale":"en"},"output":{},"humanized":{"action":"Delete","input":[["repository",{"text":"repository 'TestRepo1'","link":null}],["product",{"text":"product 'Testproduct'","link":"#/products/39/info"}],["organization",{"text":"organization 'Hilti'","link":"/organizations/3/edit"}]],"output":"","errors":[]},"cli_example":null}

Comment 2 Peter Vreman 2015-09-21 09:27:49 UTC
I noticed with greppign that there are some additional places in API where sync_task is used instead of async_task.
- All are Destroy tasks.
- Especially the Activation key is different as it returns the task also to the user.

[crash] root@li-lc-1578:/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.67/app/controllers/katello/api/v2# grep '[^a]sync_task.*Destr' *
activation_keys_controller.rb:      task = sync_task(::Actions::Katello::ActivationKey::Destroy,
environments_controller.rb:      sync_task(::Actions::Katello::Environment::Destroy, @environment)
repositories_controller.rb:      sync_task(::Actions::Katello::Repository::Destroy, @repository)
systems_bulk_actions_controller.rb:      @systems.each { |system| sync_task(::Actions::Katello::System::Destroy, system) }
systems_controller.rb:      sync_task(::Actions::Katello::System::Destroy, @system)

For API consistency i recommend to change all DELETE APIs to use async_task and return the task ID in the API response.

Comment 4 Peter Vreman 2016-08-24 09:39:36 UTC
Proposed upstream PR https://github.com/Katello/katello/pull/6230

Comment 5 Bryan Kearney 2016-08-24 12:09:38 UTC
Upstream bug component is API - Content

Comment 7 Satellite Program 2017-01-05 16:08:22 UTC
Upstream bug assigned to akarsale

Comment 8 Satellite Program 2017-01-05 16:08:26 UTC
Upstream bug assigned to akarsale

Comment 9 Peter Vreman 2017-04-11 16:22:38 UTC
The Katello API is for Delete inconsistent with Foreman

Foreman Delete returned the old record or error field:

[crash/LI] root@li-lc-1578:~# curl -v -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/api/v2/domains/4
* About to connect() to localhost port 443 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
...
> Content-Type: application/json
> Content-Length: 17
>
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 200 OK
< Date: Tue, 11 Apr 2017 16:18:42 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Foreman_version: 1.11.0.66
< Foreman_api_version: 2
< Apipie-Checksum: b5f1cb2585459688a9c5de5cfaa8f420
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 9afe1d69-2872-497a-88e7-4a6ac5290d60
< X-Runtime: 0.043318
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: request_method=DELETE; path=/
< Set-Cookie: _session_id=67793f7f6e6d2859516b87e42bdb4bfe; path=/; secure; HttpOnly
< ETag: "39430dcaf397c8751685ccc6cdd6fbd1"
< Status: 200 OK
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
* Connection #0 to host localhost left intact
{"id":4,"name":"test1","fullname":"","created_at":"2017-04-11T16:18:27.791Z","updated_at":"2017-04-11T16:18:27.791Z","dns_id":null}[crash/LI] root@li-lc-1578:~#
[crash/LI] root@li-lc-1578:~# curl -v -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/api/v2/domains/4
* About to connect() to localhost port 443 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
....
> Content-Length: 17
>
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 404 Not Found
< Date: Tue, 11 Apr 2017 16:18:46 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Foreman_version: 1.11.0.66
< Foreman_api_version: 2
< Apipie-Checksum: b5f1cb2585459688a9c5de5cfaa8f420
< Cache-Control: no-cache
< X-Request-Id: 4156c8fc-8687-4538-821c-8154a9135d3b
< X-Runtime: 0.027190
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: request_method=DELETE; path=/
< Set-Cookie: _session_id=5ba2148f21aae48f5ec8c13a41904830; path=/; secure; HttpOnly
< Status: 404 Not Found
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
{
  "error": {"message":"Resource domain not found by id '4'"}
}
* Connection #0 to host localhost left intact
[crash/LI] root@li-lc-1578:~#


Katello Delete returns empty when delete successful:

[crash/LI] root@li-lc-1578:~# curl -v -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/katello/api/v2/environments/5
* About to connect() to localhost port 443 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
...
> Content-Type: application/json
> Content-Length: 17
>
* upload completely sent off: 17 out of 17 bytes
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
< HTTP/1.1 200 OK
< Date: Tue, 11 Apr 2017 16:20:22 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Foreman_version: 1.11.0.66
< Foreman_api_version: 2
< Apipie-Checksum: b5f1cb2585459688a9c5de5cfaa8f420
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 38a30ff5-6021-4114-98ad-5f57f87adef3
< X-Runtime: 0.275926
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: request_method=DELETE; path=/
< Set-Cookie: _session_id=be4c043cc083a0171ae58e0f923ef5a4; path=/; secure; HttpOnly
< ETag: "9922c386fb520a0063f7052b116d32b6"
< Status: 200 OK
< Content-Length: 5
< Content-Type: application/json; charset=utf-8
<
  {}
* Connection #0 to host localhost left intact
[crash/LI] root@li-lc-1578:~# curl -v -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/katello/api/v2/environments/5
* About to connect() to localhost port 443 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
...
> Content-Type: application/json
> Content-Length: 17
>
* upload completely sent off: 17 out of 17 bytes
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
< HTTP/1.1 404 Not Found
< Date: Tue, 11 Apr 2017 16:20:26 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Foreman_version: 1.11.0.66
< Foreman_api_version: 2
< Apipie-Checksum: b5f1cb2585459688a9c5de5cfaa8f420
< Cache-Control: no-cache
< X-Request-Id: df431265-dd02-45c2-a12d-996a20461034
< X-Runtime: 0.025946
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: request_method=DELETE; path=/
< Set-Cookie: _session_id=fcf2ab9c14a42175147804c3136f4055; path=/; secure; HttpOnly
< Status: 404 Not Found
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
* Connection #0 to host localhost left intact
{"displayMessage":"Couldn't find Katello::KTEnvironment with 'id'=5","errors":["Couldn't find Katello::KTEnvironment with 'id'=5"]}


For me as an API user returning an empty hash is infact an unknown state if it was Success or Failure that the correct Id was deleted. With the Foreman response including the old record i know i deleted the expected entry.
is

Comment 10 Peter Vreman 2017-09-06 11:34:16 UTC
I have updated my CI scripts to check for error messages and not expect the id or taskid field anymore.

But the error message checking is a bit more complicated because Katello and Foreman do it different

See below the erors in case of a Foreman puppet environemnt or a Katello lifecycle environment:

[crash/LI] root@li-lc-1578:~# curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/katello/api/v2/environments/77777 | jq .
{
  "displayMessage": "Couldn't find Katello::KTEnvironment with 'id'=77777",
  "errors": [
    "Couldn't find Katello::KTEnvironment with 'id'=77777"
  ]
}

[crash/LI] root@li-lc-1578:~# curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XDELETE https://localhost/api/v2/environments/77777 | jq .
{
  "error": {
    "message": "Resource environment not found by id '77777'"
  }
}

This kind of differences in responses makes the life for an API user complicated.

Please think of standardizing API output throughout Foreman and its Plugins.

Renamed this BZ accordingly

Comment 11 Peter Vreman 2018-02-23 13:47:03 UTC
Not needed anymore.

I have updated my CI code to handle the different way of error messages responses between Foreman and Katello.