Bug 1506102

Summary: [ceph-mgr] Few API commands do not work as expected.
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Madhavi Kasturi <mkasturi>
Component: Ceph-Mgr PluginsAssignee: Boris Ranto <branto>
Status: CLOSED ERRATA QA Contact: Sidhant Agrawal <sagrawal>
Severity: high Docs Contact: Bara Ancincova <bancinco>
Priority: high    
Version: 3.0CC: branto, ceph-eng-bugs, ceph-qe-bugs, edonnell, flucifre, hnallurv, jbrier, john.spray, kdreyer, rperiyas, tserlin
Target Milestone: rc   
Target Release: 3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: RHEL: ceph-12.2.5-13.el7cp Ubuntu: 12.2.5-4redhat1xenial Doc Type: Bug Fix
Doc Text:
.The fixes for pg_num/pgp_num setting through the RESTful API Previously, attempts to change `pgp_num` or `pg_num` via the RESTful API plugin failed. With this update to {product}, the API is able to change the `pgp_num` and `pg_num` parameter successfully.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-26 18:16:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1494421, 1584264    

Description Madhavi Kasturi 2017-10-25 06:41:07 UTC
Description of problem:
The RestFul plugin displays incorrect output.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Deployed a cluster with ceph-mgr
2. enabled the RESTful plugin 
3. created a self-signed certificate.
The command execution status 
Section : 2.2.4 How can I change information for a Pool 
1. Change pg_num option
1.a) CURL
[root@magna084 ~]# echo -En '{"pg_num": 120}' | curl --request PATCH --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/pool/1'
Enter host password for user 'admin':
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>
[
1.b) Python
>>> result = requests.patch('https://10.8.128.84:8003/pool/1', json={"pg_num": 130}, auth=("admin", "da3923ad-5755-4579-aa00-d8ee1e028f86"),verify=False)
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
>>> print result.json()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
>>> 

Note: The pg_num reflects in pool 
--
2. Change pgp_num option
2.a) CURL
[root@magna084 ubuntu]# echo -En '{"pgp_num": 120}' | curl --request PATCH --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/pool/1'
Enter host password for user 'admin':
{
    "failed": [],
    "finished": [],
    "has_failed": false,
    "id": "94041941990608",
    "is_finished": false,
    "is_waiting": false,
    "running": [
        {
            "command": "osd pool set var=pgp_num pool=poola",
            "outb": "",
            "outs": ""
        }
    ],
    "state": "pending",
    "waiting": []
2.b)Web browser:
{
    "failed": [
        {
            "command": "osd pool set var=pgp_num pool=poola",
            "outb": "",
            "outs": "(22) Invalid argument"
        }
    ],
    "finished": [],
    "has_failed": true,
    "id": "94041941990608",
    "is_finished": true,
    "is_waiting": false,
    "running": [],
    "state": "failed",
    "waiting": []
}

3. Changing auid parameter from Ceph command works. However, with CURL and python it fails.
3.a)ceph command:
[root@magna084 ~]# ceph osd pool set poola auid 1
set pool 1 auid to 1
Web browser snippet:
{
    "application_metadata": {},
    "auid": 1,
    "cache_min_evict_age": 0,

3.b) Curl
[root@magna084 ~]# echo -En '{"auid": 1}' | curl --request PATCH --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/pool/1'
Enter host password for user 'admin':
{
    "message": "Invalid arguments found: \"[u'auid']\""
}[root@magna084 ~]# 

3.c) Python
>>> result = requests.patch('https://10.8.128.84:8003/pool/1', json={"auid" : 1},auth=("admin", "da3923ad-5755-4579-aa00-d8ee1e028f86"),verify=False)
>>> print result.json()
{u'message': u'Invalid arguments found: "[u\'auid\']"'}

Tried a few parameters and observed the similar results.
1.deep_scrub_interval
2. scrub_max_interval

Also, Observed discrepancy in CURL output,
1. Displays state = " pending" [ would need to navigate to browser request page to check the completion / failure. ]
[root@magna084 ubuntu]# echo -En '{"in": false}' | curl --request PATCH --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/osd/2'
Enter host password for user 'admin':
{
    "failed": [],
    "finished": [],
    "has_failed": false,
    "id": "94041941989840",
    "is_finished": false,
    "is_waiting": false,
    "running": [
        {
            "command": "osd out ids=['2']",
            "outb": "",
            "outs": ""
        }
    ],
    "state": "pending",
    "waiting": []

2. But in a few cases, the state reflects "success"
root@magna084 ubuntu]# echo -En '{"out": false}' | curl --request PATCH --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/osd/2'
Enter host password for user 'admin':
{
    "failed": [],
    "finished": [],
    "has_failed": false,
    "id": "94041941988816",
    "is_finished": true,
    "is_waiting": false,
    "running": [],
    "state": "success",
    "waiting": []
}

Actual results:
command execution fails 

Expected results:
Command execution should succeed.

Additional info:

Comment 7 John Spray 2017-10-26 14:42:04 UTC
I have little to do with this module -- Boris can comment on whether the commands used in testing made sense (i.e. whether it is really broken).

Comment 9 Boris Ranto 2017-10-31 11:54:42 UTC
Upstream PR for pg_num and pgp_num issues (+ auid enablement):

https://github.com/ceph/ceph/pull/18649

Comment 10 Boris Ranto 2017-10-31 12:00:22 UTC
(In reply to mkasturi from comment #0)
> 3. Changing auid parameter from Ceph command works. However, with CURL and
> python it fails.
> 3.a)ceph command:
> [root@magna084 ~]# ceph osd pool set poola auid 1
> set pool 1 auid to 1
> Web browser snippet:
> {
>     "application_metadata": {},
>     "auid": 1,
>     "cache_min_evict_age": 0,
> 
> 3.b) Curl
> [root@magna084 ~]# echo -En '{"auid": 1}' | curl --request PATCH --data @-
> --silent --insecure --user admin 'https://10.8.128.84:8003/pool/1'
> Enter host password for user 'admin':
> {
>     "message": "Invalid arguments found: \"[u'auid']\""
> }[root@magna084 ~]# 
> 
> 3.c) Python
> >>> result = requests.patch('https://10.8.128.84:8003/pool/1', json={"auid" : 1},auth=("admin", "da3923ad-5755-4579-aa00-d8ee1e028f86"),verify=False)
> >>> print result.json()
> {u'message': u'Invalid arguments found: "[u\'auid\']"'}
> 
> Tried a few parameters and observed the similar results.
> 1.deep_scrub_interval
> 2. scrub_max_interval
> 

There is only a limited number of commands that are supported by the API (these should be the same as supported by calamari). I have added auid in the PR. We may want to add a couple more.

> Also, Observed discrepancy in CURL output,
> 1. Displays state = " pending" [ would need to navigate to browser request
> page to check the completion / failure. ]
> [root@magna084 ubuntu]# echo -En '{"in": false}' | curl --request PATCH
> --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/osd/2'
> Enter host password for user 'admin':
> {
>     "failed": [],
>     "finished": [],
>     "has_failed": false,
>     "id": "94041941989840",
>     "is_finished": false,
>     "is_waiting": false,
>     "running": [
>         {
>             "command": "osd out ids=['2']",
>             "outb": "",
>             "outs": ""
>         }
>     ],
>     "state": "pending",
>     "waiting": []
> 
> 2. But in a few cases, the state reflects "success"
> root@magna084 ubuntu]# echo -En '{"out": false}' | curl --request PATCH
> --data @- --silent --insecure --user admin 'https://10.8.128.84:8003/osd/2'
> Enter host password for user 'admin':
> {
>     "failed": [],
>     "finished": [],
>     "has_failed": false,
>     "id": "94041941988816",
>     "is_finished": true,
>     "is_waiting": false,
>     "running": [],
>     "state": "success",
>     "waiting": []
> }
> 

This happens because out is not a valid option for /osd/<id> endpoint. Therefore, it won't schedule any command and the request succeeds. You should use "in": True/False to set osd in/out.

Comment 11 John Spray 2017-10-31 15:43:37 UTC
> This happens because out is not a valid option for /osd/<id> endpoint. Therefore, it won't schedule any command and the request succeeds. You should use "in": True/False to set osd in/out.

The correct status code for trying to set an attribute that doesn't exist would be 400, right?

Comment 25 errata-xmlrpc 2018-09-26 18:16:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2819