Bug 1381621

Summary: oadm manage-node with json output isn't json 3.2/3.3
Product: OKD Reporter: Kenny Woodson <kwoodson>
Component: ocAssignee: Juan Vallejo <jvallejo>
Status: CLOSED CURRENTRELEASE QA Contact: Xingxing Xia <xxia>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.xCC: aos-bugs, jvallejo, kwoodson, mmccomas, xiaocwan, xxia
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-09 21:50:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Kenny Woodson 2016-10-04 14:59:49 UTC
Description of problem:
I am running the following command:

oadm manage-node --selector="type=infra" --list-pods -o json

I receive non-json output from this command in the following format:
-------

Listing matched pods on node: ip-172-31-53-15.ec2.internal

{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "350009"
    },
    "items": [
        {
....
}

Listing matched pods on node: ip-172-31-53-16.ec2.internal

{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "350009"
    },
    "items": [
        {
...
}
 
Version-Release number of selected component (if applicable):
3.2 and 3.3

How reproducible:
Every time.

Steps to Reproduce:
1. Run the command oadm manage-nodes --selector= --list-pods
2. Inspect the output
3.

Actual results:
In 3.2, the output is intermingled with the hosts and is not json.

In 3.3, the nodes are removed from stdout and placed into stederr.  I'm not sure why these are returned in stderr as they are not errors.  The output in stderr is still not json.  If you have multiple nodes that matched the selector then you will be given a json hash of each node stacked one on top of the other but not in valid json.

Expected results:
When I call the oadm manage-nodes with -o yaml or -o  json it should do the right thing which is, put the nodes in an dict and put the pods in as an array of pods.  It should look like the following:

[{'ip-172-31-59-33.ec2.internal': [{ pod }, { pod }], },
 {'ip-172-31-59-34.ec2.internal': [{ pod }, { pod }], },
]

This would simplify the output and allow consumers of the api to be able to parse it.  This also removes erroneous stderr messages.


Additional info:

Comment 1 Juan Vallejo 2016-10-04 19:56:54 UTC
In the latest version of origin (1.3), the node's name is printed to stderr, while the actual json object containing the pods is printed to stdout (so parsing, at least, is not an issue). 

However, I like your idea of grouping pod lists with their associated node in a json / yaml object, and have opened a PR to add this here: https://github.com/openshift/origin/pull/11216

Comment 2 Kenny Woodson 2016-10-04 20:24:08 UTC
The parsing in 1.3 continues to be an issue as the json is stacked on top of each other.  The user is then expected to parse the output into json.

Example:
{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "102478"
    },
    "items": []
}
{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "123456"
    },
    "items": []
}

The previous example is what is returned and does not load into json because it is not a single hash or a list of hashes.  Its just stacked json.

I appreciate the work in the PR, it will greatly improve our ability to work with the output of oadm manage-node.

Thanks Juan.

Comment 3 Juan Vallejo 2016-10-04 20:51:50 UTC
Hi Kenny,

Per this comment on the PR thread[1], adding the suggested format to json and yaml output would make that output incompatible with other `oc` / `oadm` commands, as it would no longer follow a valid API object spec.

This comment[2] suggests alternatives for parsing the current output on origin 1.3

[1] https://github.com/openshift/origin/pull/11216#issuecomment-251500534
[2] https://github.com/openshift/origin/pull/11216#issuecomment-251500820

Comment 4 Juan Vallejo 2016-10-05 17:44:52 UTC
Revised the previously submitted PR. Rather than modify api objects when printing, list of pods from all nodes will be merged into a single list:

Example:
{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "102478"
    },
    "items": [
        ... // items from nodes 'ip-172-31-59-33.ec2.internal' and 'ip-172-31-59-34.ec2.internal'
    ]
}

this ensures that the output is easy to parse, and does not "break" the printed object's compatibility with other openshift commands.

Comment 5 Juan Vallejo 2016-10-10 19:48:22 UTC
Updated PR: https://github.com/openshift/origin/pull/11216

Comment 6 Juan Vallejo 2016-10-24 14:06:53 UTC
Output now prints objects in an `api.List`, this ensures that the entire output can be parsed as a single json or yaml object, while maintaining compatibility with the rest of `oc` commands.

Comment 8 Kenny Woodson 2016-10-27 10:43:40 UTC
Please test the case where there are multiple notes returned by the query. That will show the enhancements made. Thanks.

Comment 9 Kenny Woodson 2016-10-27 10:44:02 UTC
Please test the case where there are multiple notes returned by the query. That will show the enhancements made. Thanks.

Comment 10 XiaochuanWang 2016-10-28 05:50:08 UTC
Hmm.. tested on multi-nodes OCP env, 3.4 is not reproduced, but 3.2, 3.3 reproduced:
oadm v3.4.0.16+cc70b72 (not reproduced)
oadm v3.3.1.3 (reproduced)
oadm v3.2.2.2 (reproduced)


# oadm manage-node --selector="role=node"  --list-pods -o json

Listing matched pods on node: ip-172-18-13-3.ec2.internal

{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "12445"
    },
    "items": [...]
}

Listing matched pods on node: ip-172-18-4-108.ec2.internal

{
    "metadata": {
        "selfLink": "/api/v1/pods",
        "resourceVersion": "12524"
    },
    "items": [...]
}

Comment 11 Juan Vallejo 2016-10-28 14:12:32 UTC
kwoodson would you need this fix backported to OCP 3.3? It is currently in 3.4 which has not released yet.

Comment 12 Juan Vallejo 2016-10-31 15:01:17 UTC
xiaocwan Marking this bug as ON_QA once more. We are not backporting to 3.2, and the bug's severity is not high enough to be backported to 3.3.

3.4 has not released yet, but the bugfix can be verified on the latest master.

cc ffranz

Comment 13 XiaochuanWang 2016-11-01 05:32:35 UTC
# oadm manage-node --selector="role=node"  --list-pods -o json > pods.json
check pods.json, there is not extra line "Listing matched pods on node: ip-xxx.xxx.xxx.internal"

Verified on oc/openshift v3.4.0.17+b8a03bc

Comment 14 Kenny Woodson 2016-11-07 14:51:18 UTC
(In reply to Juan Vallejo from comment #11)
> kwoodson would you need this fix backported to OCP 3.3? It is
> currently in 3.4 which has not released yet.

It would be nice to have this backported but it is not crucial at this time.