Bug 1646733

Summary: hammer hostgroup info --output json returns json including non unique keys
Product: Red Hat Satellite Reporter: Mike McCune <mmccune>
Component: Host GroupAssignee: Rahul Bajaj <rabajaj>
Status: CLOSED ERRATA QA Contact: jcallaha
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.0CC: adujicek, akarsale, apatel, inecas, jhutar, kgaikwad, mmccune, pcreech, rabajaj, tbrisker, zhunting
Target Milestone: 6.4.1Keywords: PrioBumpQA, Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-hammer_cli_foreman-0.13.2.2-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1596504 Environment:
Last Closed: 2018-12-06 22:32:53 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mike McCune 2018-11-05 22:38:58 UTC 
+++ This bug was initially created as a clone of Bug #1596504 +++

there is "Operating System" twice and they contain different data
so one of them is lost if you parse it

<pre>
# hammer --output json -v -u admin -p changeme  hostgroup info --id="36"
{
  "Id": 36,
  "Name": "6lLfrO",
  "Title": "6lLfrO",
  "Operating System": {
    "content_source_id": 1,
    "content_source_name": "foreman.redhat.com",
    "content_view_id": 371,
  },
  "Puppet CA Proxy": {
    "content_source_id": 1,
    "content_source_name": "foreman.redhat.com",
    "content_view_id": 371,
  },
  "Puppet Master Proxy": {
  },
  "Network": {
  },
  "Operating system": {
    "Architecture": {
      "content_source_id": 1,
      "content_source_name": "foreman.redhat.com",
      "content_view_id": 371
    }
  },
  "Puppetclasses": [
  ],
  "Parameters": [
    null
  ],
  "Organizations": [
  ],
  "OpenSCAP Proxy": null,
  "Content View": {
    "ID": 371,
    "Name": "SQyJYkoRsU"
  },
  "Lifecycle Environment": {
    "ID": 265,
    "Name": "rjPGjE"
  },
  "Content Source": {
    "ID": 1,
    "Name": "sforeman.redhat.com"
  },
  "Kickstart Repository": {
    "ID": 566,
    "Name": "NSZjjWCFkVaaVuF"
  }
}
</pre>

foreman-1.18.0-0.develop.201804061848git210e295.el7.noarch
tfm-rubygem-hammer_cli-0.13.pre.develop-1.201803232224gite493035.el7.noarch

--- Additional comment from adujicek on 20180629T07:11:19

Created from redmine issue http://projects.theforeman.org/issues/23238

--- Additional comment from adujicek on 20180629T07:11:23

Upstream bug assigned to None

--- Additional comment from pm-rhel on 20180629T07:15:53

Since this issue was entered in Red Hat Bugzilla, the pm_ack has been
set to + automatically for the next planned release

--- Additional comment from adujicek on 20180629T07:16:25

just a note, it is kind of unique, once lower case, once upper case (s/System)
"Operating System"
"Operating system"

but still, it does not make much sense to have it there twice..

--- Additional comment from jhutar on 20180629T07:35:43

6.3.0 do not have this kind of duplicity:

[root@host-8-250-186 ~]# hammer --output json hostgroup info --id 1
{
  "Id": 1,
  "Name": "jhutar-delme",
  "Title": "jhutar-delme",
  "Operating System": {
    "content_source_id": null,
    "content_source_name": null,
    "content_view_id": null,
    "content_view_name": null,
    "lifecycle_environment_id": null,
    "lifecycle_environment_name": null,
    "kickstart_repository_id": null,
    "subnet_id": null,
    "subnet_name": null,
    "operatingsystem_id": 1,
    "operatingsystem_name": "RedHat 7.5",
    "domain_id": null,
    "domain_name": null,
    "environment_id": null,
    "environment_name": null,
    "compute_profile_id": null,
    "compute_profile_name": null,
    "ancestry": null,
    "parent_id": null,
    "parent_name": null,
    "ptable_id": 90,
    "ptable_name": "Kickstart default",
    "medium_id": null,
    "medium_name": null,
    "pxe_loader": "PXELinux BIOS",
    "architecture_id": 1,
    "architecture_name": "x86_64",
    "realm_id": null,
    "realm_name": null,
    "created_at": "2018-06-29 07:01:41 UTC",
    "updated_at": "2018-06-29 07:01:41 UTC",
    "id": 1,
    "name": "jhutar-delme",
    "title": "jhutar-delme",
    "puppet_proxy_id": null,
    "puppet_proxy_name": null,
    "puppet_ca_proxy_id": null,
    "puppet_ca_proxy_name": null,
    "openscap_proxy_id": null,
    "openscap_proxy_name": null,
    "puppet_proxy": null,
    "puppet_ca_proxy": null,
    "openscap_proxy": null,
    "template_combinations": [

    ],
    "puppetclasses": [

    ],
    "config_groups": [

    ],
    "parameters": [

    ],
    "all_puppetclasses": [

    ],
    "locations": [

    ],
    "organizations": [

    ]
  },
  "Architecture": {
    "content_source_id": null,
    "content_source_name": null,
    "content_view_id": null,
    "content_view_name": null,
    "lifecycle_environment_id": null,
    "lifecycle_environment_name": null,
    "kickstart_repository_id": null,
    "subnet_id": null,
    "subnet_name": null,
    "operatingsystem_id": 1,
    "operatingsystem_name": "RedHat 7.5",
    "domain_id": null,
    "domain_name": null,
    "environment_id": null,
    "environment_name": null,
    "compute_profile_id": null,
    "compute_profile_name": null,
    "ancestry": null,
    "parent_id": null,
    "parent_name": null,
    "ptable_id": 90,
    "ptable_name": "Kickstart default",
    "medium_id": null,
    "medium_name": null,
    "pxe_loader": "PXELinux BIOS",
    "architecture_id": 1,
    "architecture_name": "x86_64",
    "realm_id": null,
    "realm_name": null,
    "created_at": "2018-06-29 07:01:41 UTC",
    "updated_at": "2018-06-29 07:01:41 UTC",
    "id": 1,
    "name": "jhutar-delme",
    "title": "jhutar-delme",
    "puppet_proxy_id": null,
    "puppet_proxy_name": null,
    "puppet_ca_proxy_id": null,
    "puppet_ca_proxy_name": null,
    "openscap_proxy_id": null,
    "openscap_proxy_name": null,
    "puppet_proxy": null,
    "puppet_ca_proxy": null,
    "openscap_proxy": null,
    "template_combinations": [

    ],
    "puppetclasses": [

    ],
    "config_groups": [

    ],
    "parameters": [

    ],
    "all_puppetclasses": [

    ],
    "locations": [

    ],
    "organizations": [

    ]
  },
  "Partition Table": {
    "content_source_id": null,
    "content_source_name": null,
    "content_view_id": null,
    "content_view_name": null,
    "lifecycle_environment_id": null,
    "lifecycle_environment_name": null,
    "kickstart_repository_id": null,
    "subnet_id": null,
    "subnet_name": null,
    "operatingsystem_id": 1,
    "operatingsystem_name": "RedHat 7.5",
    "domain_id": null,
    "domain_name": null,
    "environment_id": null,
    "environment_name": null,
    "compute_profile_id": null,
    "compute_profile_name": null,
    "ancestry": null,
    "parent_id": null,
    "parent_name": null,
    "ptable_id": 90,
    "ptable_name": "Kickstart default",
    "medium_id": null,
    "medium_name": null,
    "pxe_loader": "PXELinux BIOS",
    "architecture_id": 1,
    "architecture_name": "x86_64",
    "realm_id": null,
    "realm_name": null,
    "created_at": "2018-06-29 07:01:41 UTC",
    "updated_at": "2018-06-29 07:01:41 UTC",
    "id": 1,
    "name": "jhutar-delme",
    "title": "jhutar-delme",
    "puppet_proxy_id": null,
    "puppet_proxy_name": null,
    "puppet_ca_proxy_id": null,
    "puppet_ca_proxy_name": null,
    "openscap_proxy_id": null,
    "openscap_proxy_name": null,
    "puppet_proxy": null,
    "puppet_ca_proxy": null,
    "openscap_proxy": null,
    "template_combinations": [

    ],
    "puppetclasses": [

    ],
    "config_groups": [

    ],
    "parameters": [

    ],
    "all_puppetclasses": [

    ],
    "locations": [

    ],
    "organizations": [

    ]
  },
  "Puppet CA Proxy Id": null,
  "Puppet Master Proxy Id": null,
  "Puppetclasses": [

  ],
  "Parameters": [

  ],
  "Parent Id": null,
  "OpenSCAP Proxy": null,
  "Content View": {
    "ID": null,
    "Name": null
  },
  "Lifecycle Environment": {
    "ID": null,
    "Name": null
  },
  "Content Source": {
    "ID": null,
    "Name": null
  },
  "Kickstart Repository": {
    "ID": null,
    "Name": null
  }
}
[root@host-8-250-186 ~]# rpm -qa | grep hammer | sort
tfm-rubygem-hammer_cli-0.11.0.1-1.el7sat.noarch
tfm-rubygem-hammer_cli_csv-2.3.1-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman-0.11.0.5-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_admin-0.0.8-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.1.3.3-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_discovery-1.0.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.6-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_openscap-0.1.5-1.fm1_15.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_remote_execution-0.0.6-1.fm1_15.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.12-1.fm1_15.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_virt_who_configure-0.0.3-1.el7sat.noarch
tfm-rubygem-hammer_cli_katello-0.11.3.5-1.el7sat.noarch

--- Additional comment from jhutar on 20180629T07:38:15

This is in current downstream snapshots as well:

[root@sat640snap9 ~]# hammer --output json -u admin -p changeme hostgroup info --id 23
{
  "Id": 23,
  "Name": "jhutar-delme",
  "Title": "jhutar-delme",
  "Operating System": {
    "content_source_id": null,
    "content_source_name": null,
    "content_view_id": null,
    "content_view_name": null,
    "lifecycle_environment_id": null,
    "lifecycle_environment_name": null,
    "kickstart_repository_id": null,
    "kickstart_repository_name": null,
    "subnet_id": null,
    "subnet_name": null,
    "operatingsystem_id": 1,
    "operatingsystem_name": "RedHat 7.5",
    "domain_id": null,
    "domain_name": null,
    "environment_id": null,
    "environment_name": null,
    "compute_profile_id": null,
    "compute_profile_name": null,
    "ancestry": null,
    "parent_id": null,
    "parent_name": null,
    "ptable_id": 94,
    "ptable_name": "Kickstart default",
    "medium_id": null,
    "medium_name": null,
    "pxe_loader": "Grub2 UEFI",
    "subnet6_id": null,
    "subnet6_name": null,
    "architecture_id": 1,
    "architecture_name": "x86_64",
    "realm_id": null,
    "realm_name": null,
    "created_at": "2018-06-29 07:36:47 UTC",
    "updated_at": "2018-06-29 07:36:47 UTC",
    "id": 23,
    "name": "jhutar-delme",
    "title": "jhutar-delme",
    "description": "",
    "puppet_proxy_id": null,
    "puppet_proxy_name": null,
    "puppet_ca_proxy_id": null,
    "puppet_ca_proxy_name": null,
    "openscap_proxy_id": null,
    "openscap_proxy_name": null,
    "puppet_proxy": null,
    "puppet_ca_proxy": null,
    "openscap_proxy": null,
    "template_combinations": [

    ],
    "puppetclasses": [

    ],
    "config_groups": [

    ],
    "all_puppetclasses": [

    ],
    "locations": [

    ],
    "organizations": [
      {
        "id": 1,
        "name": "Default Organization",
        "title": "Default Organization",
        "description": null
      }
    ]
  },
  "Description": "",
  "Network": {
  },
  "Operating system": {
    "Architecture": {
      "content_source_id": null,
      "content_source_name": null,
      "content_view_id": null,
      "content_view_name": null,
      "lifecycle_environment_id": null,
      "lifecycle_environment_name": null,
      "kickstart_repository_id": null,
      "kickstart_repository_name": null,
      "subnet_id": null,
      "subnet_name": null,
      "operatingsystem_id": 1,
      "operatingsystem_name": "RedHat 7.5",
      "domain_id": null,
      "domain_name": null,
      "environment_id": null,
      "environment_name": null,
      "compute_profile_id": null,
      "compute_profile_name": null,
      "ancestry": null,
      "parent_id": null,
      "parent_name": null,
      "ptable_id": 94,
      "ptable_name": "Kickstart default",
      "medium_id": null,
      "medium_name": null,
      "pxe_loader": "Grub2 UEFI",
      "subnet6_id": null,
      "subnet6_name": null,
      "architecture_id": 1,
      "architecture_name": "x86_64",
      "realm_id": null,
      "realm_name": null,
      "created_at": "2018-06-29 07:36:47 UTC",
      "updated_at": "2018-06-29 07:36:47 UTC",
      "id": 23,
      "name": "jhutar-delme",
      "title": "jhutar-delme",
      "description": "",
      "puppet_proxy_id": null,
      "puppet_proxy_name": null,
      "puppet_ca_proxy_id": null,
      "puppet_ca_proxy_name": null,
      "openscap_proxy_id": null,
      "openscap_proxy_name": null,
      "puppet_proxy": null,
      "puppet_ca_proxy": null,
      "openscap_proxy": null,
      "template_combinations": [

      ],
      "puppetclasses": [

      ],
      "config_groups": [

      ],
      "all_puppetclasses": [

      ],
      "locations": [

      ],
      "organizations": [
        {
          "id": 1,
          "name": "Default Organization",
          "title": "Default Organization",
          "description": null
        }
      ]
    },
    "Operating System": {
      "content_source_id": null,
      "content_source_name": null,
      "content_view_id": null,
      "content_view_name": null,
      "lifecycle_environment_id": null,
      "lifecycle_environment_name": null,
      "kickstart_repository_id": null,
      "kickstart_repository_name": null,
      "subnet_id": null,
      "subnet_name": null,
      "operatingsystem_id": 1,
      "operatingsystem_name": "RedHat 7.5",
      "domain_id": null,
      "domain_name": null,
      "environment_id": null,
      "environment_name": null,
      "compute_profile_id": null,
      "compute_profile_name": null,
      "ancestry": null,
      "parent_id": null,
      "parent_name": null,
      "ptable_id": 94,
      "ptable_name": "Kickstart default",
      "medium_id": null,
      "medium_name": null,
      "pxe_loader": "Grub2 UEFI",
      "subnet6_id": null,
      "subnet6_name": null,
      "architecture_id": 1,
      "architecture_name": "x86_64",
      "realm_id": null,
      "realm_name": null,
      "created_at": "2018-06-29 07:36:47 UTC",
      "updated_at": "2018-06-29 07:36:47 UTC",
      "id": 23,
      "name": "jhutar-delme",
      "title": "jhutar-delme",
      "description": "",
      "puppet_proxy_id": null,
      "puppet_proxy_name": null,
      "puppet_ca_proxy_id": null,
      "puppet_ca_proxy_name": null,
      "openscap_proxy_id": null,
      "openscap_proxy_name": null,
      "puppet_proxy": null,
      "puppet_ca_proxy": null,
      "openscap_proxy": null,
      "template_combinations": [

      ],
      "puppetclasses": [

      ],
      "config_groups": [

      ],
      "all_puppetclasses": [

      ],
      "locations": [

      ],
      "organizations": [
        {
          "id": 1,
          "name": "Default Organization",
          "title": "Default Organization",
          "description": null
        }
      ]
    },
    "Partition Table": {
      "content_source_id": null,
      "content_source_name": null,
      "content_view_id": null,
      "content_view_name": null,
      "lifecycle_environment_id": null,
      "lifecycle_environment_name": null,
      "kickstart_repository_id": null,
      "kickstart_repository_name": null,
      "subnet_id": null,
      "subnet_name": null,
      "operatingsystem_id": 1,
      "operatingsystem_name": "RedHat 7.5",
      "domain_id": null,
      "domain_name": null,
      "environment_id": null,
      "environment_name": null,
      "compute_profile_id": null,
      "compute_profile_name": null,
      "ancestry": null,
      "parent_id": null,
      "parent_name": null,
      "ptable_id": 94,
      "ptable_name": "Kickstart default",
      "medium_id": null,
      "medium_name": null,
      "pxe_loader": "Grub2 UEFI",
      "subnet6_id": null,
      "subnet6_name": null,
      "architecture_id": 1,
      "architecture_name": "x86_64",
      "realm_id": null,
      "realm_name": null,
      "created_at": "2018-06-29 07:36:47 UTC",
      "updated_at": "2018-06-29 07:36:47 UTC",
      "id": 23,
      "name": "jhutar-delme",
      "title": "jhutar-delme",
      "description": "",
      "puppet_proxy_id": null,
      "puppet_proxy_name": null,
      "puppet_ca_proxy_id": null,
      "puppet_ca_proxy_name": null,
      "openscap_proxy_id": null,
      "openscap_proxy_name": null,
      "puppet_proxy": null,
      "puppet_ca_proxy": null,
      "openscap_proxy": null,
      "template_combinations": [

      ],
      "puppetclasses": [

      ],
      "config_groups": [

      ],
      "all_puppetclasses": [

      ],
      "locations": [

      ],
      "organizations": [
        {
          "id": 1,
          "name": "Default Organization",
          "title": "Default Organization",
          "description": null
        }
      ]
    },
    "PXE Loader": "Grub2 UEFI"
  },
  "Puppetclasses": [

  ],
  "Parameters": [
    null
  ],
  "Organizations": [
    {
      "id": 1,
      "name": "Default Organization",
      "title": "Default Organization",
      "description": null
    }
  ],
  "OpenSCAP Proxy": null,
  "Content View": {
    "ID": null,
    "Name": null
  },
  "Lifecycle Environment": {
    "ID": null,
    "Name": null
  },
  "Content Source": {
    "ID": null,
    "Name": null
  },
  "Kickstart Repository": {
    "ID": null,
    "Name": null
  }
}
[root@sat640snap9 ~]# rpm -qa | grep hammer | sort
tfm-rubygem-hammer_cli-0.13.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_csv-2.3.1-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman-0.13.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_admin-0.0.8-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.1.3.3-3.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_discovery-1.0.0-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.6-3.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_openscap-0.1.6-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_remote_execution-0.0.6-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.12-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_templates-0.1.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_virt_who_configure-0.0.3-2.el7sat.noarch
tfm-rubygem-hammer_cli_katello-0.13.1-1.el7sat.noarch

--- Additional comment from jhutar on 20180629T07:40:27

Setting severity to "medium" because it is important for QE automation

--- Additional comment from pm-rhel on 20180629T07:42:01

This bug report has Keywords: Regression or TestBlocker.

Since no regressions or test blockers are allowed between releases,
it is also being identified as a blocker for this release.

Please resolve ASAP.

--- Additional comment from bbuckingham on 20180629T15:12:14

Core triage recommends this for 6.4 GA due to regression and impact to automation.

--- Additional comment from akarsale on 20180917T07:19:25

This bug is requested to have a unique attributes and the values within the hash.
In current example we have "Operating System" and "Operating system" as two hashes with different structure, Later one having "architecture" hash within "Operating system" hash and then the remaining attribute list.

So the question is how can we conclude what "Operating System" key is to be kept and which one to be merged into other.

--- Additional comment from bbuckingham on 20180920T17:34:19

Hi Amit,

Based upon comment 5, assuming that we do not want to impact existing users the value that should be kept is "Operating System". 

Adding needinfo for awareness, since we have very little time remaining to address for GA.

--- Additional comment from akarsale on 20180920T18:03:37

Okay Brad, will be keeping "Operating System" attribute and making the necessary change.

--- Additional comment from rabajaj on 20180921T07:45:40

Hello,

Looking at the issue, i have found out the following:

a) The data loaded from the api is correct and has no duplicate,
so the change has to occur on the hammer side.

b) The data in both 'Operating System' and 'Operating system' is
consistent as there is only one request being made so the data is
consistent in both of them.

--- Additional comment from rabajaj on 20180921T08:32:54

Hello,

Digging a little deeper, i found out that there are no 
duplicate values but nesting involved here:

```

- Operating system : {
    + Architecture: {51},
    + Operating System: {51},
    + Partition Table: {51},
      PXE Loader: "PXELINUX BIOS",
  },

```

After talking to Tomas, we figured out the real issue is with 
the output adapters.
Tomas explained that the output adapters basically don't apply
to json and yaml, thus we are facing this issue.

--- Additional comment from apatel on 20180921T09:22:00

Upstream ticket for the underlying issue: https://projects.theforeman.org/issues/24980

--- Additional comment from mmccune on 20180925T16:32:27

moving to 6.4.z due to lack of time and not considered a GA blocker. Acked by PM + DEV + QE

--- Additional comment from rabajaj on 20180926T11:37:20

This issue is fixed in the upstream now:

the related prs are:
1) https://github.com/theforeman/hammer-cli-foreman/pull/390
2) https://github.com/theforeman/hammer-cli/pull/288

Thanks Tomas and Martin for the help :)
Comment 4 jcallaha 2018-11-21 19:08:43 UTC 
Verified in Satellite 6.4.1 Snap 1.

the returned output now correctly nests the operating system information.

[root@host ~]# hammer --output json -v -u admin -p changeme  hostgroup info --id 1
{
  "Id": 1,
  "Name": "test",
  "Title": "test",
  "Description": "",
  "Network": {
  },
  "Operating system": {
    "Architecture": {
      "Name": "x86_64",
      "Id": 1
    },
    "Operating System": {
      "Name": "RHEL Server 7.6",
      "Id": 1
    },
    "PXE Loader": "PXELinux BIOS"
  },
  "Puppetclasses": [

  ],
  "Parameters": [

  ],
  "Organizations": [
    {
      "Name": "Default Organization",
      "Id": 1
    }
  ],
  "OpenSCAP Proxy": null,
  "Content View": {
    "ID": 1,
    "Name": "Default Organization View"
  },
  "Lifecycle Environment": {
    "ID": 1,
    "Name": "Library"
  },
  "Content Source": {
    "ID": 1,
    "Name": "sat.domain"
  },
  "Kickstart Repository": {
    "ID": null,
    "Name": null
  }
}
Comment 6 errata-xmlrpc 2018-12-06 22:32:53 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3799