Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1823396

Summary: Hosts are rejected due to mismatch of metadata.json and actual hosts included in satellite inventory report
Product: Red Hat Satellite Reporter: Parag Kamble <pakamble>
Component: RH Cloud - InventoryAssignee: Shimon Shtein <sshtein>
Status: CLOSED ERRATA QA Contact: Mirek Długosz <mzalewsk>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.7.0CC: egolov, jjeffers, kholdawa, pcreech, sjagtap, sshtein
Target Milestone: 6.7.4Keywords: Reopened, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-foreman_rh_cloud-1.0.6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-30 13:12:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
satellite json upload is attached none

Description Parag Kamble 2020-04-13 14:56:37 UTC
Created attachment 1678465 [details]
satellite json upload is attached

Created attachment 1678465 [details]
satellite json upload is attached

Description of problem:
ingress reports invalid JSON if any of the hosts having "subscription_status":"Unentitled" status. 

Version-Release number of selected component (if applicable):
Satellite : 6.7

How reproducible:


Steps to Reproduce:
1. create 4 host 2 of them are register with activation key and rest only subscription- manager register command. hence 2 host state will be subscribe and other 2 host is invalid subscription.
2. Upload inventory from the satellite.
3. verify the request on C.R.C.


Actual results:
1. host inventory is not updated.
2. Invalid Json format reported by kibana logs.

2020-04-13 10:54:17,626 | ERROR | report_processor.py:transition_to_downloaded:162 | Report(account=6291899) EXTRACT REPORT FROM TAR - The report could not be downloaded due to the following error: Report(account=6291899) EXTRACT REPORT FROM TAR - Tar does not contain valid JSON metadata & report files

Yupana also reports the ERROR around same time.
2020-04-13 13:02:26,788 | ERROR | report_processor.py:transition_to_downloaded:162 | Report(account=6291899) EXTRACT REPORT FROM TAR - The report could not be downloaded due to the following error: Report(account=6291899) EXTRACT REPORT FROM TAR - Report contained no valid report slices..


Expected results:
1. Host inventory should get updated. 

Additional info:

After this issue i have subscribed the host which is earlier not then i did upload and its get succeeded hence  its look like the problem  with the host status  "subscription_status":"Unentitled".

Comment 4 Sanket Jagtap 2020-04-16 14:08:49 UTC

*** This bug has been marked as a duplicate of bug 1812858 ***

Comment 5 Sanket Jagtap 2020-04-16 14:20:49 UTC

*** This bug has been marked as a duplicate of bug 1821335 ***

Comment 6 Shimon Shtein 2020-04-16 14:58:49 UTC

*** This bug has been marked as a duplicate of bug 1812858 ***

Comment 8 Kevan Holdaway 2020-04-20 13:21:10 UTC
Also, this writeup confuses different layers.  Ingress does not reject anything or validate anything.

Possible issues include:
1. Yupana rejects tar.gz format because you did not conform to syntax (see https://github.com/quipucords/yupana#-formatting-data-for-yupana-without-qpc).  However, this has nothing to do with subscription_status
2. HBI may reject a host (not the tar.gz) based on invalid syntax.  Yupana does NOT validate hosts.  HBI validates hosts.   So likely HBI didn't like subscription_status

I do not think these issues are the same.

- Bug https://bugzilla.redhat.com/show_bug.cgi?id=1812858 relates to yupana validation
- This bug relates to HBI validate
- Ingress doesn't validate (so the title of this bug is confusing)

Comment 9 Shimon Shtein 2020-04-20 14:01:31 UTC
Indeed, the problem is not about subscription_status. What happens here is the following sequence:

Suppose we have a Satellite with only one host.
When a report is created, it will contain only one host, and hosts count in metadata.json will also be 1
Now we add a host using subscription-manager register --org="Default_Organization" --environment="Testing/cv1"
This host does not have valid pools associated with it.
When a report is created, the amount of _potential_ hosts is reported - so metadata.json will contain a slice with count set to 2.
While generating the slice, the plugin will skip generating a report for the new host, hence the report will still contain 1 host.
If we upload this report, it will be rejected by yupana (hence the connection to 1812858).
Now we re-register the host using subscription-manager register --org="Default_Organization" --activationkey="activation_key_1" --force
When using such registration method, the host gets a pool associated to it
Now when we generate the report, metadata.json will still contain count=2 potential hosts.
The slice will contain 2 hosts as well, since both hosts have valid pools in them.
This report will not be rejected by yupana, and the new host will appear in HBI.

Originally the "Unentitled" status was suspected to have something to do with the fact the data did not appear in HBI.
Sanket's tests actually reassured my findings from reading both Yupana's and HBI validation code that status value has nothing to do with this reject.

Bottom line is that it is very hard to identify where the reject comes from - HBI or Yupana.

Kevan, does this explanation make more sense to you?

Comment 11 Sanket Jagtap 2020-04-21 07:04:01 UTC
Changing the title to make it less confusing, but I still think this is a duplicate issue and can be tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1812858

Comment 12 Kevan Holdaway 2020-05-01 19:30:02 UTC
Shim, I am not sure I follow the complication.  The metadata should be simple to generate if you order it correctly.
1. Generate the slice files
2. Since the files are generated, create a metadata section that captures what was generated.  The metadata is simply a summary of what you are sending.  It should be easy to generate right before you create the tar.gz and it will be accurate at that point.
3. Create tar.gz with the metadata and slice json files

Comment 13 Shimon Shtein 2020-05-03 08:16:32 UTC
@Kevan, this one was fixed in https://github.com/theforeman/foreman_rh_cloud/pull/160
So moving this bug to POST since the actual problem is already fixed since x.y.5 release.

Comment 14 Mirek Długosz 2020-06-19 11:41:25 UTC
Only hosts with subscriptions are included in report, report metadata matches number of hosts in slice.

Tested on:
Satellite 6.8.0 snap 5
pulp-server-2.21.2-1.el7sat.noarch
satellite-6.8.0-0.4.beta.el7sat.noarch
foreman-2.1.0-0.20.rc2.el7sat.noarch
katello-3.16.0-0.3.rc1.el7sat.noarch
tfm-rubygem-foreman_rh_cloud-2.0.7-1.el7sat.noarch

Comment 15 Mirek Długosz 2020-09-15 09:42:58 UTC
That was fixed in Satellite 6.7.2 with plugin version 1.0.9, but I gave it another round on 6.7.4. Same result - Only hosts with subscriptions are included in report, report metadata matches number of hosts in slice.


Tested on:
Satellite 6.7.4 snap 1
pulp-server-2.21.0.4-1.el7sat.noarch
foreman-1.24.1.28-2.el7sat.noarch
satellite-6.7.4-1.el7sat.noarch
katello-3.14.0-6.el7sat.noarch
tfm-rubygem-foreman_rh_cloud-1.0.10-1.el7sat.noarch

Comment 20 errata-xmlrpc 2020-09-30 13:12:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.7.4 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4127