RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2124991 - insights-client collection, wrong version of running image reported to edge management console
Summary: insights-client collection, wrong version of running image reported to edge m...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: selinux-policy
Version: 9.0
Hardware: Unspecified
OS: Linux
high
medium
Target Milestone: rc
: 9.2
Assignee: Zdenek Pytela
QA Contact: Milos Malik
URL:
Whiteboard: SCRUB_20220926
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-09-07 16:32 UTC by Micah Abbott
Modified: 2023-05-09 10:20 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-09 08:16:34 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)
screen shot from edge management inventory showing incorrect running version (161.88 KB, image/png)
2022-09-07 16:32 UTC, Micah Abbott 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-133470 0 None None None 2022-09-07 16:53:04 UTC
Red Hat Product Errata RHBA-2023:2483 0 None None None 2023-05-09 08:16:49 UTC

Description Micah Abbott 2022-09-07 16:32:57 UTC 
Created attachment 1910263 [details]
screen shot from edge management inventory showing incorrect running version

(I think it is the `rhc` client that reports this info up to the edge management console, but please reassign appropriately)

I've followed the documentation for installing RHEL for Edge, creating an update for the device, and applying the update to the device.  See: https://access.redhat.com/documentation/en-us/edge_management/2022/html/create_rhel_for_edge_images_and_configure_automated_management/proc-rhem-auto-reg

After applying the update, I went to the Edge Management section of console.redhat.com to investigate the details of the device.  See https://console.redhat.com/edge/inventory/28cbec55-d113-4309-b75e-7fe9cc8ea558?page=1&page_size=20&show_advisories=true&sort=-public_date


Under the "Image information" section, the running image is reported correctly as `miabbott`, but the running version is incorrectly reported as `3`

miabbott image - https://console.redhat.com/edge/manage-images/486/details
v3 - https://console.redhat.com/edge/manage-images/486/versions/967/details
v4 - https://console.redhat.com/edge/manage-images/486/versions/987/details

Note the v3 ostree commit is `6b10f6d0cb6a383b6207599fae5adbe3542dd6e59775c6597e8915b7b407feed` and the v4 ostree commit is `e2e37310ab91b088d2c6150f8ce7d77d533c8725cc86dd5a57951f3ca2ce7b64`

When I inpect the Edge device directly via ssh, the output of `rpm-ostree status` shows the running commit as the one that corresponds to v4.

```
[core@localhost ~]$ sudo rhc status
Connection status for localhost.localdomain:

● Connected to Red Hat Subscription Management
● Connected to Red Hat Insights
● The Red Hat connector daemon is active

Manage your Red Hat connector systems: https://red.ht/connector
[core@localhost ~]$ rpm-ostree status
State: idle
Deployments:
● rhel-edge:rhel/9/x86_64/edge
                   Version: 9.0 (2022-09-06T20:47:36Z)
                    Commit: e2e37310ab91b088d2c6150f8ce7d77d533c8725cc86dd5a57951f3ca2ce7b64

  rhel-edge:rhel/9/x86_64/edge
                   Version: 9.0 (2022-08-31T19:51:18Z)
                    Commit: 6b10f6d0cb6a383b6207599fae5adbe3542dd6e59775c6597e8915b7b407feed
```

I've been unable to debug/triage what process on the host is collecting the information or how it is being sent to console.redhat.com, but would be happy to collect more data about this.
Comment 1 Micah Abbott 2022-09-07 16:38:04 UTC 
Also worth noting that the console is indicating that an update is available, but the device is already running the latest version.
Comment 3 Alba Hita 2022-10-13 09:28:21 UTC 
Can you share the insights-client logs and the SELinux denials:

1. Run insights-client
>>>sudo insights-client

Logs are found in /var/log/insights-client.log

2. Search for SELinux denials
>>> ausearch -m AVC -ts today
Comment 4 Micah Abbott 2022-10-13 19:50:19 UTC 
(In reply to Alba Hita from comment #3)
> Can you share the insights-client logs and the SELinux denials:
> 
> 1. Run insights-client
> >>>sudo insights-client
> 
> Logs are found in /var/log/insights-client.log
> 
> 2. Search for SELinux denials
> >>> ausearch -m AVC -ts today

I no longer have the VM where this was observed at the ready.  I'll have to try to recreate the error conditions.
Comment 5 Micah Abbott 2022-10-14 20:51:17 UTC 
(In reply to Alba Hita from comment #3)
> Can you share the insights-client logs and the SELinux denials:
> 
> 1. Run insights-client
> >>>sudo insights-client
> 
> Logs are found in /var/log/insights-client.log

```
$ sudo insights-client
Starting to collect Insights data for localhost.localdomain
Uploading Insights data.
Successfully uploaded report from localhost.localdomain to account 1460290.
View details about this system on console.redhat.com:
https://console.redhat.com/insights/inventory/f33fb461-3d45-4f70-8b84-6b58397b70fc
```

See attached `insights-client.log`

After running the command, I checked the console and found that the device was properly updated to reflect it is running the most up-to-date version of the image

https://console.redhat.com/edge/inventory/f33fb461-3d45-4f70-8b84-6b58397b70fc

> 
> 2. Search for SELinux denials
> >>> ausearch -m AVC -ts today

Definitely some denials in there; see attached `ausearch.log`



Perhaps in the original problem, the insights-client wasn't properly sending updates to the console.redhat.com backend?
Comment 9 Alba Hita 2022-10-17 09:27:33 UTC 
(In reply to Micah Abbott from comment #5)
> (In reply to Alba Hita from comment #3)
> > Can you share the insights-client logs and the SELinux denials:
> > 
> > 1. Run insights-client
> > >>>sudo insights-client
> > 
> > Logs are found in /var/log/insights-client.log
> 
> ```
> $ sudo insights-client
> Starting to collect Insights data for localhost.localdomain
> Uploading Insights data.
> Successfully uploaded report from localhost.localdomain to account 1460290.
> View details about this system on console.redhat.com:
> https://console.redhat.com/insights/inventory/f33fb461-3d45-4f70-8b84-
> 6b58397b70fc
> ```
> 
> See attached `insights-client.log`
> 
> After running the command, I checked the console and found that the device
> was properly updated to reflect it is running the most up-to-date version of
> the image
> 
> https://console.redhat.com/edge/inventory/f33fb461-3d45-4f70-8b84-
> 6b58397b70fc
> 
> > 
> > 2. Search for SELinux denials
> > >>> ausearch -m AVC -ts today
> 
> Definitely some denials in there; see attached `ausearch.log`
> 
> 
> 
> Perhaps in the original problem, the insights-client wasn't properly sending
> updates to the console.redhat.com backend?

Can you also paste here the version of the SELinux policies?

Yes it seems that SELinux is preventing to upload the archive, you can see in insights-client logs. We are facing some issues with the new SELinux policies that makes insights-client not working as expected.
I'm adding Zdenek and Milos as followers as they are working on SELinux fixes.
Comment 10 Zdenek Pytela 2022-10-17 11:02:26 UTC 
Micah,

What was the selinux-policy version?
It is likely these issues have already been addressed with selinux-policy-3.14.3-103.el8.
Comment 11 Zdenek Pytela 2022-10-17 12:05:48 UTC 
(In reply to Zdenek Pytela from comment #10)
> It is likely these issues have already been addressed with
> selinux-policy-3.14.3-103.el8.

This was RHEL 8 version, in RHEL 9 it is selinux-policy-34.1.43-1, sorry for that.
Comment 12 Micah Abbott 2022-10-17 13:18:28 UTC 
(In reply to Zdenek Pytela from comment #11)
> (In reply to Zdenek Pytela from comment #10)
> > It is likely these issues have already been addressed with
> > selinux-policy-3.14.3-103.el8.
> 
> This was RHEL 8 version, in RHEL 9 it is selinux-policy-34.1.43-1, sorry for
> that.

```
$ rpm -q selinux-policy
selinux-policy-34.1.29-1.el9_0.2.noarch
```
Comment 13 Zdenek Pytela 2022-10-17 14:06:51 UTC 
(In reply to Micah Abbott from comment #12)
> (In reply to Zdenek Pytela from comment #11)
> > (In reply to Zdenek Pytela from comment #10)
> > > It is likely these issues have already been addressed with
> > > selinux-policy-3.14.3-103.el8.
> > 
> > This was RHEL 8 version, in RHEL 9 it is selinux-policy-34.1.43-1, sorry for
> > that.
> 
> ```
> $ rpm -q selinux-policy
> selinux-policy-34.1.29-1.el9_0.2.noarch
> ```

Can you confirm the issue is gone with the latest selinux-policy package available?
It is selinux-policy-34.1.43-1 in RHEL 9.1 and selinux-policy-34.1.44-1 which is available for RHEL 9.2.
Comment 14 Micah Abbott 2022-10-20 19:18:05 UTC 
(In reply to Zdenek Pytela from comment #13)
> (In reply to Micah Abbott from comment #12)
> > (In reply to Zdenek Pytela from comment #11)
> > > (In reply to Zdenek Pytela from comment #10)
> > > > It is likely these issues have already been addressed with
> > > > selinux-policy-3.14.3-103.el8.
> > > 
> > > This was RHEL 8 version, in RHEL 9 it is selinux-policy-34.1.43-1, sorry for
> > > that.
> > 
> > ```
> > $ rpm -q selinux-policy
> > selinux-policy-34.1.29-1.el9_0.2.noarch
> > ```
> 
> Can you confirm the issue is gone with the latest selinux-policy package
> available?
> It is selinux-policy-34.1.43-1 in RHEL 9.1 and selinux-policy-34.1.44-1
> which is available for RHEL 9.2.

I was able to construct an ostree commit that includes `selinux-policy-34.1.43-1.el9.noarch` and other 9.1 content.  But I had to do that using an on-prem Image Builder setup, since the 9.1 content isn't available on the IB instance on console.redhat.com (as far as I know).  This presents a problem when `insights-client` reports about the version of the running image since it has no history or knowledge of the commit I created on the on-prem IB setup.

Anyways, I still see some AVC denials.  I'll attach an updated set of logs.
Comment 25 errata-xmlrpc 2023-05-09 08:16:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (selinux-policy bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2483
Note You need to log in before you can comment on or make changes to this bug.