Bug 1970057

Created attachment 1789636 [details]
Screenshot of software Updates page when not registered

That means that you still have subscription-manager and the RHEL repositories enabled? Can you please show the output of

    sudo dnf repolist

and

    sudo busctl call com.redhat.RHSM1 /com/redhat/RHSM1/Entitlement com.redhat.RHSM1.Entitlement GetStatus ss '' ''

does that say "Unknown" and "valid: false"? Unfortunately subscription-manager is not very precise wrt. disamgiguating a system which is not subscribed, but needs a subscription, from  a system which is RHEL-ish, but only uses third-party repos.

If that is the case, and your system is only using non-RHEL repositories, I'll reassign to subscription-manager. But let's confirm that first.

FTR, as you already saw we did have that "try anyway" behaviour in cockpit. But it was later decided to stop that, as adding a single third-party repo (like a COPR) would trick admins into thinking that everything was alright. So we won't go back there (that's not really technical, but more like a business/design decision).

Thanks!

The problem is not detecting if a system is registered or not. It's that the cockpit-packagekit now only works on registered systems. Customers who are not registered, who use RHUI, ubi, or local repos can not use this page at all. 

> But it was later decided to stop that, as adding a single third-party repo (like a COPR) 
> would trick admins into thinking that everything was alright. So we won't go back there 
> (that's not really technical, but more like a business/design decision).

If thats the case, then I suppose nothing can be done.. In my opinion admins can still be "tricked" by being registered and not having repositories enabled. Or having the wrong repositories enabled. I see this more often than you'd think. So we aren't preventing this behaviour, only limiting who can use this page.

Right, hence I was curious to see what repolist and sub-man GetStatus say, to see if there is anything useful that can be squeezed out of that.

Heres the output you're requesting. There's no reliably way to ensure any customer has the repositories they require or not regardless of registration. They can release lock, they can have exclude=\* in /etc/yum.conf, they can use versionlock plugins, they can register and use local repos or even no repos and still see this same output. What we have done here is stop admins from using 1, of many, methods to update a system. 

# subscription-manager status
  +-------------------------------------------+
     System Status Details
  +-------------------------------------------+
  Overall Status: Unknown

  System Purpose Status: Unknown

# sudo busctl call com.redhat.RHSM1 /com/redhat/RHSM1/Entitlement com.redhat.RHSM1.Entitlement GetStatus ss '' ''
  s "{\"status\": \"Unknown\", \"status_id\": \"unknown\", \"reasons\": {}, \"reason_ids\": {}, \"valid\": false}"

# dnf repolist -v
  baseos-local                                                8.4 MB/s |  33 MB     00:03
  appstream-local                                              11 MB/s |  30 MB     00:02
  Last metadata expiration check: 0:00:08 ago on Fri 11 Jun 2021 02:30:58 PM EDT.
  Repo-id            : appstream-local
  Repo-name          : appstream-local
  Repo-revision      : 1623323577
  Repo-updated       : Thu 10 Jun 2021 07:12:56 AM EDT
  Repo-pkgs          : 18,447
  Repo-available-pkgs: 16,616
  Repo-size          : 44 G
  Repo-baseurl       : http://rhsm-pulp.corp.redhat.com/content/dist/rhel8/8/x86_64/appstream/os
  Repo-expire        : 172,800 second(s) (last: Fri 11 Jun 2021 02:30:58 PM EDT)
  Repo-filename      : /etc/yum.repos.d/rhsm-pulp.repo

  Repo-id            : baseos-local
  Repo-name          : baseos-local
  Repo-revision      : 1623288293
  Repo-updated       : Wed 09 Jun 2021 09:24:50 PM EDT
  Repo-pkgs          : 7,732
  Repo-available-pkgs: 7,729
  Repo-size          : 11 G
  Repo-baseurl       : http://rhsm-pulp.corp.redhat.com/content/dist/rhel8/8/x86_64/baseos/os
  Repo-expire        : 172,800 second(s) (last: Fri 11 Jun 2021 02:30:48 PM EDT)
  Repo-filename      : /etc/yum.repos.d/rhsm-pulp.repo
  Total packages: 26,179


I don't understand why we would block a utility from being used because admins might be mislead if they're not registered WHEN they can do this anyways. This only limits the amount of people who can use this and make the same mistake.

I see, so you are using a repo behind the Red Hat VPN, thanks.

@dmach , @thozza , do you have some advice how Cockpit should handle this? I. e. how to ask dnf/sub-man the question "does your main package repo (say, the one from coreutils) require a subscription"? Right now we are just calling com.redhat.RHSM1.Entitlement.GetStatus and check the returned status.valid bool.

Thanks!

(In reply to Martin Pitt from comment #6)
> I see, so you are using a repo behind the Red Hat VPN, thanks.
> 
> @dmach , @thozza , do you have some advice how Cockpit
> should handle this? I. e. how to ask dnf/sub-man the question "does your
> main package repo (say, the one from coreutils) require a subscription"?
> Right now we are just calling com.redhat.RHSM1.Entitlement.GetStatus and
> check the returned status.valid bool.
> 
> Thanks!

Based on my experience as a person not working primarily on DNF/Subscription-manager, there is no notion of a "main" RPM repo on the system. Subscription-manager maintains the /etc/yum.repos.d/redhat.repo content, which lists the sslclientkey and sslclientcert for each repository. In osbuild/osbuild-composer, we take this as an indication that a subscription is needed and pair the repo with available entitlement certificates on the system. However this does not mean that the user can't have a custom .repo file defined with available content. Subscription-manager AFAIK won't tell you if the system has access to any content without a subscription. It can tell you only that the system is, or is not registered using subscription-manager.

However I think that the definitive answer should be provided by someone on the sub-man team, e.g. Chris (csnyder)...

Thanks Tomáš! I am trying to reconcile "The existence of /etc/yum.repos.d/redhat.repo indicates that a subscription is needed" with the original description's "when a system is pulling from UBI, RHUI, or Local repositories, they do not have to be registered to access content". So on such a system you would have redhat.repo (which would point to the inaccessible) Red Hat CDN, and some *additional* yum.repos.d/ubi.repo or similar which would *also* have the basic system packages? If such systems are such a common configuration (like, public EC2 AMIs or similar), why do they even configure subscription-manager and redhat.repo if it's not being used? Or does redhat.repo contain these alternative (UBI/local/etc) repos then?

Sorry for my dumb questions, I'm afraid subscriptions still confuse me to no end..

>  there is no notion of a "main" RPM repo on the system.

This is probably not 100% accurate to pin down, but "from which repository is coreutils installed" should be a fairly good heuristic at least for a human.

> So on such a system you would have redhat.repo (which would point to the inaccessible) Red Hat CDN, and some *additional* yum.repos.d/ubi.repo or similar which would *also* have the basic system packages? 

if subscription-manager is installed, then the /etc/yum/pluginconf.d/subscription-manager.conf plugin is enabled by default. When yum is run, this automatically updates the redhat.repo file. If you have no entitlements or you are not registered, it makes an empty file.

The ubi, rhui, and local repos are either identical or near identical copies of the CDN repos. 

> If such systems are such a common configuration (like, public EC2 AMIs or similar), why do they even configure subscription-manager and redhat.repo if it's not being used? 

It's part of the @core group which means that it's installed by default on all systems (except ubi who doesn't have to have it installed but still can if they want to)

  # repoquery --groupmember subscription-manager
    subscription-manager-1.28.13-2.el8.x86_64
      @core

> Or does redhat.repo contain these alternative (UBI/local/etc) repos then?

They are not configured in redhat.repo. They have their own .repo files. 

=======================================

> I. e. how to ask dnf/sub-man the question "does your main package repo (say, the one from coreutils) require a subscription"?

The problem here is that we can not control what the repository contents are. We don't know if they've rebuilt the packages, if they are 3rd party, if the repository contains security information to identify which packages are security errata or not. 

* You can use a DVD as a repo
* You can exclude all the important packages
* You could have RHEL 8 content available on a RHEL 7 system
* You could build a repo without the updateinfo.xml and it will never say anything is security related. 
* You could build a repo with the latest updateinfo.xml but not the latest packages
* They could be release locked to 7.5 and think they have the latest content.

Right now they just need to register. Then they can disable all the redhat repos and use whatever they want. 
If you check for a repo with coreutils, then they will build a repo with 1 package, coreutils, and then they can use the functionality of the page with their repos. 

We can not ensure this utility is providing anything accurate. Customers can configure 100 ways to really mess this up and we can't stop that. (If I ask you to bake a cake, and you don't shop at the store I like, should you still be allowed to bake a cake? Right now you're not even allowed to look at the recipe [per this strange analogy])

All it can do is help them to remotely update their systems using the configurations they have in place.

Thanks jcatran. I'm not really concerned about users "hacking around" that test, like creating a repo with a single coreutils. You can more easily achieve that by simply disabling sub-man. What we want to do is to warn people if their system can't be updated because of missing subscriptions. Adding a COPR to an unregistered RHEL system and getting two updates of your pencil ordering app every year should *not* constitute as "proper upgrades", but without the sub-man check this would be exactly what happens -- cockpit would say "your system is up to date" (or "1 update available") when it's really badly broken.

So with that in mind I am really none the wiser -- with this design, I suppose the bottom line is "if you use RHEL, subscribe the machine or disable sub-man"? (Better ideas much appreciated!)

Disabling subscription-manager does not clear this issue for me unless I'm doing this wrong. If you are not registered you can't access that software updates page. 

Do I have to completely remove subscription-manager to access this page without being registered? Disabling the plugin and no redhat.repo file while being unregistered still tells me only to register and shows me nothing else

Indeed, since [1] it queries com.redhat.RHSM1.Products.ListInstalledProducts, and this gets D-Bus activated (/usr/share/dbus-1/system-services/com.redhat.RHSM1.service). cockpit used to check the dnf plugin, but unfortunately 
/etc/yum/pluginconf.d/subscription-manager.conf is not really a configuration file, but a dynamically updated state file, so checking that for the plugin was broken [2] and we were told to query ListInstalledProducts instead.

So I think disabling without installing would only work through `systemctl mask rhsm`.

If this is a big issue, we can also give up and say "look, dnf and sub-man fundamentally don't understand each other", and treat the update list and "not subscribed" warning as independent. It's just after like 3 decades of RHEL I find it hard to accept that this problem has never been solved properly? This is why I'm asking around for some better idea.

As I said, my current idea is "query PackageKit if there is any available version of coreutils" if sub-man says "not registered". If there is, then show available updates. The "not registered" warning would appear in either case, though. Is that something you could live with?


[1] https://github.com/cockpit-project/cockpit/commit/de780972ad9
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1931429

> So I think disabling without installing would only work through `systemctl mask rhsm`.

Of course that was meant to be "without *un*installing". And I don't think that's something which we should officially document/recommend, it was just a reply to your question. We clearly need a better solution here :-)

If subscription-manager is installed then the service is by default enabled. If we're willing to have an option to bypass this check, can we instead make it a warning "you are not pulling from the CDN and the validity of this errata check can not be guaranteed. Please register to the CDN to ensure the latest updates or errata have been applied". 

Otherwise anyone who installs RHEL which pulls @core by default, must stop the service and then mask it to access this page. As opposed to have a warning. Something similar to when you go to an insecure page and the browser asks if you want to continue. "You are not pulling from the CDN and/or you are not registered, do you wish to continue".

(In reply to jcastran from comment #14)
> If subscription-manager is installed then the service is by default enabled.

The rhsmd D-Bus service, yes.

> If we're willing to have an option to bypass this check

What do you mean by "this" check here?

The current check that cockpit does is "does this machine have any installed RHEL products". So far this was the best approximation of "does this machine require a subscription to get proper updates?" that we know of. As this bug report shows, this is not good enough, and we need to refine this.

Or do you mean something else, like subscription-manager.conf or redhat.repo?

>, can we instead make
> it a warning "you are not pulling from the CDN and the validity of this
> errata check can not be guaranteed. Please register to the CDN to ensure the
> latest updates or errata have been applied". 

Yes, that's roughly what I was proposing. But we need to use better verbiage -- as such it is factually wrong. As far as I understand, at least the UBI and the cloud specific mirrors (like on EC2) are "proper" RHEL mirrors, come configured *by default*, and as such should be considered a bona fide RHEL system. Showing this kind of verbiage is both confusing and inactionable, as e.g. on an EC2 RHEL instance we actually *want* users to pull from the EC2 mirror, not from the RHEL CDN. Also, we should not say "CDN" in the UI, as that is (1) an internal implementation detail, (2) meaningless to someone who is not familiar with the term, and (3) ambiguous for someone who does know what a CDN is.

In order to give a meaningful text, we still need to differ between the cases of "RHEL CDN repos with additional non-RHEL repos" vs. "RHEL core packages are pulled from a non-RHEL repo". If anyone has a better idea than "check if coreutils is available from any repo", then please speak up.

Thanks,

Martin

No further comments, so I'll go with the above heuristics with coreutils. Cleaning up needinfo RFCs.

Just for posterity and tracing history:

> This was an intentional change we would like undone or modified to not prevent updates. 
> https://github.com/cockpit-project/cockpit/commit/58459bdca885aa41a2fa6b8fa36a3c647b24b13a

That commit still showed the available updates (usually partial), showed a warning about the missing subscription, and disabled the action buttons. That behaviour was found to be bad in UX testing (https://github.com/cockpit-project/cockpit/issues/9926) and made stricter in https://github.com/cockpit-project/cockpit/commit/6451c58679153ba to always just show the empty state in the unregistered case.

https://github.com/cockpit-project/cockpit/pull/16151

Fixing component, this is in AppStream.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (cockpit-appstream bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4178