Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2229997

Summary: libdnf may create /run/user/0 directory, causing a bad context to be applied, leading to further issues
Product: Red Hat Enterprise Linux 8 Reporter: Renaud Métrich <rmetrich>
Component: libdnfAssignee: Petr Pisar <ppisar>
Status: CLOSED MIGRATED QA Contact: swm-qe
Severity: high Docs Contact:
Priority: high    
Version: 8.8CC: ppisar
Target Milestone: rcKeywords: MigratedToJIRA, Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-21 18:06:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Renaud Métrich 2023-08-08 12:51:21 UTC
Description of problem:

This is observed on a customer system when insights-client executes.
In this specific case (but there are likely other cases), the /run/user/0 directory gets created with insights_client_tmp_t context, causing systemd's user-runtime-dir unit to fail forever:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
Aug 08 13:24:22 vm-insights8 systemd[1]: Stopping User runtime directory /run/user/0...
Aug 08 13:24:22 vm-insights8 systemd-user-runtime-dir[34749]: Failed to remove runtime directory /run/user/0 (after unmounting): Permission denied
Aug 08 13:24:22 vm-insights8 systemd[1]: user-runtime-dir: Control process exited, code=exited status=1
Aug 08 13:24:22 vm-insights8 systemd[1]: user-runtime-dir: Failed with result 'exit-code'.
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

The root cause for this is the libdnf code forcibly creates the /run/user/0 directory when trying to import GPG keys:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
 865 static void ensure_socket_dir_exists() {
 866     auto logger(Log::getLogger());
 867     std::string dirname = "/run/user/" + std::to_string(getuid());
 868     int res = mkdir(dirname.c_str(), 0700);     <<<<<<< HERE
 869     if (res != 0 && errno != EEXIST) {
 870         logger->debug(tfm::format("Failed to create directory \"%s\": %d - %s",
 871                                   dirname, errno, strerror(errno)));
 872     }
 873 }
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

This is not an issue when /run/user/0 already exists since mkdir is then a no-op, but this is not always the case.
For example, when the root user has no session, this is an issue if a libdnf client, such as yum, executes through sudo (e.g. "sudo yum check-update").
This code then creates the directory in the context of the caller, which may be problematic when there is no transition rule in the SELinux policy to automatically create the directory with proper "user_tmp_t" context.

In the case of "insights-client", the context is "insights_client_t", which leads to creating the directory with "insights_client_tmp_t" context, which then prevents systemd from deleting that directory once root user logged in and logged out again.


Version-Release number of selected component (if applicable):

libdnf-0.63.0-14.el8_8.x86_64

How reproducible:

Always when executing in "insights_client_t" contxt

Steps to Reproduce:
1. Setup EPEL and REMI repositories (to get the key for REMI repo imported)

  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  # yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
  # yum install https://rpms.remirepo.net/enterprise/remi-release-8.rpm
  # yum clean all
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

2. Create a wrapper to execute libdnf as a service running as if it was executed from insights-client

  You need to have insights-client package installed to get access to the context.

  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  # cat > /usr/local/bin/fake_insights << EOF
  #!/bin/sh
  exec /usr/libexec/platform-python /usr/bin/yum -y check-update
  EOF

  # chmod +x /usr/local/bin/fake_insights
  # chcon -t insights_client_exec_t /usr/local/bin/fake_insights
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

  Note: here above we use "/usr/libexec/platform-python" to avoid the transition to "rpm_t".

3. Create an admin user which will start the wrapper in insights-client context

  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  # useradd -G wheel admin
  # echo "redhat" | passwd --stdin admin
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

4. Login as "admin" and make sure no root session is present

  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  $ ssh admin@system
  $ sudo systemctl stop user-0.slice
  $ ls -Zd /run/user/0
  ls: cannot access '/run/user/0': No such file or directory
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

5. Execute "yum check-update" in insights-client context and wait for /run/user/0 to be created

  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  $ sudo systemd-run /usr/local/bin/fake_insights
  $ while :; do date; ls -Z /run/user; sleep 1; done
  Tue Aug  8 14:26:07 CEST 2023
  system_u:object_r:user_tmp_t:s0 1000
  [...]
  system_u:object_r:user_tmp_t:s0 1000
  Tue Aug  8 14:26:12 CEST 2023
  system_u:object_r:insights_client_tmp_t:s0 0             system_u:object_r:user_tmp_t:s0 1000
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Actual results:

  Here above the directory got created with bad context.

Expected results:

  No directory /run/user/0 created at all or proper "user_tmp_t" context set.

Additional info:

  You may also reproduce with executing "subscription-manager repos --disable \*" instead of "yum" in the wrapper:
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
  # cat > /usr/local/bin/fake_insights << EOF
  #!/bin/sh
  exec subscription-manager repos --disable \*
  EOF

  # chmod +x /usr/local/bin/fake_insights
  # chcon -t insights_client_exec_t /usr/local/bin/fake_insights
  -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

  IMHO it's not up to libdnf to create the /run/user/0 directory but systemd and this should be considered as a systemd-private thing.
  I would expect the GPG command to not populate this directory at all either.
  The comment above function ensure_socket_dir_exists() tends to indicate this thing is a workaround for GPG issues actually.

  Not creating this directory is the only reliable solution to make sure whatever the caller is, this will work.

Comment 1 Petr Pisar 2023-09-08 12:41:22 UTC
A workaround is adding an explicit SELinux fcontext definition for that directory:

   # semanage fcontext -a -t user_tmp_t '/var/run/user/0'

Comment 2 Renaud Métrich 2023-09-08 12:44:40 UTC
Customers cannot implement the workaround because /var/run/user/0 is managed by systemd, hence will be removed if the last root session exited.
The issue then happens if insights-client executes while there is no remaining root session.

The solution I gave to the customer was to set lingering for the root user, so that the directory is always (mounted and) there.

Comment 3 Petr Pisar 2023-09-08 12:48:52 UTC
My work around assures that the directory is created with the expected context type when there is transition rule defined. That's what happens when insights client executes dnf. Doesn't?
I think your workaround has a gap between boot and loggin in root for the first time.

Comment 4 Renaud Métrich 2023-09-08 13:25:40 UTC
This won't work because insights_client_t has a transition built-in:
~~~
# sesearch -T -s insights_client_t -c dir -t user_tmp_t
[...]
type_transition insights_client_t user_tmp_t:dir insights_client_tmp_t;
[...]
~~~

The fcontext database is anyway not used by types at all, it's just a static database used by matchpathcon/chcon/restorecon binaries.

Comment 5 Petr Pisar 2023-09-13 11:53:40 UTC
According to DNF developers, DNF creates /run/user/0 for GnuPG as a workaround for containers where it does not exists and other locations where GnuGP searches an agent socket are read only. This is a list RHEL-8 GnuPG searches for <BASE>/user/<UID>:

  static const char * const bases[] = {
#ifdef USE_RUN_GNUPG_USER_SOCKET
    "/run/gnupg",
#endif
    "/run",
#ifdef USE_RUN_GNUPG_USER_SOCKET
    "/var/run/gnupg",
#endif
    "/var/run",
    NULL
  };

USE_RUN_GNUPG_USER_SOCKET is undefined according to RHEL's gnupg2.spec. It is documented in GnuPG's configure.ac like this:

# To avoid problems with systemd cleaning up the /run/user directory,
# this option will make GnuPG try to use /run/gnupg/user as socket dir
# before /run/user

I think DNF cannot resort to a path after /run because in case first an insight client run, and then a user logged in, there were two independent agent sockets. If gnupg2.spec enabled USE_RUN_GNUPG_USER_SOCKET with --enable-run-gnupg-user-socket option, DNF could resort to creating /run/gnupd/user/0 which is not managed by systemd (except that /run is a tmpfs mounted by systemd).

Comment 6 RHEL Program Management 2023-09-21 17:33:06 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 7 RHEL Program Management 2023-09-21 18:06:01 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.