RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1487352 - BUG: the kernel does not initialize audit before forking PID 1
Summary: BUG: the kernel does not initialize audit before forking PID 1
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel
Version: 7.4
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Paul Moore
QA Contact: Ondrej Moriš
URL:
Whiteboard:
: 1478517 1478521 1478528 (view as bug list)
Depends On:
Blocks: 1476406 1966454
TreeView+ depends on / blocked
 
Reported: 2017-08-31 17:33 UTC by Paul Moore
Modified: 2022-08-23 16:51 UTC (History)
3 users (show)

Fixed In Version: kernel-3.10.0-738.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1966454 (view as bug list)
Environment:
Last Closed: 2018-04-10 21:57:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
01-bz1487352-1.patch (2.46 KB, patch)
2017-10-05 13:31 UTC, Paul Moore 		no flags Details | Diff
01-bz1487352-1.patch (2.46 KB, patch)
2017-10-05 13:32 UTC, Paul Moore 		no flags Details | Diff
Show Obsolete (1) View All


Links
System ID Private Priority Status Summary Last Updated
Github linux-audit audit-kernel issues 66 0 None closed BUG: the kernel does not initialize audit before forking PID 1 2020-02-10 10:51:47 UTC
Red Hat Product Errata RHSA-2018:1062 0 None None None 2018-04-10 21:58:45 UTC

Description Paul Moore 2017-08-31 17:33:04 UTC 
Description of problem:

Taken from the upstream GitHub issue:

 "The kernel needs to initialize the audit subsystem before forking init/systemd
  so that the audit subsystem can allocate an audit_context for the process, 
  enabling the collection of audit data if configured (e.g. audit=1 on the kernel 
  command line).

  The current audit code is initialized via the generic kernel initcall 
  mechanism, which is eventually called after the kernel forks PID 1 (PID 1 -> 
  kernel_init(...) -> kernel_init_freeable(...) -> do_basic_setup(...) -> 
  do_initcalls(...)). We most likely want to add a call to audit_init(...) before 
  key_init(...) or security_init(...) in start_kernel(...)."

Additional info:

Upstream GitHub issue:

* https://github.com/linux-audit/audit-kernel/issues/66
Comment 2 Steve Grubb 2017-10-04 16:00:40 UTC 
*** Bug 1478517 has been marked as a duplicate of this bug. ***
Comment 3 Steve Grubb 2017-10-04 16:01:29 UTC 
*** Bug 1478521 has been marked as a duplicate of this bug. ***
Comment 4 Steve Grubb 2017-10-04 16:02:06 UTC 
*** Bug 1478528 has been marked as a duplicate of this bug. ***
Comment 5 Paul Moore 2017-10-05 13:22:33 UTC 
For reference, this is the commit that should fix the problem; it currently lives in the audit/next branch and should go into Linux v4.15.

  commit d3b09ee2b47df7bb97f1b72a71d73835f79cd958
  Author: Paul Moore <paul>
  Date:   Fri Sep 1 09:44:34 2017 -0400

    audit: ensure that 'audit=1' actually enables audit for PID 1
    
    Prior to this patch we enabled audit in audit_init(), which is too
    late for PID 1 as the standard initcalls are run after the PID 1 task
    is forked.  This means that we never allocate an audit_context (see
    audit_alloc()) for PID 1 and therefore miss a lot of audit events
    generated by PID 1.
    
    This patch enables audit as early as possible to help ensure that when
    PID 1 is forked it can allocate an audit_context if required.
    
    Reviewed-by: Richard Guy Briggs <rgb>
    Signed-off-by: Paul Moore <paul>
Comment 6 Paul Moore 2017-10-05 13:31:35 UTC 
Created attachment 1334814 [details]
01-bz1487352-1.patch
Comment 7 Paul Moore 2017-10-05 13:32:38 UTC 
Created attachment 1334815 [details]
01-bz1487352-1.patch
Comment 8 Paul Moore 2017-10-05 15:27:59 UTC 
Testing notes, pay attention to the extra records (PROCTITLE, SYSCALL) in the working kernel.

* Current RHEL-7.x kernel (BAD)

 # uname -r
 3.10.0-709.el7.x86_64
 # grep "^SELINUX=" /etc/selinux/config 
 SELINUX=enforcing
 # grep "audit=1" /proc/cmdline 
 BOOT_IMAGE=/vmlinuz-3.10.0-709.el7.x86_64 root=UUID=84615751-d377-4530-a005-
 d24ff46b5c19 ro rd.md=0 rd.lvm=0 rd.dm=0 rd.luks=0 vconsole.keymap=us 
 nomodeset 
 console=ttyS0 LANG=en_US.UTF-8 audit=1
 # ausearch -ts boot -m MAC_STATUS
 ----
 time->Thu Oct  5 11:22:22 2017
 type=MAC_STATUS msg=audit(1507216942.353:40): enforcing=1 old_enforcing=0 
 auid=4294967295 ses=4294967295

* Patches RHEL-7.x kernel (GOOD)

 # uname -r
 3.10.0-725.el7.1.bz1487352.x86_64
 # grep "^SELINUX=" /etc/selinux/config
 SELINUX=enforcing
 # grep "audit=1" /proc/cmdline
 BOOT_IMAGE=/vmlinuz-3.10.0-725.el7.1.bz1487352.x86_64 root=UUID=84615751-
 d377-4530-a005-d24ff46b5c19 ro rd.md=0 rd.lvm=0 rd.dm=0 rd.luks=0 
 vconsole.keymap=us nomodeset console=ttyS0 LANG=en_US.UTF-8 audit=1
 # ausearch -ts boot -m MAC_STATUS
 ----
 time->Thu Oct  5 11:26:21 2017
 type=PROCTITLE msg=audit(1507217181.241:40): 
 proctitle=2F7573722F6C69622F73797374656D642F73797374656D64002D2D73776974636865
 642D726F6F74002D2D73797374656D002D2D646573657269616C697A65003231
 type=SYSCALL msg=audit(1507217181.241:40): arch=c000003e syscall=1 success=yes 
 exit=1 a0=3 a1=7fffdd628a30 a2=1 a3=7fffdd628740 items=0 ppid=0 pid=1 
 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=
 (none) ses=4294967295 comm="systemd" exe="/usr/lib/systemd/systemd" 
 subj=kernel key=(null)
 type=MAC_STATUS msg=audit(1507217181.241:40): enforcing=1 old_enforcing=0 
 auid=4294967295 ses=4294967295
Comment 11 Rafael Aquini 2017-10-17 17:48:13 UTC 
Patch(es) committed on kernel repository and an interim kernel build is undergoing testing
Comment 13 Richard Guy Briggs 2017-10-17 18:11:29 UTC 
This is not a bug in RHEL7 because the patch that broke things upstream:
  https://github.com/linux-audit/audit-kernel/commit/d3ca0344b21f04786219bf0f49647f24e4e17323
was never backported.  Recommend closing NOTABUG.
Comment 14 Paul Moore 2017-10-17 18:32:46 UTC 
Richard, see comment #8.
Comment 15 Richard Guy Briggs 2017-10-17 21:13:14 UTC 
(In reply to Paul Moore from comment #14)
> Richard, see comment #8.

Ok, my bad.  I had seen comment 8 but evidently didn't let it sink in.  I thought I had identified when it broke (because I remember it working at one point), but that isn't it on further inspection.  Perhaps it was other changes that moved around initialization of other subsystems that changed this startup timing.

From your testing this appears to be the minimum necessary to fix this issue.  The other 4 patches from that set (plus the one mentioned above) would make it tidier, but are evidently not needed.

I'm surprised this works alone since the rest of the audit infrastructure isn't ready until audit_initialized is set.  Other processes will just bounce off audit_log_start() until that is set anyways as it should have until audit_enabled was set just after audit_initialized.
Comment 16 Paul Moore 2017-10-17 21:39:56 UTC 
(In reply to Richard Guy Briggs from comment #15)
> (In reply to Paul Moore from comment #14)
> > Richard, see comment #8.
> 
> Ok, my bad.  I had seen comment 8 but evidently didn't let it sink in ...

I guess it doesn't hurt, but I don't think you need to repeat what is said on RHKL here, you can simply reference the patchwork link in comment #10.  Anyway, here is my RHKL response:

"You are focusing on the wrong thing, the key with the init problem
 lies with audit_ever_enabled and the audit_alloc() function (see the
 commit description).  We need to make sure audit_ever_enabled is set
 *before* we fork for the init process; if audit_ever_enabled is not
 set before we fork, audit_alloc() never allocates an audit_context
 struct for the task.

 As you say, early audit_log_start() calls are going to fail, but that
 isn't the problem."
Comment 17 Steve Grubb 2017-10-17 21:43:54 UTC 
Paul, you're right. It's not _the_ problem but it is a problem. I thought you said you moved the module initialization up to earlier in the boot. Did that not make it as part of this set?
Comment 18 Paul Moore 2017-10-17 21:54:43 UTC 
(In reply to Steve Grubb from comment #17)
> Paul, you're right. It's not _the_ problem but it is a problem. I thought
> you said you moved the module initialization up to earlier in the boot. Did
> that not make it as part of this set?

Steve that change does not have any bearing on this issue.  If you are concerned because you heard about audit_log_start() failing on early boot, rest assured those have always been failing.  We are talking about a small corner case, I'm not even sure there are any affected audit_log_start() calls (maybe, I haven't checked).

If you have a specific problem Steve, open either a BZ (RHEL-7.x) or a GH issue (upstream); you know the drill.
Comment 19 Rafael Aquini 2017-10-18 19:08:20 UTC 
Patch(es) available on kernel-3.10.0-738.el7
Comment 21 Ondrej Moriš 2017-12-20 12:51:35 UTC 
Successfully reproduced and verified on x86_64.

OLD (kernel-3.10.0-693.el7)
===========================
type=MAC_STATUS msg=audit(12/20/2017 07:27:55.266:40) : enforcing=1 old_enforcing=0 auid=unset ses=unset 

NEW (kernel-3.10.0-823.el7)
===========================
type=PROCTITLE msg=audit(12/20/2017 07:11:39.221:40) : proctitle=/usr/lib/systemd/systemd --switched-root --system --deserialize 22 
type=SYSCALL msg=audit(12/20/2017 07:11:39.221:40) : arch=x86_64 syscall=write success=yes exit=1 a0=0x3 a1=0x7ffe4ae79f30 a2=0x1 a3=0x7ffe4ae799a0 items=0 ppid=0 pid=1 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=systemd exe=/usr/lib/systemd/systemd subj=kernel key=(null) 
type=MAC_STATUS msg=audit(12/20/2017 07:11:39.221:40) : enforcing=1 old_enforcing=0 auid=unset ses=unset 

For more details please see TJ#2212673 and TJ#2212678.
Comment 22 errata-xmlrpc 2018-04-10 21:57:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1062
Note You need to log in before you can comment on or make changes to this bug.