RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1873492 - kernel lockdown blocks debugfs ops used by stap/staprun/stapio to talk to signed kernel module
Summary: kernel lockdown blocks debugfs ops used by stap/staprun/stapio to talk to sig...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: systemtap
Version: 8.3
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: 8.3
Assignee: Frank Ch. Eigler
QA Contact: Martin Cermak
Oss Tikhomirova
URL:
Whiteboard:
Depends On: 1805299
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-28 13:53 UTC by Martin Cermak
Modified: 2021-09-17 14:55 UTC (History)
7 users (show)

Fixed In Version: systemtap-4.4-3.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-18 15:44:32 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Martin Cermak 2020-08-28 13:53:22 UTC
systemtap-4.3-4.el8 doesn't seem to cope with secureboot on rhel-8.3:

=======
[root@localhost ~]# stap --trust-servers=ssl,signer,all-users,no-prompt --use-server=$SERVER_IP:$SERVER_PORT
Adding trust in the following servers as an SSL peer for all users and as a module signer for all users
   host=unknown address=127.0.0.1 port=33645 sysinfo="unknown" version=unknown certinfo="unknown"
[root@localhost ~]# stap --use-server=$SERVER_IP:$SERVER_PORT -v -e 'probe oneshot { log("hey") }'
Using a compile server.
Pass 1: parsed user script and 485 library scripts using 452652virt/85596res/12876shr/72180data kb, in 160usr/40sys/227real ms.
Pass 2: analyzed script: 1 probe, 2 functions, 0 embeds, 0 globals using 454236virt/87244res/12948shr/73764data kb, in 10usr/0sys/7real ms.
Pass 3: translated to C into "<server>/stap000000/stap_7422abb93e345a17b57111389bdf1c69_1121_src.c" using 454372virt/87244res/12948shr/73900data kb, in 0usr/0sys/0real ms.
Pass 4: compiled C into "stap_7422abb93e345a17b57111389bdf1c69_1121.ko" in 12570usr/2020sys/14743real ms.
Server: No matching machine owner key (MOK) available on the server to sign the
 module.
Server: The server has no machine owner key (MOK) in common with this
system. Use the following command to import a server MOK into this
system, then reboot:

        mokutil --import signing_key.x509
Passes: via server  host=unknown address=127.0.0.1 port=33645 sysinfo="unknown" version=unknown certinfo="unknown" using 384836virt/17476res/15136shr/1740data kb, in 50usr/10sys/16499real ms.
Passes: via server failed.  Try again with another '-v' option.
The kernel on your system requires modules to be signed for loading.
The module created by compiling your script must be signed by a systemtap compile-server.  [man stap-server]
--use-server was automatically selected in order to request compilation by a compile-server.
[root@localhost ~]#
[root@localhost ~]# mokutil --import signing_key.x509
input password: 
input password again: 
[root@localhost ~]# 
[root@localhost ~]# reboot 
Connection to 192.168.0.171 closed by remote host.
Connection to 192.168.0.171 closed.
$ 

---
Reboot; Key successfully enrolled; Reboot
---

[root@localhost ~]# service stap-server start
Redirecting to /bin/systemctl start stap-server.service
[root@localhost ~]# netstat -tlp | grep stap
tcp6       0      0 [::]:32901              [::]:*                  LISTEN      1730/stap-serverd   
[root@localhost ~]# SERVER_PORT=32901
[root@localhost ~]# SERVER_IP=127.0.0.1
[root@localhost ~]# stap --trust-servers=ssl,signer,all-users,no-prompt --use-server=$SERVER_IP:$SERVER_PORT
Adding trust in the following servers as an SSL peer for all users and as a module signer for all users
   host=unknown address=127.0.0.1 port=32901 sysinfo="unknown" version=unknown certinfo="unknown"
[root@localhost ~]# stap --use-server=$SERVER_IP:$SERVER_PORT -v -e 'probe oneshot { log("hey") }'
Using a compile server.
Pass 1: parsed user script and 485 library scripts using 452652virt/85288res/12564shr/72180data kb, in 170usr/40sys/242real ms.
Pass 2: analyzed script: 1 probe, 2 functions, 0 embeds, 0 globals using 454236virt/86996res/12700shr/73764data kb, in 10usr/0sys/7real ms.
Pass 3: using cached <server>/.systemtap/cache/74/stap_7422abb93e345a17b57111389bdf1c69_1121.c
Pass 4: using cached <server>/.systemtap/cache/74/stap_7422abb93e345a17b57111389bdf1c69_1121.ko
Server: Module signed with MOK, fingerprint "3e:12:09:be:ab:7f:6c:de:89:fb:e9:9c:cd:92:84:a4:63:73:b2:a2"
Passes: via server  host=unknown address=127.0.0.1 port=32901 sysinfo="unknown" version=unknown certinfo="unknown" using 384836virt/17548res/15204shr/1740data kb, in 30usr/0sys/552real ms.
The kernel on your system requires modules to be signed for loading.
The module created by compiling your script must be signed by a systemtap compile-server.  [man stap-server]
--use-server was automatically selected in order to request compilation by a compile-server.
Pass 5: starting run.
ERROR: Cannot attach to module stap_7422abb93e345a17b57111389bdf1c69_1121 control channel; not running?
ERROR: Cannot attach to module stap_7422abb93e345a17b57111389bdf1c69_1121 control channel; not running?
ERROR: 'stap_7422abb93e345a17b57111389bdf1c69_1121' is not a zombie systemtap module.
WARNING: /usr/bin/staprun exited with status: 1
Pass 5: run completed in 10usr/10sys/33real ms.
Pass 5: run failed.  [man error::pass5]
[root@localhost ~]#
=======

Comment 2 Frank Ch. Eigler 2020-08-28 16:35:34 UTC
dmesg on the affected machine indicates that the module signing appears to be working correctly, BUT, kernel lockdown is blocking routine staprun/stapio <-> module communication across debugfs:

fs/debugfs/file.c:

/*
 * Only permit access to world-readable files when the kernel is locked down.
 * We also need to exclude any file that has ways to write or alter it as root
 * can bypass the permissions check.
 */
static bool debugfs_is_locked_down(struct inode *inode,
                                   struct file *filp,
                                   const struct file_operations *real_fops)
[...]

Systemtap userspace must be able to write messages to the kernel space.  If debugfs has been changed to prevent us from using it in lockdown mode, even for signed modules, we must find another channel.

cc:'ing rhkernel folks for advice.

Comment 3 Martin Cermak 2020-08-28 17:57:19 UTC
Looks like `mokutil --disable-validation` works the problem around (for my 4.18.0-234.el8.x86_64).  After this setting gets effective (requires reboot), `mokutil --sb-state` still says: SecureBoot enabled.  But this appears to turn off whole that stap's signing mechanism.

Comment 8 Martin Cermak 2020-12-03 13:42:53 UTC
Verified with systemtap-4.4-3.el8.

Comment 10 errata-xmlrpc 2021-05-18 15:44:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (systemtap bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1829


Note You need to log in before you can comment on or make changes to this bug.