Bug 1552745 - WARNING: Missing unwind data for a module.
Summary: WARNING: Missing unwind data for a module.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: systemtap
Version: 27
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Frank Ch. Eigler
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-07 16:27 UTC by Mark Wielaard
Modified: 2018-11-28 14:35 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-28 14:35:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Mark Wielaard 2018-03-07 16:27:39 UTC
$ rpm -q kernel systemtap kernel-debuginfo
kernel-4.14.13-300.fc27.x86_64
kernel-4.16.0-0.rc2.git1.1.fc28.x86_64
systemtap-3.2-3.fc27.x86_64
kernel-debuginfo-4.15.6-300.fc27.x86_64

Even though stap seems to find the unwind data as seen with stap -k -v -v -e 'probe module("nfs").function("nfs_*") { print_backtrace(); exit(); }':

dump_unwindsyms nfs index=0 base=0x10000
Found build-id in nfs, length 20, start at 0x10

Then when a probe hits you get a warning:

WARNING: Missing unwind data for a module, rerun with 'stap -d nfs'
 0xffffffffc0a238d0 [nfs]
 0x0 (inexact)

So, either it is lying earlier, or there is some error reading the unwind tables that isn't properly reported.

Comment 1 Frank Ch. Eigler 2018-03-07 17:22:45 UTC
see also https://sourceware.org/ml/systemtap/2018-q1/msg00008.html

Comment 2 Mark Wielaard 2018-03-08 12:18:01 UTC
(In reply to Frank Ch. Eigler from comment #1)
> see also https://sourceware.org/ml/systemtap/2018-q1/msg00008.html

Maybe related, but it looks very different. There are no modules involved and I don't see that example throwing WARNING: Missing unwind data. It might deserve its own bug.

Comment 3 Mark Wielaard 2018-04-18 22:44:46 UTC
With the systemtap package from this copr repo:
https://copr.fedorainfracloud.org/coprs/mjw/systemtap-fallback-unwind/

If you add --all-modules to the stap invocation you'll get somewhat reasonable looking backtrace, but it will still claim to not have found nfs...

 0xffffffffc08ae4e0 [nfs]
 0xffffffffa728c113 : complete_walk+0x63/0x80 [kernel] (inexact)
 0xffffffffa728e617 : path_lookupat+0xd7/0x1f0 [kernel] (inexact)
 0xffffffffa7291b96 : filename_lookup+0xb6/0x190 [kernel] (inexact)
 0xffffffffa7285d23 : vfs_statx+0x73/0xe0 [kernel] (inexact)
 0xffffffffa7286279 : SYSC_newstat+0x39/0x70 [kernel] (inexact)
 0xffffffffa7003a24 : do_syscall_64+0x74/0x180 [kernel] (inexact)
 0xffffffffa7a00081 : entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [kernel] (inexact)
 0x0 (inexact)
WARNING: Missing unwind data for a module, rerun with 'stap -d nfs'

Comment 4 Mark Wielaard 2018-04-20 11:25:48 UTC
The reason seems to be that on older kernels we had the following in /sys/module/nfs/sections:

$ sudo more /sys/module/nfs/sections/.*text* | cat
::::::::::::::
/sys/module/nfs/sections/.exit.text
::::::::::::::
0xffffffffc0dad42e
::::::::::::::
/sys/module/nfs/sections/.init.text
::::::::::::::
0xffffffffc0dcf000
::::::::::::::
/sys/module/nfs/sections/.text
::::::::::::::
0xffffffffc0d8e000
::::::::::::::
/sys/module/nfs/sections/.text.unlikely
::::::::::::::
0xffffffffc0dad3a7

But on newer kernels we get:

$ sudo more /sys/module/nfs/sections/.*text* | cat
::::::::::::::
/sys/module/nfs/sections/.exit.text
::::::::::::::
0x000000005383cc0f
::::::::::::::
/sys/module/nfs/sections/.init.text
::::::::::::::
0x0000000015dcfa56
::::::::::::::
/sys/module/nfs/sections/.text
::::::::::::::
0x00000000c6200c19
::::::::::::::
/sys/module/nfs/sections/.text..refcount
::::::::::::::
0x00000000b788876d

So those are the values that get reported by staprun relocation messages for the modules. But that isn't really where the module seems to be loaded:

$ sudo grep ^nfs\  /proc/modules 
nfs 303104 3 nfsv3,nfsv4, Live 0xffffffffc08ab000

I have no idea yet how these values are related.

Comment 5 Mark Wielaard 2018-04-20 12:15:14 UTC
The /sys/module/nfs/sections/... addresses are printed with kernel/module.c:

static ssize_t module_sect_show(struct module_attribute *mattr,
                                struct module_kobject *mk, char *buf)
{
        struct module_sect_attr *sattr =
                container_of(mattr, struct module_sect_attr, mattr);
        return sprintf(buf, "0x%pK\n", (void *)sattr->address);
}

Documentation/core-api/printk-formats.rst explains:

        %pK     01234567 or 0123456789abcdef

For printing kernel pointers which should be hidden from unprivileged
users. The behaviour of %pK depends on the kptr_restrict sysctl - see
Documentation/sysctl/kernel.txt for more details.

And Documentation/sysctl/kernel.txt was recently updated to say:

commit da271403a894f1139b3a49fca8fa19585902890e
Author: Tobin C. Harding <me>
Date:   Wed Dec 20 08:17:16 2017 +1100

    doc: update kptr_restrict documentation
    
    Recently the behaviour of printk specifier %pK was changed. The
    documentation does not currently mirror this.
    
    Update documentation for sysctl kptr_restrict.
    
    Signed-off-by: Tobin C. Harding <me>
    Signed-off-by: Jonathan Corbet <corbet>

diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index 63663039..412314e 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -391,7 +391,8 @@ kptr_restrict:
 This toggle indicates whether restrictions are placed on
 exposing kernel addresses via /proc and other interfaces.
 
-When kptr_restrict is set to (0), the default, there are no restrictions.
+When kptr_restrict is set to 0 (the default) the address is hashed before
+printing. (This is the equivalent to %p.)
 
 When kptr_restrict is set to (1), kernel pointers printed using the %pK
 format specifier will be replaced with 0's unless the user has CAP_SYSLOG

Comment 6 Mark Wielaard 2018-04-20 12:19:53 UTC
So, the workaround for now is to do:
echo 1 | sudo tee /proc/sys/kernel/kptr_restrict
Before running a stap script that might need module support.

Comment 7 Frank Ch. Eigler 2018-04-22 18:39:11 UTC
Added commit b3aeaf02fef02ee2c04f482d5a29ac8dd1412c27 to
automatically set this for kernels >= 4.15.

Comment 8 Ben Cotton 2018-11-27 13:30:12 UTC
This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 Mark Wielaard 2018-11-28 14:35:33 UTC
Fixed according to comment #7.


Note You need to log in before you can comment on or make changes to this bug.