Bug 1970347

Summary:	Systemtap Error at _stp_transport_data_fs_init:294
Product:	Red Hat Enterprise Linux 8	Reporter:	Martin Cermak <mcermak>
Component:	systemtap	Assignee:	Frank Ch. Eigler <fche>
systemtap sub component:	system-version	QA Contact:	qe-baseos-tools-bugs
Status:	CLOSED CURRENTRELEASE	Docs Contact:
Severity:	unspecified
Priority:	unspecified	CC:	lberk, mcermak, mjw
Version:	8.5	Keywords:	Bugfix, Triaged
Target Milestone:	beta
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-11-17 17:54:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Martin Cermak 2021-06-10 10:30:14 UTC

Using QE testcase /tools/systemtap/Regression/bz711976-debugfs-dir-removal-after-OOM (*) I can pretty reliably reproduce following problem:

* http://pkgs.devel.redhat.com/cgit/tests/systemtap/tree/Regression/bz711976-debugfs-dir-removal-after-OOM/runtest.sh

=======
[ 7277.404115] Systemtap Error at _stp_transport_data_fs_init:294 
[ 7277.404134] Not enough free+buffered memory(1624MB) for log buffer(16380MB)
[ 7282.652979] ------------[ cut here ]------------
[ 7282.750526] proc_dir_entry 'systemtap/bz711976' already registered
[ 7282.752537] WARNING: CPU: 1 PID: 156994 at fs/proc/generic.c:360 proc_register+0x14c/0x1a0
[ 7282.754365] Modules linked in: bz711976(OE+) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nfsd auth_rpcgss nfs_acl lockd grace sunrpc crct10dif_ce ghash_ce sha2_ce sha256_arm64 sha1_ce vfat fat ip_tables xfs libcrc32c virtio_net virtio_blk virtio_console net_failover failover virtio_mmio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: stap_9a40aec8a696de58190864b2b80380_154469]
[ 7282.762042] CPU: 1 PID: 156994 Comm: staprun Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-310.el8.aarch64 #1
[ 7282.764774] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 7282.766418] pstate: 60400005 (nZCv daif +PAN -UAO)
[ 7282.767561] pc : proc_register+0x14c/0x1a0
[ 7282.768548] lr : proc_register+0x14c/0x1a0
[ 7282.769543] sp : ffff00001448fa40
[ 7282.770343] x29: ffff00001448fa40 x28: ffff3b591c129768 
[ 7282.771636] x27: ffff3b58ccbb2318 x26: ffff3b58ccbb2188 
[ 7282.772916] x25: ffffe5474fc8f380 x24: ffffe547418619ab 
[ 7282.774192] x23: ffffe5474fc8f300 x22: ffff3b591d18a080 
[ 7282.775483] x21: ffff3b591db55000 x20: ffffe5474101f888 
[ 7282.776753] x19: ffffe54741861900 x18: 0000000000000010 
[ 7282.778025] x17: 0000000069bd8501 x16: ffff3b591ba1f570 
[ 7282.779299] x15: ffffffffffffffff x14: ffff3b591d0d8788 
[ 7282.780611] x13: ffff00009448f747 x12: ffff00001448f74f 
[ 7282.781875] x11: ffff3b591d114000 x10: ffff00001448f6d0 
[ 7282.783159] x9 : 00000000ffffffd0 x8 : ffff3b591bc7d2a0 
[ 7282.784445] x7 : 000000000000027e x6 : ffffe547ff4d2498 
[ 7282.785734] x5 : ffffe547ff4d2498 x4 : 0000000000000000 
[ 7282.787017] x3 : ffffe547ff55a288 x2 : ffffe547ff4d2498 
[ 7282.788291] x1 : 0000000000000000 x0 : 0000000000000000 
[ 7282.789573] Call trace:
[ 7282.790169]  proc_register+0x14c/0x1a0
[ 7282.791101]  proc_mkdir_data+0x84/0xb0
[ 7282.792006]  proc_mkdir+0x34/0x40
[ 7282.792824]  _stp_mkdir_proc_module.part.11+0x9c/0x1a8 [bz711976]
[ 7282.794288]  init_module+0x4e0/0x5d0 [bz711976]
[ 7282.795392]  do_one_initcall+0x58/0x1ec
[ 7282.796342]  do_init_module+0x64/0x238
[ 7282.797257]  load_module+0x1404/0x16a8
[ 7282.798160]  __se_sys_init_module+0x168/0x280
[ 7282.799214]  __arm64_sys_init_module+0x28/0x38
[ 7282.800298]  el0_svc_handler+0xb4/0x188
[ 7282.801240]  el0_svc+0x8/0xc
[ 7282.801938] ---[ end trace 021abb280f7b183a ]---
[ 7282.803095] Systemtap Error at _stp_mkdir_proc_module:112 
[ 7282.803097] Unable to create '/proc/systemap/bz711976': proc_mkdir failed.
=======

Both base rhel systemtap and gccts systemtap seem to reproduce it:

8.4 Server x86_64 # uname -r; rpm -qa |fgrep systemtap|sort
4.18.0-305.el8.x86_64
gcc-toolset-11-systemtap-4.5-3.el8.x86_64
gcc-toolset-11-systemtap-client-4.5-3.el8.x86_64
gcc-toolset-11-systemtap-devel-4.5-3.el8.x86_64
gcc-toolset-11-systemtap-runtime-4.5-3.el8.x86_64
systemtap-4.5-1.el8.x86_64
systemtap-client-4.5-1.el8.x86_64
systemtap-devel-4.5-1.el8.x86_64
systemtap-runtime-4.5-1.el8.x86_64
8.4 Server x86_64 # 


In case this is a real issue the bug should be cloned for gccts.

Comment 1 Martin Cermak 2022-02-17 10:47:16 UTC

Confirmed when retesting gcc-toolset-11-systemtap-devel-4.5-6.el8 now with 4.18.0-367.el8.

Comment 3 Frank Ch. Eigler 2022-11-17 17:54:25 UTC

A hand-run version of this test passes with systemtap-4.8-1.el8 - no complaints about procfs leftovers after an OOMy run.