Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2096536

Summary: stap --remote libvirt:// does not work as expected
Product: Red Hat Enterprise Linux 9 Reporter: XinhuaLi <xili>
Component: systemtapAssignee: Frank Ch. Eigler <fche>
systemtap sub component: system-version QA Contact: qe-baseos-tools-bugs
Status: CLOSED MIGRATED Docs Contact:
Severity: unspecified    
Priority: unspecified CC: lberk, mcermak, mjw, scox, wcohen
Version: 9.0Keywords: MigratedToJIRA, Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-22 16:00:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description XinhuaLi 2022-06-14 07:41:54 UTC
Description of problem:
Execute from HOST 
Works well if we use ssh connection 
# stap --remote-prefix --remote root.100.30 -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
Pass 1: parsed user script and 496 library scripts using 387992virt/148384res/16740shr/131468data kb, in 430usr/50sys/484real ms.
Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392216virt/153116res/17004shr/135692data kb, in 30usr/0sys/32real ms.
Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
Pass 5: starting run.
0: Hello from VM3-RHEL9-DEV! <<== WE CAN SEE HELLO 
^CPass 5: run completed in 20usr/0sys/3359real ms.


Can not work is we use libvirt
 # stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
Pass 5: starting run.
<<== NOTHING HAPPEN WE CAN NOT SEE HELLO 

Version-Release number of selected component (if applicable):

HOST INFO 
systemtap-runtime-virthost-4.6-12.el9_0.x86_64
systemtap-sdt-devel-4.6-12.el9_0.x86_64
systemtap-runtime-4.6-12.el9_0.x86_64
systemtap-client-4.6-12.el9_0.x86_64
systemtap-devel-4.6-12.el9_0.x86_64
systemtap-4.6-12.el9_0.x86_64
systemtap-server-4.6-12.el9_0.x86_64

libvirt-libs-8.0.0-8.el9_0.x86_64
libvirt-glib-4.0.0-3.el9.x86_64
libvirt-dbus-1.4.1-5.el9.x86_64
libvirt-client-8.0.0-8.el9_0.x86_64
python3-libvirt-8.0.0-1.el9.x86_64
libvirt-daemon-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-core-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-network-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-nwfilter-8.0.0-8.el9_0.x86_64
libvirt-daemon-config-nwfilter-8.0.0-8.el9_0.x86_64
libvirt-daemon-config-network-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-disk-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-mpath-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-scsi-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-iscsi-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-rbd-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-logical-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-storage-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-interface-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-qemu-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-secret-8.0.0-8.el9_0.x86_64
libvirt-daemon-driver-nodedev-8.0.0-8.el9_0.x86_64
libvirt-8.0.0-8.el9_0.x86_64


VM3-RHEL9-DEV INFO 

systemtap-runtime-4.6-12.el9_0.x86_64
systemtap-client-4.6-12.el9_0.x86_64
systemtap-devel-4.6-12.el9_0.x86_64
systemtap-4.6-12.el9_0.x86_64
systemtap-server-4.6-12.el9_0.x86_64
systemtap-runtime-virtguest-4.6-12.el9_0.x86_64
systemtap-initscript-4.6-12.el9_0.x86_64
systemtap-runtime-virthost-4.6-12.el9_0.x86_64
systemtap-debugsource-4.6-12.el9_0.x86_64
systemtap-debuginfo-4.6-12.el9_0.x86_64
systemtap-runtime-debuginfo-4.6-12.el9_0.x86_64

How reproducible:
Execute command 
Can not work is we use libvirt
 # stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
Pass 5: starting run.
<<== NOTHING HAPPEN WE CAN NOT SEE HELLO 

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
It should work if use libvirt connection and we should see the output 

Additional info:
Execute from HOST 
Works well if we use ssh connection 
# stap --remote-prefix --remote root.100.30 -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
Pass 1: parsed user script and 496 library scripts using 387992virt/148384res/16740shr/131468data kb, in 430usr/50sys/484real ms.
Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392216virt/153116res/17004shr/135692data kb, in 30usr/0sys/32real ms.
Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
Pass 5: starting run.
0: Hello from VM3-RHEL9-DEV! <<== WE CAN SEE HELLO 
^CPass 5: run completed in 20usr/0sys/3359real ms.


Can not work is we use libvirt
 # stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
Pass 5: starting run.
<<== NOTHING HAPPEN WE CAN NOT SEE HELLO 

stapvirt info 
# stapvirt query VM3-RHEL9-DEV

              Name:  VM3-RHEL9-DEV
              UUID:  a719c10c-0b9b-4e3d-bfa8-af5bee183e44
             State:  running
                ID:  63
              Type:  persistent
   Permanent Ports:  2
       Hotplugging:  supported

# stapvirt port-list VM3-RHEL9-DEV
/var/lib/libvirt/qemu/VM3-RHEL9-DEV.org.systemtap.stapsh.0.sock
/var/lib/libvirt/qemu/VM3-RHEL9-DEV.org.systemtap.stapsh.1.sock

On VM3-RHEL9-DEV
# systemctl status stapsh.stapsh.0.service 
● stapsh.stapsh.0.service - SystemTap stapsh on org.systemtap.stapsh.0
     Loaded: loaded (/usr/lib/systemd/system/stapsh@.service; static)
     Active: active (running) since Tue 2022-06-14 14:41:10 CST; 52min ago
       Docs: man:stap
             man:stapsh
   Main PID: 73714 (stapsh)
      Tasks: 1 (limit: 20701)
     Memory: 176.0K
        CPU: 30ms
     CGroup: /system.slice/system-stapsh.slice/stapsh.stapsh.0.service
             └─73714 /usr/bin/stapsh -l /dev/virtio-ports/org.systemtap.stapsh.0

Jun 14 14:41:10 RHEL9DEV systemd[1]: Started SystemTap stapsh on org.systemtap.stapsh.0.

#lspci |grep console
09:00.0 Communication controller: Red Hat, Inc. Virtio console (rev 01)

Comment 1 Martin Cermak 2022-06-20 11:18:43 UTC
This bug is filed against gcc-toolset-11. For GTS, this functionality was dropped per bz1965277.  So for GTS, this is not expected to work.

However, I've tested this using systemtap-4.7-1.el8 and systemtap-4.7-2.el9 (in the host role) and in both cases I've reproduced the problem.

Comment 2 Stan Cox 2022-07-08 19:21:29 UTC
stap command pipes to a stapvirt command that is the libvirt interface.  The remote is running a stapsh tool that receives the requests.  The initial commands are sent and received okay (strace output on remote) butthe stap kernel module is not received.  stapvirt seems has a quite similar layout to virsh-console.  It sets up callbacks for stdin, stdout, stream. 

All is fine for short packets: 
1. Send file command to remote, stdin_event gets event
stdin_event:1000 fd 0 events 0x1 stbuf_off 0 stbuf 0x7ffd0e99f848 termbuf_off 58 buf "file 926928 stap_1f18b4b"
2. stream_event gets event
stream_event:1067 events 0x2 stbuf_off 0 stbuf 0x7ffd0e99f848 buf "file 926928 stap_1f18b4b"

but stream_event is not triggered when kernel module is being sent in chunks

3. stdin_event reads first chunk of kernel module
   (which does virStreamEventUpdateCallback when it notices buffer contents)
stdin_event:1000 fd 0 events 0x1 stbuf_off 0 stbuf 0x7ffd0e99f848 termbuf_off 1024 buf "^?ELF^B^A^A"
4. but stream_event is never triggered 

The libvirt event loop is using the glib event loop mechanism.  Somehow the stream_event is not getting invoked.

Comment 3 RHEL Program Management 2023-09-22 15:55:18 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 4 RHEL Program Management 2023-09-22 16:00:21 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.