Description of problem: When the sss_ssh_knownhostsproxy is enabled on OpenSSH, it causes high latency with X11 forwarding. Version-Release number of selected component (if applicable): libssh2-1.9.0-5.el8.x86_64 Tue Oct 4 11:43:45 2022 libssh2-devel-1.9.0-5.el8.x86_64 Tue Oct 4 11:43:46 2022 libssh-0.9.6-3.el8.x86_64 Tue Oct 4 11:39:11 2022 libssh-host4ig-0.9.6-3.el8.noarch Tue Oct 4 11:39:06 2022 libssh-devel-0.9.6-3.el8.x86_64 Tue Oct 4 11:40:24 2022 libxkbcommon-x11-0.9.1-1.el8.x86_64 Tue Oct 4 11:38:45 2022 openssh-8.0p1-13.el8.x86_64 Tue Oct 4 11:39:25 2022 openssh-askpass-8.0p1-13.el8.x86_64 Tue Oct 4 11:43:24 2022 openssh-clients-8.0p1-13.el8.x86_64 Tue Oct 4 11:40:19 2022 openssh-server-8.0p1-13.el8.x86_64 Tue Oct 4 11:40:19 2022 python3-sssdhost4ig-2.6.2-4.el86.1.noarch Tue Oct 4 11:44:50 2022 sssd-client-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:39:38 2022 sssd-common-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:39:44 2022 sssd-common-pac-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:44:51 2022 sssd-dbus-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:44:49 2022 sssd-ipa-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:44:51 2022 sssd-kcm-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:40:00 2022 sssd-krb5-common-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:44:50 2022 sssd-nfs-host56ap-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:39:44 2022 sssd-tools-2.6.2-4.el8_6.1.x86_64 Tue Oct 4 11:44:51 2022 x11vnc-0.9.16-3.el8.x86_64 Tue Oct 4 11:43:46 2022 xorg-x11-apps-7.7-21.el8.x86_64 Tue Oct 4 11:39:31 2022 xorg-x11-drv-fbdev-0.5.0-2.el8.x86_64 Tue Oct 4 11:43:20 2022 xorg-x11-drv-libinput-0.29.0-1.el8.x86_64 Tue Oct 4 11:43:20 2022 xorg-x11-drv-vesa-2.4.0-3.el8.x86_64 Tue Oct 4 11:43:20 2022 xorg-x11-fonts-75dpi-7.5-19.el8.noarch Tue Oct 4 11:40:26 2022 xorg-x11-fonts-100dpi-7.5-19.el8.noarch Tue Oct 4 11:40:25 2022 xorg-x11-fonts-ISO8859-1-75dpi-7.5-19.el8.noarch Tue Oct 4 11:38:52 2022 xorg-x11-fonts-ISO8859-1-100dpi-7.5-19.el8.noarch Tue Oct 4 11:38:52 2022 xorg-x11-fonts-Type1-7.5-19.el8.noarch Tue Oct 4 11:43:08 2022 xorg-x11-fonts-misc-7.5-19.el8.noarch Tue Oct 4 11:39:23 2022 xorg-x11-font-utils-7.5-41.el8.x86_64 Tue Oct 4 11:38:51 2022 xorg-x11-proto-devel-2020.1-3.el8.noarch Tue Oct 4 11:38:51 2022 xorg-x11-server-Xorg-1.20.11-5.el8_6.2.x86_64 Tue Oct 4 11:43:21 2022 xorg-x11-server-Xvfb-1.20.11-5.el8_6.2.x86_64 Tue Oct 4 11:43:45 2022 xorg-x11-server-common-1.20.11-5.el8_6.2.x86_64 Tue Oct 4 11:43:20 2022 xorg-x11-server-utils-7.7-27.el8.x86_64 Tue Oct 4 11:39:07 2022 xorg-x11-utils-7.5-28.el8.x86_64 Tue Oct 4 11:39:07 2022 xorg-x11-xauth-1.0.9-12.el8.x86_64 Tue Oct 4 11:39:07 2022 xorg-x11-xbitmaps-1.1.1-13.el8.noarch Tue Oct 4 11:38:51 2022 xorg-x11-xinit-1.3.4-18.el8.x86_64 Tue Oct 4 11:39:44 2022 xorg-x11-xkb-utils-7.7-28.el8.x86_64 Tue Oct 4 11:43:19 2022 How reproducible: I've had trouble reproducing, but the customer has been able to successfully on multiple machines. Steps to Reproduce: 1. Make sure glxgears is installed 2. $ ssh -X -o 'ProxyCommand=/usr/bin/sss_ssh_knownhostsproxy -p %p %h' user@machine glxgears Actual results: 45 frames in 5.0 seconds = 8.978 FPS 71 frames in 5.0 seconds = 14.158 FPS 64 frames in 5.0 seconds = 12.783 FPS 75 frames in 5.1 seconds = 14.679 FPS Expected results: 705 frames in 5.0 seconds = 140.936 FPS 649 frames in 5.0 seconds = 129.643 FPS 701 frames in 5.0 seconds = 140.121 FPS 717 frames in 5.0 seconds = 143.343 FPS Additional info: Case c#13 has a detailed example from customer on how to reproduce it.
Probably performance of `sss_ssh_knownhostsproxy :: proxy_data()` could be improved using something like `sendfile()` (or other zero-copy technique (MSG_ZEROCOPY?)) But not sure if it's worth the effort taking into account `sss_ssh_knownhostsproxy` is deprecated and will be removed / needs to be replaced by `KnownHostsCommand` - see https://github.com/SSSD/sssd/issues/5518
(In reply to Alexey Tikhonov from comment #1) > > But not sure if it's worth the effort taking into account > `sss_ssh_knownhostsproxy` is deprecated A note: I didn't mean "oficially deprecated here". I merely meant "have to be replaced by `KnownHostsCommand`".
Well, an attempt to improve performance of SSSD proxy helper - https://github.com/SSSD/sssd/pull/6757 - didn't help a real user. No further attempt will be made to improve it. Instead team will focus on implementing support of new ssh configuration option - `KnownHostsCommand`, that should be much more promising area to put effort into (see https://github.com/SSSD/sssd/issues/5518 )
The behaviour of sss_ssh_knownhostsproxy is: 1. Query IdM server and get the host public key 2. Store the publickey to /var/lib/sss/pubconf/known_hosts So if 1 is slow, patching one SSSD side probably will not help much. Let time how long does it require to get the public key ~~~ time ssh_ssh_knownhostproxy -k -p $PORT $HOST_NAME ~~~
(In reply to Ding-Yi Chen from comment #11) > The behaviour of sss_ssh_knownhostsproxy is: > > 1. Query IdM server and get the host public key > 2. Store the publickey to /var/lib/sss/pubconf/known_hosts > > So if 1 is slow, patching one SSSD side probably will not help much. What is slow is: 3. proxy data: https://github.com/SSSD/sssd/blob/34ef9c5f3e90d5c50c7ac5161c39daa2840c92f2/src/sss_client/ssh/sss_ssh_knownhostsproxy.c#L77