Description of problem: Login sessions created by our in-house monitoring script (they connect to a database using the database instance owner ID) occasionally remain stuck in the "closing" state for days, accumulating thousands of un-closed sessions over time. On one of the machines: $ sudo loginctl | grep -c db2inst1 44551 One of the sessions (from 6 days ago, the script that was called terminated successfully, but the session is still there now): $ loginctl session-status c50825 c50825 - db2inst1 (1121) Since: Fri 2018-02-16 12:23:42 UTC; 6 days ago Leader: 4218 Remote: user root Service: su-l; type unspecified; class background State: closing Unit: session-c50825.scope Feb 16 12:23:42 hostname-1.net systemd[1]: Started Session c50825 of user db2inst1. Feb 16 12:23:42 hostname-1.net systemd[1]: Starting Session c50825 of user db2inst1. Feb 16 12:23:42 hostname-1.net su[4218]: pam_unix(su-l:session): session opened for user db2inst1 by (uid=0) Feb 16 12:23:42 hostname-1.net su[4218]: pam_unix(su-l:session): session closed for user db2inst1 Version-Release number of selected component (if applicable): Linux hostname-1.net 3.10.0-514.21.1.el7.x86_64 #1 SMP Sat Apr 22 02:41:35 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux $ /usr/lib/systemd/systemd --version systemd 219 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN How reproducible: Unsure how to reproduce, it consistently happens on some machines and never happens on others. I'm not sure what to look at trying to understand what the difference might be. Steps to Reproduce: 1. 2. 3. Actual results: Session remains active after the script terminates. Expected results: Session does not show on the active list. Additional info:
What is the exact version of systemd that you are using? (rpm -q systemd)
The one machine I have access to that exhibits this behaviour: $ sudo rpm -q systemd systemd-219-42.el7_4.4.x86_64 I'm asking other folks to check if it's the same on the other servers where I cannot log in myself. It should be the same but who knows.
I've confirmed that all known machines with this behaviour have systemd-219-42.el7_4.4.x86_64.
(In reply to nicki from comment #0) > Login sessions created by our in-house monitoring script (they connect to a > database using the database instance owner ID) occasionally remain stuck in > the "closing" state for days, accumulating thousands of un-closed sessions > over time. Can you describe in more detail how does the script end up running as database user? Does it ssh as root and then switches UID using "su -l"?
The monitoring scripts do not log in remotely -- they are run locally by the cluster mangager software (Tivoli System Automation) and by collecd. Both run as root, and in those scripts they do a `su - <username> -c <something>` to execute certain commands against the database.
The problems seems _somewhat_ related to https://access.redhat.com/solutions/3325431 but the reboot only fixes this temporarily. The system in question was rebooted recently # uptime 19:00:46 up 7 days, 14:15, 1 user, load average: 0.00, 0.02, 0.12 but already accumulated about 29000 "closing" sessions # sudo loginctl | grep -c db2inst1 28956 Normally there would be only one session for db2inst1 at a time.
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.