Red Hat Bugzilla – Bug 248782
pam_abl seems to do nothing, PAM breakage?
Last modified: 2010-01-11 09:55:59 EST
Description of problem: When installing pam_abl at a Fedora Rawhide system, it looks like pam_abl seems to do absolutely nothing - especially when doing many failed logins. The abl database directory in /var/lib/abl is still empty. The correct behaviour would be to create the hosts.db and users.db files and maybe to log something to syslog. Don't know, whether it's of belong, but working stopped for me on April 14th, 2007. I'm guessing it's a PAM breakage somewhere, but I'm not sure. As Alexander, the current pam_abl maintainer isn't a C freak at all and upstream is dead (looking for a new maintainer), I'm going to add Tomas Mraz, Nalin Dahyabhai, Dan Walsh and Steve Grubb as Cc to this bugreport, because these made changes to PAM in the past, so they've maybe an idea how to figger out, what is broken and how to fix it. Are you able to support us? Version-Release number of selected component (if applicable): pam_abl-0.2.3-3 How reproducible: Everytime, see above. Actual results: pam_abl seems to do nothing. Expected results: Working pam_abl like before this breakage.
Oh and I would say, my configuration is correct at all. Finally the same configuration worked in the past for a long long time: $ cat /etc/pam.d/system-auth #%PAM-1.0 # This file is auto-generated. # User changes will be destroyed the next time authconfig is run. auth required pam_env.so auth required pam_abl.so config=/etc/security/pam_abl.conf auth sufficient pam_unix.so try_first_pass nullok auth required pam_deny.so account required pam_unix.so password required pam_cracklib.so try_first_pass retry=3 password sufficient pam_unix.so try_first_pass use_authtok nullok md5 shadow password required pam_deny.so session optional pam_keyinit.so revoke session required pam_limits.so session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid session required pam_unix.so $
PAM library did not change in any way it could affect pam_abl. But it might be a SELinux permissions problem.
SELinux is not enforced, just permissive (and nothing pam_abl relevant in audit logs). Do you know the last PAM change that could have affected pam_abl? Any hints for debugging?
I don't know of any change that could affect pam_abl. In fact the PAM library is strictly backwards compatible so it should never break for older modules. Have you tried if it works over console login?
Simply creates no /var/lib/abl/*.db anyway over console login. I'll try to install tomorrow at RHEL5 to see whether it works there. Maybe this gives a point to dig into.
Just did it now before sleeping. At RHEL5 the stuff works as expected, just two "ssh robert@localhost" tries and entering about 6 times the wrong password: -rw------- 1 root root 8192 18. Jul 23:47 /var/lib/abl/hosts.db -rw------- 1 root root 8192 18. Jul 23:47 /var/lib/abl/users.db I'll compare the configurations and versions etc. tomorrow (really tomorrow).
selinux policy allows all domains that use nsswitch (login porgrams) full access to files in /var/lib/abl directory.
Looks like selinux policy to use this app is broken in Rawhide and probably FC7. But I don't see how this library would ever work. It can check the database if it exists, but I don't see how it can update it on failed login attempts. Who is responsible for populating the database? I think that pam_abl will update the database if a user/host on the black list attempts to login again, but I don't see it adding new hosts/users to the black list. There is another tool "faillog" that does something similar outside of the pam stack. It monitors the /var/log/secure or /var/log/messages for failed logins and then updates the iptables to remove the offending hosts.
Created attachment 160207 [details] patch to get pam_abl working again The attached patch seems to solve the problem for me. The problem seems to be that the cleanup function is never called on an unsuccessful login attempt. The cleanup function in pam_abl is the one that records a login failure, but if it isn't called, it can't. So in this patch I work around that by first always recording a failure, and then if the attempt was correct after all (i.e. when pam_sm_setcred is called) removing that attempt again from the database. Note that there is a potential race condition in this fix (if two attempts happen simultaneously, one successful and one not, the wrong one may be deleted. The patch is against the F7 source.
Created attachment 160265 [details] improved fix I got a crash of my X server (or rather the gdb greeter) when I tried to login using the previous patch. Here is an improved version. Only a single line changed, making sure that the size of the data that is to be stored into the database is greater than 0.
Sjoerd, just being curious: How were you able to reproduce this and how did you find this bug report? The problem doesn't exist since yesterday - yes, but nobody else found this before, as it seems. Well and are you able to tell me what did change, that stuff broke? Do you think, it's possible to have a patch without race conditions? During SSH login attacks, race conditions are worse. Finally, I didn't try your patch yet, but I already appreciate your work :) I'll try to look into this evening.
Sjoerd, still alive? Can you have a look to comment #11? Tomas Mraz or so, can you maybe also have a look to it and suggest how to remove the race condition to get things working a saner way?
Somehow I missed the arrival of comment #11. I noticed that pam_abl wasn't logging, so I started searching. That's how I found this bug report which described my symptoms. I have no idea why in F7 pam_abl didn't work. Some change in the pam layer, I presume, but I didn't look at that. The race condition is actually not very serious. If there is a login attack, all logins are failures (presumably), so they get logged at the start of the try as a failure and never get erased. Only if, while such an attack is going on, somebody tries to login using the account being attacked, there is a potential race. The problem in solving the race condition is that you need to remember the details of the entry written at the beginning until the cleanup phase, but the cleanup phase may not come. So you can't use malloc to get some memory since you may not be able to free it. And a static buffer may not be a good idea because of concurrency issues (I don't know whether the code needs to be thread safe, so I assume it should be). I have been running with this patch for a month now on all my (four) systems. I now regularly see that attacks are being thwarted by pam_abl.
Sjoerd...since adding your patch, I've some things in my syslog: Sep 3 14:42:05 tux pam_abl[18548]: Cleaning up, err is 20000000 Sep 3 14:42:32 tux pam_abl[18643]: Cleaning up, err is 00000000 Sep 3 14:42:33 tux pam_abl[18640]: Cleaning up, err is 20000000 Sep 3 14:42:33 tux pam_abl[18640]: Cleaning up, err is 00000000 Sep 3 14:42:34 tux pam_abl[18511]: Cleaning up, err is 20000000 Oh and your patch seems to be buggy anyway - regarding the possible race condition. After the first *successful* login for a new user and/or host, even one failed login per user and per host is written down. This is IMHO wrong and caused by your patch. I can't remember such a behaviour on my RHEL4/5 systems where I'm running pam_abl, too.
Sjoerd, the main problem IMHO of your patch is, that you're treating now one SSH login sequence with multiple password prompts as several failed logins which isn't the case. So if the password is entered 4 times wrong during one login session, 4 failures are logged. On RHEL one failure per login session (independent of the count of password prompts) is logged.
Could you elaborate on comment #14? When you login, and you use the correct passwd at the first attempt, the sequence is: pam_sm_authenticate() is called which calls record_attempt() to record a failed login; then pam_sm_setcred() is called which calls record_attempt() to erase the record. The net effect is that an empty record is added to the database (both for the host and the user). Is that a problem? It is true that pam_sm_authenticate() is called once for each login attempt, whether failed or successful, and pam_sm_setcred() is only called once for the final successful attempt. So if you try multiple passwords in one attempt, they will get recorded. Even if this is different from the original, I don't really see that as a problem. As to fixing the race condition properly, I'm getting more and more convinced that it is not really a problem. There isn't actually much data being recorded that identifies the login attempt. Basically only the time stamp. Remember that the race only occurs when the same user name or host name is involved in the failed and successful attempts, and they are used as keys in the database. The record contains a list of time stamps, and the race causes the wrong time stamp to be erased. As to the log message. The call to log_debug that causes the message to appear was introduced by my patch. It may well be that I added that call to help me figure out what was happening and forgot to remove it. The messages certainly didn't appear in the unpatched version since the call to log_debug wasn't there. That doesn't mean that the call to cleanup didn't happen (nor that it did).
After an upgrade from FC6 to F7 I have the exact same problem. pam_abl is not working on F7! I've removed the db-file (forced to create an new one)...also removed the pam_abl package and reinstalled again. theres no way to get pam_abl working!
https://sourceforge.net/tracker/index.php?func=detail&aid=1795382&group_id=148927&atid=773100 Regarding to this Bug-Report at pam_abl the problem is located to openssh...it was corrected on openssh 4.7 but Fedora 7 uses 4.5....any possibility to include this openssh patch (which causes this problem in cooperation with pam) in Fedora ??
Probably you also want to check this Bug report: https://bugzilla.mindrot.org/show_bug.cgi?id=1322
Changing version to '9' as part of upcoming Fedora 9 GA. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
This bug can be closed with the note to "currentrelease 8" because since fedora8 this problem is not pending any more...
I can't see a valid fix for that?
As noted above, the problem does only rely to openssh version 4.5...but since fedora8 it is at least 4.7. and on a fedora8 system you were able to use pam_abl again.... see here on one of my fedora8 boxes: [root@ ]# rpm -q openssh openssh-4.7p1-4.fc8 [root@ ]# uname -r 2.6.21.7-3.fc8xen [root@ ]# pam_abl Failed hosts: 194.106.175.102 (6) Blocking users [*] 200-160-96-54-ma.static.vivax.com.br (7) Not blocking 211.100.30.222 (7) Not blocking 59.188.10.20 (8) Not blocking 61.131.81.192 (3) Not blocking 61.185.220.249 (5) Not blocking hs44.order-vault.net (6) Not blocking ns202004.ovh.net (6) Not blocking static-ip-58-91-134-202.rev.dyxnet.com (7) Not blocking just install again pam_abl as you have before and pam_abl is working again...
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle. Changing version to '10'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
This message is a reminder that Fedora 10 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 10. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '10'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 10's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 10 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 10 changed to end-of-life (EOL) status on 2009-12-17. Fedora 10 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.
This is still a bug in Fedora 12. I'm using pam_abl-0.2.3-8.fc12.x86_64. pam_abl appears to do nothing and does not generate its database file.
This is probably a different bug as it worked on older Fedoras apparently. Please open a new bug report. Please also try testing with SELinux in permissive mode.
Could be a labeling problem on /var/log/abl restorecon -R -v /var/log/abl Check /var/log/audit/audit.log for any avc messages.
Thanks - I have SE linux turned off, so that's not the problem. I was never able to get it working under F10, so I assumed this was the same problem as above. I found I had opened a bug for Fedora 10 (#524847), but nothing happened there.