Red Hat Bugzilla – Bug 248782
pam_abl seems to do nothing, PAM breakage?
Last modified: 2010-01-11 09:55:59 EST
Description of problem:
When installing pam_abl at a Fedora Rawhide system, it looks like pam_abl
seems to do absolutely nothing - especially when doing many failed logins. The
abl database directory in /var/lib/abl is still empty. The correct behaviour
would be to create the hosts.db and users.db files and maybe to log something
to syslog. Don't know, whether it's of belong, but working stopped for me on
April 14th, 2007. I'm guessing it's a PAM breakage somewhere, but I'm not sure.
As Alexander, the current pam_abl maintainer isn't a C freak at all and
upstream is dead (looking for a new maintainer), I'm going to add Tomas Mraz,
Nalin Dahyabhai, Dan Walsh and Steve Grubb as Cc to this bugreport, because
these made changes to PAM in the past, so they've maybe an idea how to figger
out, what is broken and how to fix it. Are you able to support us?
Version-Release number of selected component (if applicable):
Everytime, see above.
pam_abl seems to do nothing.
Working pam_abl like before this breakage.
Oh and I would say, my configuration is correct at all. Finally the same
configuration worked in the past for a long long time:
$ cat /etc/pam.d/system-auth
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth required pam_env.so
auth required pam_abl.so config=/etc/security/pam_abl.conf
auth sufficient pam_unix.so try_first_pass nullok
auth required pam_deny.so
account required pam_unix.so
password required pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so try_first_pass use_authtok nullok md5
password required pam_deny.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond
session required pam_unix.so
PAM library did not change in any way it could affect pam_abl. But it might be a
SELinux permissions problem.
SELinux is not enforced, just permissive (and nothing pam_abl relevant in
audit logs). Do you know the last PAM change that could have affected pam_abl?
Any hints for debugging?
I don't know of any change that could affect pam_abl. In fact the PAM library is
strictly backwards compatible so it should never break for older modules.
Have you tried if it works over console login?
Simply creates no /var/lib/abl/*.db anyway over console login. I'll try to
install tomorrow at RHEL5 to see whether it works there. Maybe this gives a
point to dig into.
Just did it now before sleeping. At RHEL5 the stuff works as expected, just
two "ssh robert@localhost" tries and entering about 6 times the wrong password:
-rw------- 1 root root 8192 18. Jul 23:47 /var/lib/abl/hosts.db
-rw------- 1 root root 8192 18. Jul 23:47 /var/lib/abl/users.db
I'll compare the configurations and versions etc. tomorrow (really tomorrow).
selinux policy allows all domains that use nsswitch (login porgrams) full access
to files in /var/lib/abl directory.
Looks like selinux policy to use this app is broken in Rawhide and probably FC7.
But I don't see how this library would ever work. It can check the database if
it exists, but I don't see how it can update it on failed login attempts. Who
is responsible for populating the database? I think that pam_abl will update
the database if a user/host on the black list attempts to login again, but I
don't see it adding new hosts/users to the black list.
There is another tool "faillog" that does something similar outside of the pam
stack. It monitors the /var/log/secure or /var/log/messages for failed logins
and then updates the iptables to remove the offending hosts.
Created attachment 160207 [details]
patch to get pam_abl working again
The attached patch seems to solve the problem for me.
The problem seems to be that the cleanup function is never called on an
unsuccessful login attempt. The cleanup function in pam_abl is the one that
records a login failure, but if it isn't called, it can't. So in this patch I
work around that by first always recording a failure, and then if the attempt
was correct after all (i.e. when pam_sm_setcred is called) removing that
attempt again from the database.
Note that there is a potential race condition in this fix (if two attempts
happen simultaneously, one successful and one not, the wrong one may be
The patch is against the F7 source.
Created attachment 160265 [details]
I got a crash of my X server (or rather the gdb greeter) when I tried to login
using the previous patch. Here is an improved version. Only a single line
changed, making sure that the size of the data that is to be stored into the
database is greater than 0.
Sjoerd, just being curious: How were you able to reproduce this and how did
you find this bug report? The problem doesn't exist since yesterday - yes, but
nobody else found this before, as it seems.
Well and are you able to tell me what did change, that stuff broke?
Do you think, it's possible to have a patch without race conditions? During
SSH login attacks, race conditions are worse. Finally, I didn't try your patch
yet, but I already appreciate your work :) I'll try to look into this evening.
Sjoerd, still alive? Can you have a look to comment #11? Tomas Mraz or so, can
you maybe also have a look to it and suggest how to remove the race condition
to get things working a saner way?
Somehow I missed the arrival of comment #11.
I noticed that pam_abl wasn't logging, so I started searching. That's how I
found this bug report which described my symptoms.
I have no idea why in F7 pam_abl didn't work. Some change in the pam layer, I
presume, but I didn't look at that.
The race condition is actually not very serious. If there is a login attack,
all logins are failures (presumably), so they get logged at the start of the try
as a failure and never get erased. Only if, while such an attack is going on,
somebody tries to login using the account being attacked, there is a potential race.
The problem in solving the race condition is that you need to remember the
details of the entry written at the beginning until the cleanup phase, but the
cleanup phase may not come. So you can't use malloc to get some memory since
you may not be able to free it. And a static buffer may not be a good idea
because of concurrency issues (I don't know whether the code needs to be thread
safe, so I assume it should be).
I have been running with this patch for a month now on all my (four) systems. I
now regularly see that attacks are being thwarted by pam_abl.
Sjoerd...since adding your patch, I've some things in my syslog:
Sep 3 14:42:05 tux pam_abl: Cleaning up, err is 20000000
Sep 3 14:42:32 tux pam_abl: Cleaning up, err is 00000000
Sep 3 14:42:33 tux pam_abl: Cleaning up, err is 20000000
Sep 3 14:42:33 tux pam_abl: Cleaning up, err is 00000000
Sep 3 14:42:34 tux pam_abl: Cleaning up, err is 20000000
Oh and your patch seems to be buggy anyway - regarding the possible race
condition. After the first *successful* login for a new user and/or host,
even one failed login per user and per host is written down. This is IMHO
wrong and caused by your patch. I can't remember such a behaviour on my
RHEL4/5 systems where I'm running pam_abl, too.
Sjoerd, the main problem IMHO of your patch is, that you're treating now one
SSH login sequence with multiple password prompts as several failed logins
which isn't the case. So if the password is entered 4 times wrong during one
login session, 4 failures are logged. On RHEL one failure per login session
(independent of the count of password prompts) is logged.
Could you elaborate on comment #14? When you login, and you use the correct
passwd at the first attempt, the sequence is:
pam_sm_authenticate() is called which calls record_attempt() to record a failed
pam_sm_setcred() is called which calls record_attempt() to erase the record.
The net effect is that an empty record is added to the database (both for the
host and the user). Is that a problem?
It is true that pam_sm_authenticate() is called once for each login attempt,
whether failed or successful, and pam_sm_setcred() is only called once for the
final successful attempt. So if you try multiple passwords in one attempt, they
will get recorded. Even if this is different from the original, I don't really
see that as a problem.
As to fixing the race condition properly, I'm getting more and more convinced
that it is not really a problem. There isn't actually much data being recorded
that identifies the login attempt. Basically only the time stamp. Remember
that the race only occurs when the same user name or host name is involved in
the failed and successful attempts, and they are used as keys in the database.
The record contains a list of time stamps, and the race causes the wrong time
stamp to be erased.
As to the log message. The call to log_debug that causes the message to appear
was introduced by my patch. It may well be that I added that call to help me
figure out what was happening and forgot to remove it. The messages certainly
didn't appear in the unpatched version since the call to log_debug wasn't there.
That doesn't mean that the call to cleanup didn't happen (nor that it did).
After an upgrade from FC6 to F7 I have the exact same problem.
pam_abl is not working on F7! I've removed the db-file (forced to create an new
one)...also removed the pam_abl package and reinstalled again.
theres no way to get pam_abl working!
Regarding to this Bug-Report at pam_abl the problem is located to openssh...it
was corrected on openssh 4.7 but Fedora 7 uses 4.5....any possibility to include
this openssh patch (which causes this problem in cooperation with pam) in Fedora ??
Probably you also want to check this Bug report:
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
This bug can be closed with the note to "currentrelease 8" because since fedora8
this problem is not pending any more...
I can't see a valid fix for that?
As noted above, the problem does only rely to openssh version 4.5...but since
fedora8 it is at least 4.7. and on a fedora8 system you were able to use pam_abl
see here on one of my fedora8 boxes:
[root@ ]# rpm -q openssh
[root@ ]# uname -r
[root@ ]# pam_abl
Blocking users [*]
just install again pam_abl as you have before and pam_abl is working again...
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.
More information and reason for this action is here:
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '10'.
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 10's end of life.
Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 10 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora please change the 'version' of this
bug to the applicable version. If you are unable to change the version,
please add a comment here and someone will do it for you.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
The process we are following is described here:
Fedora 10 changed to end-of-life (EOL) status on 2009-12-17. Fedora 10 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.
If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.
Thank you for reporting this bug and we are sorry it could not be fixed.
This is still a bug in Fedora 12. I'm using pam_abl-0.2.3-8.fc12.x86_64.
pam_abl appears to do nothing and does not generate its database file.
This is probably a different bug as it worked on older Fedoras apparently. Please open a new bug report. Please also try testing with SELinux in permissive mode.
Could be a labeling problem on /var/log/abl
restorecon -R -v /var/log/abl
Check /var/log/audit/audit.log for any avc messages.
Thanks - I have SE linux turned off, so that's not the problem. I was never able to get it working under F10, so I assumed this was the same problem as above.
I found I had opened a bug for Fedora 10 (#524847), but nothing happened there.