I don't have an explanation, but I upgraded to krb5-libs-1.21-2.fc38.x86_64 this morning and sendmail relay authentication stopped working, both for Outlook and iOS Mail app. I just downgraded back to krb5-libs-1.20.1-8.fc38.x86_64 and now sendmail relay authentication is working again. Sample message from maillog with authentication broken: Jul 11 21:37:20 vin sendmail[920681]: STARTTLS=server, relay=[192.168.0.221], version=TLSv1.3, verify=NOT, cipher=TLS_AES_128_GCM_SHA256, bits=128/128 Jul 11 21:37:21 vin sendmail[920681]: 36C5bK97920681: AUTH failure (PLAIN): authentication failure (-13) SASL(-13): authentication failure: Password verification failed, user=toby, relay=[192.168.0.221] After downgrade (and restarting sendmail for good measure): Jul 11 21:40:51 vin sendmail[921095]: STARTTLS=server, relay=visage.ovod-everett.org [192.168.0.2], version=TLSv1.2, verify=NOT, cipher=ECDHE-ECDSA-AES256-GCM-SHA384, bits=256/256 Jul 11 21:40:52 vin sendmail[921095]: AUTH=server, relay=visage.ovod-everett.org [192.168.0.2], authid=toby, mech=LOGIN, bits=0 Also potentially of note, here are some entries from the messages log around the same times: Failure (updated krb5-libs): messages:Jul 11 21:37:20 vin audit[1165]: USER_AUTH pid=1165 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:authentication grantors=? acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=failed' messages:Jul 11 21:37:21 vin saslauthd[1165]: : auth failure: [user=toby] [service=smtp] [realm=] [mech=pam] [reason=PAM auth error] Similar (but success instead of failure messages) from after downgrading: messages:Jul 11 21:40:51 vin audit[1166]: USER_AUTH pid=1166 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:authentication grantors=pam_usertype,pam_localuser,pam_unix acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=success' messages:Jul 11 21:40:52 vin audit[1166]: USER_ACCT pid=1166 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=success' Reproducible: Always Steps to Reproduce: 1. Install and configure sendmail authentication for relay (ideally with TLS) 2. Upgrade krb5-libs to 1.21-2.fc38 3. Trying using sendmail server to relay from a mail client Actual Results: Authentication failures Expected Results: Authentication successes and mail is successfully sent out
Hello Toby, This regression is a bit surprising, given the fact you don't seem to be using GSSAPI as an authentication method. I see in the maillog that the relay destination is different, and so are the TLS versions and ciphers. Could you check this request using both krb5-libs versions to the same SMTP server, please? I would like to confirm this is not related to SMTP server properties.
Hi Julien, TL;DR: It appears I have to reboot the server after installing the krb5-libs upgrade to resolve the issue with mail clients authenticating to sendmail. Perhaps there is an unidentified service that needs to restart - restarting sendmail wasn't sufficient, and I checked my logs from last night to confirm that was the first thing I tried. Lengthier message from before I tried rebooting the server: Sorry for the confusion - this is when Outlook or iOS Mail are connecting to sendmail via port 587 TLS to send an outbound email. This isn't an issue with sendmail attempting to connect to my ISP's relay server - that uses authinfo and an authinfo.db file. This is solely an issue with email clients connecting to sendmail on my Fedora machine for sending messages. I have the following configured in my sendnail.mc file regarding my sendmail authentication requirements for connecting machines: define(`confAUTH_OPTIONS', `A p')dnl TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl DAEMON_OPTIONS(`Port=submission, Name=MSA, M=Ea')dnl DAEMON_OPTIONS(`Port=smtps, Name=TLSMTA, M=s')dnl You are correct that the source for the failure was different - in the first one I mistakenly grabbed a record created when my iPhone attempted to send a message. I'll go ahead and toggle krb5-libs back and forth just to make sure I'm capturing apples to apples (both from Outlook attempting to send a message). Failure in maillog w/krb5-libs-1.21-2.fc38.x86_64: Jul 12 05:31:54 vin sendmail[975240]: STARTTLS=server, relay=visage.ovod-everett.org [192.168.0.2], version=TLSv1.2, verify=NOT, cipher=ECDHE-ECDSA-AES256-GCM-SHA384, bits=256/256 Jul 12 05:31:56 vin sendmail[975240]: 36CDVsvr975240: AUTH failure (LOGIN): authentication failure (-13) SASL(-13): authentication failure: checkpass failed, user=toby, relay=visage.ovod-everett.org [192.168.0.2] Jul 12 05:31:56 vin sendmail[975240]: 36CDVsvr975240: visage.ovod-everett.org [192.168.0.2] did not issue MAIL/EXPN/VRFY/ETRN during connection to MSA Failure in messages w/krb5-libs-1.21-2.fc38.x86_64: Jul 12 05:31:54 vin audit[1163]: USER_AUTH pid=1163 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:authentication grantors=? acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=failed' Jul 12 05:31:56 vin saslauthd[1163]: : auth failure: [user=toby] [service=smtp] [realm=] [mech=pam] [reason=PAM auth error] Success in maillog w/ krb5-libs-1.20.1-8.fc38.x86_64: Jul 12 05:32:22 vin sendmail[975315]: STARTTLS=server, relay=visage.ovod-everett.org [192.168.0.2], version=TLSv1.2, verify=NOT, cipher=ECDHE-ECDSA-AES256-GCM-SHA384, bits=256/256 Jul 12 05:32:22 vin sendmail[975315]: AUTH=server, relay=visage.ovod-everett.org [192.168.0.2], authid=toby, mech=LOGIN, bits=0 Jul 12 05:32:23 vin sendmail[975315]: 36CDWMEC975315: from=<toby>, size=2360, class=0, nrcpts=1, msgid=<002f01d9b4c5$49bb50f0$dd31f2d0$@ovod-everett.org>, proto=ESMTPSA, daemon=MSA, relay=visage.ovod-everett.org [192.168.0.2] Success in messages w/ krb5-libs-1.20.1-8.fc38.x86_64 (I'm including the denied messages that don't seem to be negatively impacting anything): Jul 12 05:32:22 vin audit[1162]: USER_AUTH pid=1162 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:authentication grantors=pam_usertype,pam_localuser,pam_unix acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=success' Jul 12 05:32:22 vin audit[1162]: AVC avc: denied { dac_read_search } for pid=1162 comm="saslauthd" capability=2 scontext=system_u:system_r:saslauthd_t:s0 tcontext=system_u:system_r:saslauthd_t:s0 tclass=capability permissive=0 Jul 12 05:32:22 vin audit[1162]: AVC avc: denied { dac_override } for pid=1162 comm="saslauthd" capability=1 scontext=system_u:system_r:saslauthd_t:s0 tcontext=system_u:system_r:saslauthd_t:s0 tclass=capability permissive=0 Jul 12 05:32:22 vin audit[1162]: USER_ACCT pid=1162 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:saslauthd_t:s0 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="toby" exe="/usr/sbin/saslauthd" hostname=? addr=? terminal=? res=success' I didn't bounce sendmail this time. Just to rule it out as a possibility, I'm going to upgrade again and reboot the server completely. OK - that fixed it! I did bounce sendmail last night when I was troubleshooting the issue, so whatever the issue is with the upgrade, simply restarting sendmail (I used systemctl restart sendmail) is insufficient. Perhaps there's another daemon somewhere that needs to restart when krb5-libs upgrades. Sorry for all the work, although perhaps it's useful to know that a reboot might be needed when krb5-libs upgrades (this isn't the sort of thing I normally expect on a Linux server ;-), so I don't always attempt the initial Windows troubleshooting step). Now I'll go reboot for the kernel upgrade and do a few more tests just to be paranoid. I'll leave the Status unchanged in case you want to leave it open for investigating which service might need to restart as part of the krb5-libs upgrade. Thanks, Toby
Hello Tody, The issue you faced is very similar to the one described in bug 2222804. This missing k5_buf_cstring() function was added in krb5 1.21. Apparently libgssapi_krb5.so.2 was loaded with an outdated version of libkrb5support.so, which should not happen in practice. This is quite surprising, especially given the fact both these shared object files are part of the same RPM. I will keep this BZ open, and I may try to investigate it further later.