Bug 1778380

Summary: crash in policyd-spf blocking incoming email
Product: [Fedora] Fedora Reporter: Brad <brad>
Component: pypolicyd-spfAssignee: Bojan Smojver <bojan>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 30CC: bojan
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-12-11 11:15:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Brad 2019-11-30 13:40:45 UTC
Description of problem:

Postfix is detecting SPF crashes and rejecting mail due to undefined variable in /usr/lib/python3.7/site-packages/spf.py.


Version-Release number of selected component (if applicable):
python3-pyspf-2.0.13-1.fc30.noarch
pypolicyd-spf-2.0.2-4.fc30.noarch


How reproducible:

This was noticed while processing email verification email from linkedin.com.

Actual results:

From syslog:

Nov 27 00:49:51 x policyd-spf[3264]: Traceback (most recent call last):
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/libexec/postfix/policyd-spf", line 809, in <module>
Nov 27 00:49:51 x policyd-spf[3264]:     instance_dict, configData, peruser, peruserconfigData)
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/libexec/postfix/policyd-spf", line 623, in _spfcheck
Nov 27 00:49:51 x policyd-spf[3264]:     mres = mfromquery.check()
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/lib/python3.7/site-packages/spf.py", line 590, in check
Nov 27 00:49:51 x policyd-spf[3264]:     spf = self.dns_spf(self.d)
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1159, in dns_spf
Nov 27 00:49:51 x policyd-spf[3264]:     a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1209, in dns_txt
Nov 27 00:49:51 x policyd-spf[3264]:     dns_list = self.dns(domainname, rr,ignore_void=ignore_void)
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1353, in dns
Nov 27 00:49:51 x policyd-spf[3264]:     for k, v in DNSLookup(name, qtype, self.strict, timeout):
Nov 27 00:49:51 x policyd-spf[3264]:   File "/usr/lib/python3.7/site-packages/spf.py", line 105, in DNSLookup_pydns
Nov 27 00:49:51 x policyd-spf[3264]:     if strict > 1:
Nov 27 00:49:51 x policyd-spf[3264]: NameError: name 'strict' is not defined
Nov 27 00:49:51 x postfix/spawn[3263]: warning: command /usr/libexec/postfix/policyd-spf exit status 1
Nov 27 00:49:51 x postfix/smtpd[3256]: warning: premature end-of-input on private/policyd-spf while reading input attribute name



Expected results:

Success or graceful failure (rather than a python NameError)


Additional info:

Comment 1 Brad 2019-11-30 14:03:36 UTC
Additional info:

Log lines before crash:

Nov 27 00:49:50 x postfix/smtpd[3256]: connect from mx0a-00000703.pphosted.com[148.163.135.169]
Nov 27 00:49:50 x postfix/smtpd[3256]: Anonymous TLS connection established from mx0a-00000703.pphosted.com[148.163.135.169]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)

The proximity to incoming mail from linkedin.com is a red herring.  The offending sender domain is actually kp.org:

# dig kp.org txt | grep spf
kp.org.			3577	IN	TXT	"v=spf1 ip4:162.119.241.162 ip4:162.119.241.85 ip4:162.119.241.86 ip4:162.119.131.61 ip4:162.119.131.55 ip4:162.119.131.57 ip4:162.119.233.57 ip4:162.119.233.49 ip4:162.119.233.50 ip4:148.163.135.169 ip4:148.163.139.169  include:_spf-x1.kp.org include:_spf" "-x3.kp.org ~all"

Notable that the text field is split into two strings.  Is self.dns_spf() choking on that?

Comment 2 Brad 2019-12-05 02:10:06 UTC
In /etc/postfix/main.cf:

smtpd_recipient_restrictions =
  permit_mynetworks,
  reject_rbl_client zen.spamhaus.org,
  check_policy_service unix:private/policyd-spf

In /etc/postfix/master.cf:

policyd-spf  unix  -    n       n       -       0       spawn
  user=nobody argv=/usr/libexec/postfix/policyd-spf

Comment 3 Brad 2019-12-05 02:15:54 UTC
More complete example of failure (mildly redacted), bouncing an email from linkedin.com:

Nov 30 01:30:04 mail.example.com postfix/smtpd[5045]: Anonymous TLS connection established from maila-ba.linkedin.com[108.174.3.137]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]: Traceback (most recent call last):
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/libexec/postfix/policyd-spf", line 809, in <module>
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     instance_dict, configData, peruser, peruserconfigData)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/libexec/postfix/policyd-spf", line 623, in _spfcheck
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     mres = mfromquery.check()
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 595, in check
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     rc = self.check1(spf, self.d, 0)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 634, in check1
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     return self.check0(spf, recursion)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 974, in check0
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     redirect_record = self.dns_spf(redirect)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1159, in dns_spf
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1209, in dns_txt
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     dns_list = self.dns(domainname, rr,ignore_void=ignore_void)
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1353, in dns
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     for k, v in DNSLookup(name, qtype, self.strict, timeout):
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:   File "/usr/lib/python3.7/site-packages/spf.py", line 105, in DNSLookup_pydns
Nov 30 01:30:04 mail.example.com policyd-spf[5717]:     if strict > 1:
Nov 30 01:30:04 mail.example.com policyd-spf[5717]: NameError: name 'strict' is not defined
Nov 30 01:30:04 mail.example.com postfix/spawn[5716]: warning: command /usr/libexec/postfix/policyd-spf exit status 1
Nov 30 01:30:04 mail.example.com postfix/smtpd[5045]: warning: premature end-of-input on private/policyd-spf while reading input attribute name
Nov 30 01:30:05 mail.example.com policyd-spf[5718]: Traceback (most recent call last):
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/libexec/postfix/policyd-spf", line 809, in <module>
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     instance_dict, configData, peruser, peruserconfigData)
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/libexec/postfix/policyd-spf", line 623, in _spfcheck
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     mres = mfromquery.check()
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 595, in check
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     rc = self.check1(spf, self.d, 0)
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 634, in check1
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     return self.check0(spf, recursion)
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 974, in check0
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     redirect_record = self.dns_spf(redirect)
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1159, in dns_spf
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1209, in dns_txt
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     dns_list = self.dns(domainname, rr,ignore_void=ignore_void)
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 1353, in dns
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     for k, v in DNSLookup(name, qtype, self.strict, timeout):
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:   File "/usr/lib/python3.7/site-packages/spf.py", line 105, in DNSLookup_pydns
Nov 30 01:30:05 mail.example.com policyd-spf[5718]:     if strict > 1:
Nov 30 01:30:05 mail.example.com policyd-spf[5718]: NameError: name 'strict' is not defined
Nov 30 01:30:05 mail.example.com postfix/spawn[5716]: warning: command /usr/libexec/postfix/policyd-spf exit status 1
Nov 30 01:30:05 mail.example.com postfix/smtpd[5045]: warning: premature end-of-input on private/policyd-spf while reading input attribute name
Nov 30 01:30:05 mail.example.com postfix/smtpd[5045]: warning: problem talking to server private/policyd-spf: Success
Nov 30 01:30:05 mail.example.com postfix/smtpd[5045]: NOQUEUE: reject: RCPT from maila-ba.linkedin.com[108.174.3.137]: 451 4.3.5 <brad>: Recipient address rejected: Server configuration problem; from=<[redacted]@bounce.linkedin.com> to=<brad> proto=ESMTP helo=<maila-ba.linkedin.com>
Nov 30 01:30:10 mail.example.com postfix/smtpd[5045]: disconnect from maila-ba.linkedin.com[108.174.3.137] ehlo=2 starttls=1 mail=1 rcpt=0/1 quit=1 commands=5/6

Comment 4 Bojan Smojver 2019-12-11 11:15:08 UTC

*** This bug has been marked as a duplicate of bug 1770636 ***