Bug 1573072 - policyd-spf crashes on buggy SPF record entry instead being more relaxed
Summary: policyd-spf crashes on buggy SPF record entry instead being more relaxed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: python-pyspf
Version: epel7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Paul Wouters
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-30 06:06 UTC by Peter Bieringer
Modified: 2020-03-15 13:06 UTC (History)
2 users (show)

Fixed In Version: python-pyspf-2.0.14-3.fc31 python-pyspf-2.0.14-3.fc30 python-pyspf-2.0.14-3.el8 python-pyspf-2.0.14-10.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-01 22:24:38 UTC
Type: Bug


Attachments (Terms of Use)

Description Peter Bieringer 2018-04-30 06:06:58 UTC
Description of problem:
found a buggy SPF record entry of a foreign domain in DNS causing policyd-spf crashing instead of simply ignoring that particular entry

Version-Release number of selected component (if applicable):
pypolicyd-spf-1.3.2-5.el7.noarch


How reproducible:
Always


Steps to Reproduce (demo to reproduce created)
1. create test file:

cat <<END /tmp/spfbuggy.txt 
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=SMTP
helo_name=mail.spfbuggy.demo.bieringer.de
queue_id=1234567890
instance=12345.6789
sender=sender@spfbuggy.demo.bieringer.de
recipient=test@demo.example
client_address=192.0.2.1
client_name=mail.spfbuggy.demo.bieringer.de
END


2. execute manual check

cat /tmp/spfbuggy.txt | /usr/libexec/postfix/policyd-spf ; echo $?

Actual results:

Traceback (most recent call last):
  File "/usr/libexec/postfix/policyd-spf", line 697, in <module>
    instance_dict, configData, peruser)
  File "/usr/libexec/postfix/policyd-spf", line 521, in _spfcheck
    res = spf.check2(ip, sender, helo, querytime=configData.get('Lookup_Time'))
  File "/usr/lib/python2.7/site-packages/spf.py", line 293, in check2
    receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
  File "/usr/lib/python2.7/site-packages/spf.py", line 543, in check
    rc = self.check1(spf, self.d, 0)
  File "/usr/lib/python2.7/site-packages/spf.py", line 582, in check1
    return self.check0(spf, recursion)
  File "/usr/lib/python2.7/site-packages/spf.py", line 902, in check0
    if self.cidrmatch([arg], cidrlength): break
  File "/usr/lib/python2.7/site-packages/spf.py", line 1355, in cidrmatch
    for netwrk in [ipaddress.IPNetwork(ip,strict=False) for ip in ipaddrs]:
  File "/usr/lib/python2.7/site-packages/ipaddr.py", line 118, in IPNetwork
    address)
ValueError: '192.0.2.1\n' does not appear to be an IPv4 or IPv6 network
1


(this caused a lot of entry in maillog, had to whitelist particular client now)


Expected results:

being more relaxed and simply skip but log with level notice or warning the buggy entry 


Additional info:

DNS entry:
spfbuggy.demo.bieringer.de. 86400 IN	TXT	"v=spf1 a mx ip4:192.0.2.1\010 ip4:192.0.2.2 ~all"

(have no clue how this was created, but taken from real example).

btw: replacing "spfbuggy" with "spfok" will point you to a correct SPF record.


https://mxtoolbox.com/SuperTool.aspx tells:

+ 	a 		Pass 	Match if IP has a DNS 'A' record in given domain
+ 	mx 		Pass 	Match if IP is one of the MX hosts for given domain name
+ 	error 	192.0.2.1 	Permerror 	Syntax Error near: 192.0.2.1 Is not a valid value for ip4 mechanism
+ 	ip4 	192.0.2.2 	Pass 	Match if IP is in the given range
~ 	all 		SoftFail 	Always matches. It goes at the end of your record. 



I haven't took a look what RFC tells about broken SPF record entries, but failing totally is not a good option imho....a try-catch block around particular record check would be the better one.

Comment 1 Peter Bieringer 2018-04-30 06:27:59 UTC
Participating files in the stack trace:

rpm -qf /usr/lib/python2.7/site-packages/ipaddr.py
python-ipaddr-2.1.11-1.el7.noarch

rpm -qf /usr/lib/python2.7/site-packages/spf.py
python-pyspf-2.0.11-5.el7.noarch

rpm -qf /usr/libexec/postfix/policyd-spf
pypolicyd-spf-1.3.2-5.el7.noarch

Comment 2 Fedora Update System 2020-02-15 10:55:06 UTC
FEDORA-EPEL-2020-8d987efa26 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8d987efa26

Comment 3 Peter Bieringer 2020-02-15 17:41:25 UTC
can't be reproduced anymore, also https://mxtoolbox.com/SuperTool.aspx is no longer claiming about...

Comment 4 Peter Bieringer 2020-02-15 21:36:03 UTC
able to reproduce it again, the piping is no longer working, one has to enter the SPF contents afterwards...

/usr/libexec/postfix/policyd-spf 
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=SMTP
helo_name=mail.spfbuggy.demo.bieringer.de
queue_id=1234567890
instance=12345.6789
sender=sender@spfbuggy.demo.bieringer.de
recipient=test@demo.example
client_address=192.0.2.1
client_name=mail.spfbuggy.demo.bieringer.de

Traceback (most recent call last):
  File "/usr/libexec/postfix/policyd-spf", line 697, in <module>
    instance_dict, configData, peruser)
  File "/usr/libexec/postfix/policyd-spf", line 521, in _spfcheck
    res = spf.check2(ip, sender, helo, querytime=configData.get('Lookup_Time'))
  File "/usr/lib/python2.7/site-packages/spf.py", line 293, in check2
    receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
  File "/usr/lib/python2.7/site-packages/spf.py", line 543, in check
    rc = self.check1(spf, self.d, 0)
  File "/usr/lib/python2.7/site-packages/spf.py", line 582, in check1
    return self.check0(spf, recursion)
  File "/usr/lib/python2.7/site-packages/spf.py", line 902, in check0
    if self.cidrmatch([arg], cidrlength): break
  File "/usr/lib/python2.7/site-packages/spf.py", line 1355, in cidrmatch
    for netwrk in [ipaddress.IPNetwork(ip,strict=False) for ip in ipaddrs]:
  File "/usr/lib/python2.7/site-packages/ipaddr.py", line 118, in IPNetwork
    address)
ValueError: '192.0.2.1\n' does not appear to be an IPv4 or IPv6 network

Comment 5 Fedora Update System 2020-02-17 03:43:13 UTC
FEDORA-EPEL-2020-8d987efa26 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8d987efa26

Comment 6 Peter Bieringer 2020-02-17 06:38:58 UTC
Here more easy to execute tests:

# /usr/bin/spfquery --ip 192.0.2.1 --sender sender@spfbuggy.demo.bieringer.de --helo mail.spfbuggy.demo.bieringer.de --rcpt-to=test@demo.example
Traceback (most recent call last):
  File "/usr/bin/spfquery", line 136, in <module>
    main(sys.argv[1:])
  File "/usr/bin/spfquery", line 120, in main
    format(q)
  File "/usr/bin/spfquery", line 97, in format
    res,code,txt = q.check()
  File "/usr/lib/python2.7/site-packages/spf.py", line 543, in check
    rc = self.check1(spf, self.d, 0)
  File "/usr/lib/python2.7/site-packages/spf.py", line 582, in check1
    return self.check0(spf, recursion)
  File "/usr/lib/python2.7/site-packages/spf.py", line 902, in check0
    if self.cidrmatch([arg], cidrlength): break
  File "/usr/lib/python2.7/site-packages/spf.py", line 1355, in cidrmatch
    for netwrk in [ipaddress.IPNetwork(ip,strict=False) for ip in ipaddrs]:
  File "/usr/lib/python2.7/site-packages/ipaddr.py", line 118, in IPNetwork
    address)
ValueError: '192.0.2.1\n' does not appear to be an IPv4 or IPv6 network


# /usr/bin/spfquery --ip 192.0.2.1 --sender sender@spfok.demo.bieringer.de --helo mail.spfok.demo.bieringer.de --rcpt-to=test@demo.example
pass
()
('spfquery:', 'domain of spfok.demo.bieringer.de designates 192.0.2.1 as permitted sender')
('Received-SPF:', 'Pass (spfquery: domain of spfok.demo.bieringer.de designates 192.0.2.1 as permitted sender) client-ip=192.0.2.1; envelope-from="sender@spfok.demo.bieringer.de"; helo=mail.spfok.demo.bieringer.de; receiver=spfquery; mechanism="ip4:192.0.2.1"; identity=mailfrom')

Comment 7 Fedora Update System 2020-02-18 09:45:15 UTC
FEDORA-EPEL-2020-8d987efa26 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8d987efa26

Comment 8 Peter Bieringer 2020-02-18 20:48:59 UTC
Following patch would help also for this semi-buggy SPF entries found

@@ -587,6 +587,7 @@
             if not spf:
                 spf = self.dns_spf(self.d)
                 if self.verbose: self.log("top",self.d,spf)
+	    spf = spf.replace('\n','')
             if self.libspf_local and spf: 
                 spf = insert_libspf_local_policy(
                     spf, self.libspf_local)

Comment 9 Bojan Smojver 2020-02-18 21:12:48 UTC
OK, I'll spin up -7 when I get some time and include your patch. Once again, thank you for testing!

Comment 10 Fedora Update System 2020-02-19 02:59:12 UTC
python-pyspf-2.0.14-6.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8d987efa26

Comment 11 Bojan Smojver 2020-02-19 10:41:58 UTC
(In reply to Peter Bieringer from comment #8)
> Following patch would help also for this semi-buggy SPF entries found
> 
> @@ -587,6 +587,7 @@
>              if not spf:
>                  spf = self.dns_spf(self.d)
>                  if self.verbose: self.log("top",self.d,spf)
> +	    spf = spf.replace('\n','')
>              if self.libspf_local and spf: 
>                  spf = insert_libspf_local_policy(
>                      spf, self.libspf_local)

Just spent some time reading SPF RFC and I am not sure whether we are going to be breaking SPF format if this is done. I know a lot of people probably mis-configured their TXT/SPF records, but I'm worried we may introduce a non-standard behaviour with this.

Here is a question, does the package as is work with well formed TXT/SPF records?

Comment 12 Peter Bieringer 2020-02-19 21:23:51 UTC
sorry, the patch shown above was the first one I've tried, correct one should replace newline by a space (and not simply removing it)


@@ -587,6 +587,7 @@
             if not spf:
                 spf = self.dns_spf(self.d)
                 if self.verbose: self.log("top",self.d,spf)
+	    spf = spf.replace('\n',' ')
             if self.libspf_local and spf: 
                 spf = insert_libspf_local_policy(
                     spf, self.libspf_local)

Regarding testing of a broken SPF record placed in spfbuggy.demo.bieringer.de

https://mxtoolbox.com/SuperTool.aspx -> OK

https://www.dmarcanalyzer.com/de/spf-de/checker/ -> OK

https://www.spf-record.de/spf-lookup -> OK

=> all 3 are not claiming about the newline in the record


https://www.kitterman.com/spf/validate.html

=> reporting an issue, but using this particular library...so no wonder

Comment 13 Bojan Smojver 2020-02-20 21:30:23 UTC
OK, thanks for providing additional info. I guess the idea is to relax a bit about newline mid-record and just turn it into a space, so that incorrectly coded SPF/TXT records can still pass validation. I know this is not 100% RFC compliant, but it seems to be practical and I cannot see a downside to it. I'll integrate your patch into the next builds.

Comment 14 Fedora Update System 2020-02-20 23:38:51 UTC
FEDORA-2020-9e8c8dedb3 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-9e8c8dedb3

Comment 15 Fedora Update System 2020-02-20 23:38:52 UTC
FEDORA-EPEL-2020-08c57e33f8 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 16 Fedora Update System 2020-02-20 23:38:53 UTC
FEDORA-EPEL-2020-582b05fe3c has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-582b05fe3c

Comment 17 Fedora Update System 2020-02-20 23:38:55 UTC
FEDORA-2020-3fa8f469a0 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-3fa8f469a0

Comment 18 Fedora Update System 2020-02-21 01:10:32 UTC
python-pyspf-2.0.14-2.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3fa8f469a0

Comment 19 Fedora Update System 2020-02-21 01:33:03 UTC
python-pyspf-2.0.14-2.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9e8c8dedb3

Comment 20 Fedora Update System 2020-02-21 01:55:11 UTC
python-pyspf-2.0.14-2.el8 has been pushed to the Fedora EPEL 8 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-582b05fe3c

Comment 21 Fedora Update System 2020-02-21 01:56:00 UTC
python-pyspf-2.0.14-7.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 22 Fedora Update System 2020-02-21 09:04:45 UTC
FEDORA-EPEL-2020-08c57e33f8 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 23 Fedora Update System 2020-02-22 01:37:45 UTC
python-pyspf-2.0.14-8.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 24 Fedora Update System 2020-02-22 08:10:01 UTC
FEDORA-2020-9e8c8dedb3 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-9e8c8dedb3

Comment 25 Fedora Update System 2020-02-22 08:16:02 UTC
FEDORA-EPEL-2020-08c57e33f8 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 26 Fedora Update System 2020-02-23 01:35:04 UTC
python-pyspf-2.0.14-9.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 27 Fedora Update System 2020-02-23 01:35:17 UTC
python-pyspf-2.0.14-3.el8 has been pushed to the Fedora EPEL 8 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-582b05fe3c

Comment 28 Fedora Update System 2020-02-23 01:56:26 UTC
python-pyspf-2.0.14-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3fa8f469a0

Comment 29 Fedora Update System 2020-02-23 02:17:39 UTC
python-pyspf-2.0.14-3.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9e8c8dedb3

Comment 30 Fedora Update System 2020-02-23 07:46:14 UTC
FEDORA-EPEL-2020-08c57e33f8 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 31 Fedora Update System 2020-02-24 01:48:34 UTC
python-pyspf-2.0.14-10.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-08c57e33f8

Comment 32 Fedora Update System 2020-03-01 22:24:38 UTC
python-pyspf-2.0.14-3.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 33 Fedora Update System 2020-03-01 23:19:38 UTC
python-pyspf-2.0.14-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 34 Fedora Update System 2020-03-09 02:05:41 UTC
python-pyspf-2.0.14-3.el8 has been pushed to the Fedora EPEL 8 stable repository. If problems still persist, please make note of it in this bug report.

Comment 35 Fedora Update System 2020-03-15 13:06:55 UTC
python-pyspf-2.0.14-10.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.