Bug 1232595 - Improper use of python's ipaddress
Summary: Improper use of python's ipaddress
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pyspf
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Paul Wouters
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1230373
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-17 06:42 UTC by Bojan Smojver
Modified: 2015-12-11 02:34 UTC (History)
8 users (show)

Fixed In Version: python-pyspf-2.0.11-3.fc23 python-pyspf-2.0.11-3.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of: 1230373
Environment:
Last Closed: 2015-12-02 21:50:22 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Patch for RPM package that fixes the problem (2.71 KB, patch)
2015-11-06 05:56 UTC, Bojan Smojver
no flags Details | Diff

Description Bojan Smojver 2015-06-17 06:42:41 UTC
+++ This bug was initially created as a clone of Bug #1230373 +++

The daemon tries to use either the ipaddr module or the ipaddress module.

...
try:
    import ipaddress
except ImportError:
    import ipaddr as ipaddress
...

However, the ipaddress module codepath contains a bug as reported by user bojan against python-ipaddress: https://admin.fedoraproject.org/updates/FEDORA-2015-8290/python-ipaddress-1.0.7-1.fc21

I believe this is because it presumes that if the ipaddress module is present that the environment is Python 3.x+. This is a bad assumption. The end result is that the daemon breaks when python-ipaddress is installed.

I propose that pypolicyd-spf be fixed and that it depend on either python-ipaddr or python-ipaddress.

--- Additional comment from Bojan Smojver on 2015-06-17 01:53:53 EDT ---

Not as simple as that, it seems. For instance, this package:
----------------
python-pyspf-2.0.11-1.fc22.noarch
----------------

Has the file:
----------------
/usr/lib/python2.7/site-packages/spf.py
----------------

Which does:
----------------
try:
    # Python standard libarary as of python3.3
    import ipaddress
except ImportError:
    try:
        import ipaddr as ipaddress
    except ImportError:
        print('ipaddr module required: http://code.google.com/p/ipaddr-py/')
----------------

So, essentially the same thing as policyd-spf.

Even if I just import ipaddr inside /usr/libexec/postfix/policyd-spf, that's not sufficient. It still throws a whole lot of exceptions because the python-pyspf is also using the new ipaddress thing and fails. The python-spf package needs the same change to make things work.

And I'm not sure whether that's the whole story...

PS. The comment on the updates page is me. :-)

--- Additional comment from Bojan Smojver on 2015-06-17 02:39:51 EDT ---

OK, so something like this maybe:
-------------------------------------------
--- policyd-spf.original	2014-10-01 13:06:12.000000000 +1000
+++ policyd-spf	2015-06-17 16:38:06.869298047 +1000
@@ -38,10 +38,16 @@
 if int(micro) < 7:
     raise ImportError("At least pyspf 2.0.7 is required.")
 import policydspfsupp
-try:
-    import ipaddress
-except ImportError:
-    import ipaddr as ipaddress
+if sys.version_info[0] >= 3:
+    try:
+        import ipaddress
+    except ImportError:
+        print('ipaddress module required: http://code.google.com/p/ipaddr-py/')
+else:
+    try:
+        import ipaddr as ipaddress
+    except ImportError:
+        print('ipaddr module required: http://code.google.com/p/ipaddr-py/')
 try:
     import authres
 except:
-------------------------------------------

And:
-------------------------------------------
--- spf.py.original	2014-12-06 03:20:07.000000000 +1100
+++ spf.py	2015-06-17 16:35:40.192512523 +1000
@@ -98,10 +98,13 @@
     from email.message import Message
 except ImportError:
     from email.Message import Message
-try:
+if sys.version_info[0] >= 3:
     # Python standard libarary as of python3.3
-    import ipaddress
-except ImportError:
+    try:
+        import ipaddress
+    except ImportError:
+        print('ipaddress module required: http://code.google.com/p/ipaddr-py/')
+else:
     try:
         import ipaddr as ipaddress
     except ImportError:
-------------------------------------------

Comment 1 Bojan Smojver 2015-06-18 04:49:53 UTC
According to comments in bug #1230373, we should do somehting like this instead:
-------------------------------------------
--- spf.py.original	2015-06-18 14:46:52.592205731 +1000
+++ spf.py	2014-12-06 03:20:07.000000000 +1100
@@ -401,9 +401,6 @@
             self.iplist = []
             ip6 = True
         else:
-            if sys.version_info.major < 3:
-                if isinstance(i, str):
-                    i = i.decode('ascii')
             try:
                 self.ipaddr = ipaddress.ip_address(i)
             except AttributeError:
@@ -1348,11 +1345,6 @@
         True
         """
         try:
-            if sys.version_info.major < 3:
-                for idx, ip in enumerate(ipaddrs):
-                    if isinstance(ipaddrs[idx], str):
-                        ipaddrs[idx] = ipaddrs[idx].decode('ascii')
-
             for netwrk in [ipaddress.ip_network(ip) for ip in ipaddrs]:
                 network = netwrk.supernet(new_prefix=n)
                 if isinstance(self.iplist, bool):
-------------------------------------------

Grain, salt, apply... :-)

Comment 2 Bojan Smojver 2015-06-18 05:02:03 UTC
(In reply to Bojan Smojver from comment #1)
> According to comments in bug #1230373, we should do somehting like this
> instead:

Plus the removal of import of ipaddr and of the dependency on python-ipddr in the package.

Comment 3 Bojan Smojver 2015-07-15 22:33:38 UTC
Ping... Anyone cares to rebuild this package with the fix?

Comment 4 Paul Wouters 2015-07-28 09:19:24 UTC
can you confirm the scratch build listed works for you:

http://koji.fedoraproject.org/koji/taskinfo?taskID=10508195

I assume you meant the reverse of the patch you posted here?

Comment 5 Bojan Smojver 2015-07-28 11:23:29 UTC
Throwing errors like this:
--------------
Jul 28 21:14:33 beauty policyd-spf[26699]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '81.21.85.197' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
--------------

Tested with both python-ipaddr and python-ipaddress installed (that's the problem situation, I think).

Yeah, I meant the reverse of the patch. But I think there may be a bit missing around the inclusion of python-ipaddr package. Essentially, ipaddr should no longer be included at all.

Like here:

http://pkgs.fedoraproject.org/cgit/pypolicyd-spf.git/tree/pypolicyd-spf-1.3.1-ipaddress.patch

Comment 6 Bojan Smojver 2015-07-28 11:26:35 UTC
(In reply to Bojan Smojver from comment #5)
> Throwing errors like this:
> --------------
> Jul 28 21:14:33 beauty policyd-spf[26699]: ERROR: 127.0.0.0/8 in
> skip_addresses not IP network.  Message: '81.21.85.197' does not appear to
> be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2)
> instead of a unicode object?. Aborting whitelist processing.
> --------------
> 
> Tested with both python-ipaddr and python-ipaddress installed (that's the
> problem situation, I think).

Yeah, never mind this error. This is from my pypolicyd-spf release -2 package on this box.

Let me retest with -3 and with your package. Anyhow, that first bit (inclusion of just ipaddress) still appears to be missing.

Comment 7 Bojan Smojver 2015-07-28 11:31:11 UTC
(In reply to Bojan Smojver from comment #6)
 
> Let me retest with -3 and with your package. Anyhow, that first bit
> (inclusion of just ipaddress) still appears to be missing.

Yeah, that worked.

Anyhow, the idea is that the package should no longer depend on python-ipaddr at all. Once that inclusion bit gets patched, that will then become the case I think.

Comment 8 Bojan Smojver 2015-09-08 02:32:38 UTC
(In reply to Bojan Smojver from comment #7)

> Anyhow, the idea is that the package should no longer depend on
> python-ipaddr at all. Once that inclusion bit gets patched, that will then
> become the case I think.

Could we get this bit fixed as well in an official build? It would be great if we could leave these problems behind in F23.

Comment 9 Bojan Smojver 2015-10-19 04:21:57 UTC
(In reply to Bojan Smojver from comment #8)
> (In reply to Bojan Smojver from comment #7)
> 
> > Anyhow, the idea is that the package should no longer depend on
> > python-ipaddr at all. Once that inclusion bit gets patched, that will then
> > become the case I think.
> 
> Could we get this bit fixed as well in an official build? It would be great
> if we could leave these problems behind in F23.

Ping...

Could we get a properly fixed packages into F22/23, so that I can push fixed pypolicyd-spf as well?

Comment 10 Bojan Smojver 2015-11-06 05:56:10 UTC
Created attachment 1090446 [details]
Patch for RPM package that fixes the problem

Comment 11 Fedora Update System 2015-11-26 21:57:48 UTC
python-pyspf-2.0.11-3.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-05de62603f

Comment 12 Fedora Update System 2015-11-26 21:57:48 UTC
python-pyspf-2.0.11-3.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-1532cc0a3c

Comment 13 Fedora Update System 2015-11-27 22:21:29 UTC
python-pyspf-2.0.11-3.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update python-pyspf'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-1532cc0a3c

Comment 14 Fedora Update System 2015-11-28 02:20:32 UTC
python-pyspf-2.0.11-3.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update python-pyspf'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-05de62603f

Comment 15 Harald Reindl 2015-11-30 10:38:38 UTC
i doubt that this fixes anything because there are errors which did not exist the last year

* Fr Nov 06 2015 Bojan Smojver <bojan> - 2.0.11-3
- Actually resolves: rhbz#1232595 Improper use of python's ipaddress

Nov 29 23:17:11 INFO Upgraded: python-pyspf-2.0.11-3.fc22.noarch

Nov 29 23:17:33 mail-gw policyd-spf[23380]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '209.85.214.182' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:22:43 mail-gw policyd-spf[24414]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '65.54.190.154' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:23:34 mail-gw policyd-spf[24464]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '211.169.178.17' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:27:51 mail-gw policyd-spf[5914]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '93.115.136.88' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:31:34 mail-gw policyd-spf[7876]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '74.125.82.54' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:38:08 mail-gw policyd-spf[8225]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '208.94.241.106' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:38:09 mail-gw policyd-spf[8229]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '212.82.96.82' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:40:06 mail-gw policyd-spf[8331]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '91.224.183.210' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:42:06 mail-gw policyd-spf[8331]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '209.85.160.181' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:42:12 mail-gw policyd-spf[8708]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '124.146.182.116' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:42:28 mail-gw policyd-spf[8723]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '124.146.182.116' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:43:10 mail-gw policyd-spf[8229]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '174.139.99.6' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:43:51 mail-gw policyd-spf[8708]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '209.85.220.49' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:47:08 mail-gw policyd-spf[8919]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '69.89.23.142' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:49:59 mail-gw policyd-spf[9039]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '149.62.168.188' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:50:44 mail-gw policyd-spf[9374]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '188.117.99.82' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 29 23:54:32 mail-gw policyd-spf[9540]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '205.201.135.70' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.

Comment 16 Bojan Smojver 2015-11-30 11:55:31 UTC
Do you still have ipaddr installed or just ipaddress?

Comment 17 Harald Reindl 2015-11-30 12:01:50 UTC
you hardly can remove python-ipaddr

[root@mail-gw:~]$ rpm -qa | grep pyth | grep ipadd
python-ipaddress-1.0.7-3.fc22.noarch
python-ipaddr-2.1.10-2.fc21.noarch

[root@mail-gw:~]$ package-cleanup --leaves --all
Yum-utils package has been deprecated, use dnf instead.
See 'man yum2dnf' for more information.

lounge-spamfirewall-22.0-2.fc22.20151028.rh.noarch
python-ipaddress-1.0.7-3.fc22.noarch

Resolving Dependencies
--> Running transaction check
---> Package python-ipaddr.noarch 0:2.1.10-2.fc21 will be erased
--> Processing Dependency: python-ipaddr for package: pypolicyd-spf-1.3.1-2.fc22.noarch
--> Processing Dependency: python-ipaddr for package: python-pyspf-2.0.11-1.fc22.noarch
--> Running transaction check
---> Package pypolicyd-spf.noarch 0:1.3.1-2.fc22 will be erased
--> Processing Dependency: pypolicyd-spf for package: lounge-spamfirewall-22.0-2.fc22.20151028.rh.noarch
---> Package python-pyspf.noarch 0:2.0.11-1.fc22 will be erased
--> Running transaction check
---> Package lounge-spamfirewall.noarch 0:22.0-2.fc22.20151028.rh will be erased
--> Finished Dependency Resolution
--> Finding unneeded leftover dependencies
Found and removing 0 unneeded dependencies

Comment 18 Bojan Smojver 2015-11-30 12:18:36 UTC
I think the problem may be that we never actually pushed the new build of policyd to F22, only to F23 and above. I will check that tomorrow my time.

Comment 19 Harald Reindl 2015-11-30 12:31:06 UTC
however, it's terrible that as long the pulled "python-ipaddress" is installed even after downgrade "python-pyspf" things are going down

Nov 30 13:28:42 mail-gw policyd-spf[12589]: ERROR: 127.0.0.0/8 in skip_addresses not IP network.  Message: '91.237.33.20' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?. Aborting whitelist processing.
Nov 30 13:28:42 mail-gw policyd-spf[12589]: Traceback (most recent call last):
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/libexec/postfix/policyd-spf", line 680, in <module>
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    instance_dict, configData, peruser)
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/libexec/postfix/policyd-spf", line 517, in _spfcheck
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    res = spf.check2(ip, sender, helo, querytime=configData.get('Lookup_Time'))
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/lib/python2.7/site-packages/spf.py", line 297, in check2
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/lib/python2.7/site-packages/spf.py", line 378, in __init__
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    self.set_ip(i)
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/lib/python2.7/site-packages/spf.py", line 405, in set_ip
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    self.ipaddr = ipaddress.ip_address(i)
Nov 30 13:28:42 mail-gw policyd-spf[12589]:  File "/usr/lib/python2.7/site-packages/ipaddress.py", line 115, in ip_address
Nov 30 13:28:42 mail-gw policyd-spf[12589]:    ' a unicode object?' % address)
Nov 30 13:28:42 mail-gw policyd-spf[12589]: AddressValueError: '91.237.33.20' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?

Comment 20 Bojan Smojver 2015-11-30 20:08:41 UTC
(In reply to Harald Reindl from comment #19)
> however, it's terrible that as long the pulled "python-ipaddress" is
> installed even after downgrade "python-pyspf" things are going down

Yep, that was the original problem.

I am building the F-22 pypolicyd-spf package now and will put it into testing.

Comment 22 Harald Reindl 2015-12-01 10:50:33 UTC
that package combination works

Dec 01 11:47:40 INFO Upgraded: python-pyspf-2.0.11-3.fc22.noarch
Dec 01 11:47:40 INFO Upgraded: pypolicyd-spf-1.3.1-4.fc22.noarch
Dec 01 11:48:07 INFO Erased: python-ipaddr-2.1.10-2.fc21.noarch

Dec  1 11:49:51 mail-gw policyd-spf[21815]: Pass; identity=mailfrom; client-ip=74.125.82.47; helo=mail-wm0-f47.google.com; envelope-from=****@gmail.com; receiver=****@thelounge.net

Comment 23 Bojan Smojver 2015-12-01 23:02:55 UTC
(In reply to Harald Reindl from comment #22)
> that package combination works

Excellent. Thank you very much for testing!

Comment 24 Fedora Update System 2015-12-02 21:50:18 UTC
python-pyspf-2.0.11-3.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2015-12-02 23:50:03 UTC
python-pyspf-2.0.11-3.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Comment 26 Kohei Takahashi 2015-12-04 02:01:37 UTC
After updating 2.0.11-3.fc23, it seems to run SPF check properly for IPv4.
But for IPv6, it is still broken (however trackback is a bit different).


Tracback is here:

Dec 04 10:48:46 **** policyd-spf[3048]: Traceback (most recent call last):
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/libexec/postfix/policyd-spf", line 687, in <module>
Dec 04 10:48:46 **** policyd-spf[3048]:     instance_dict, configData, peruser)
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/libexec/postfix/policyd-spf", line 524, in _spfcheck
Dec 04 10:48:46 **** policyd-spf[3048]:     res = spf.check2(ip, sender, helo, querytime=configData.get('Lookup_Time'))
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 290, in check2
Dec 04 10:48:46 **** policyd-spf[3048]:     receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 543, in check
Dec 04 10:48:46 **** policyd-spf[3048]:     rc = self.check1(spf, self.d, 0)
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 582, in check1
Dec 04 10:48:46 **** policyd-spf[3048]:     return self.check0(spf, recursion)
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 896, in check0
Dec 04 10:48:46 **** policyd-spf[3048]:     if self.cidrmatch(self.dns_mx(arg), cidrlength):
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 1190, in dns_mx
Dec 04 10:48:46 **** policyd-spf[3048]:     return [a for mx in mxnames[:max] for a in self.dns_a(mx[1],self.A)]
Dec 04 10:48:46 **** policyd-spf[3048]:   File "/usr/lib/python2.7/site-packages/spf.py", line 1202, in dns_a
Dec 04 10:48:46 **** policyd-spf[3048]:     return [ipaddress.Bytes(ip) for ip in r]
Dec 04 10:48:46 **** policyd-spf[3048]: AttributeError: 'module' object has no attribute 'Bytes'
Dec 04 10:48:46 **** postfix/spawn[3021]: warning: command /usr/bin/python exit status 1
Dec 04 10:48:46 **** postfix/smtpd[3044]: warning: premature end-of-input on private/policy-spf while reading input attribute name
Dec 04 10:48:46 **** postfix/smtpd[3044]: warning: problem talking to server private/policy-spf: Success


Installed packages:

pypolicyd-spf-1.3.1-3.fc23.noarch
python-ipaddr-2.1.10-3.fc23.noarch
python-ipaddress-1.0.7-4.fc23.noarch
python-pyspf-2.0.11-3.fc23.noarch

Comment 27 Bojan Smojver 2015-12-04 23:39:54 UTC
I am not really a python guy, but it seems that the new package (ipaddress) doesn't implement Bytes attribute. Anyone knows how to fix this without changing ipaddress package?

Comment 28 Bojan Smojver 2015-12-05 01:35:57 UTC
Absolutely no idea whether something like this is what this error is about:

iff -ruN pyspf-2.0.11-v/spf.py pyspf-2.0.11/spf.py
--- pyspf-2.0.11-v/spf.py	2014-12-06 03:20:07.000000000 +1100
+++ pyspf-2.0.11/spf.py	2015-12-05 12:35:19.811673067 +1100
@@ -1203,7 +1203,7 @@
                     'No %s records found for'%A, domainname)
         if A == 'AAAA' and bytes is str:
           # work around pydns inconsistency plus python2 bytes/str ambiguity
-          return [ipaddress.Bytes(ip) for ip in r]
+          return [ipaddress.v6_int_to_packed(ip) for ip in r]
         return r
 
     def validated_ptrs(self):

Comment 29 Anthony Messina 2015-12-05 06:01:08 UTC
(In reply to Bojan Smojver from comment #28)
> Absolutely no idea whether something like this is what this error is about:
> 
> iff -ruN pyspf-2.0.11-v/spf.py pyspf-2.0.11/spf.py
> --- pyspf-2.0.11-v/spf.py	2014-12-06 03:20:07.000000000 +1100
> +++ pyspf-2.0.11/spf.py	2015-12-05 12:35:19.811673067 +1100
> @@ -1203,7 +1203,7 @@
>                      'No %s records found for'%A, domainname)
>          if A == 'AAAA' and bytes is str:
>            # work around pydns inconsistency plus python2 bytes/str ambiguity
> -          return [ipaddress.Bytes(ip) for ip in r]
> +          return [ipaddress.v6_int_to_packed(ip) for ip in r]
>          return r
>  
>      def validated_ptrs(self):

I do wish that would have worked, but alas:

File "/usr/lib/python2.7/site-packages/spf.py", line 1203, in dns_a
  return [ipaddress.v6_int_to_packed(ip) for ip in r]
File "/usr/lib/python2.7/site-packages/ipaddress.py", line 219, in v6_int_to_packed
  raise ValueError("Address negative or too large for IPv6")

Comment 30 Bojan Smojver 2015-12-05 07:41:33 UTC
Yeah, that ip is probably not an integer there. I will have to look a bit closer at all this. Not really my area, python, so it may take a while.

Comment 31 Bojan Smojver 2015-12-08 00:47:18 UTC
Sorry about all the guessing, but I really do not speak python:
----------------------
diff -ruN pyspf-2.0.11-v/spf.py pyspf-2.0.11/spf.py
--- pyspf-2.0.11-v/spf.py	2014-12-06 03:20:07.000000000 +1100
+++ pyspf-2.0.11/spf.py	2015-12-08 11:41:35.025136287 +1100
@@ -1203,7 +1203,7 @@
                     'No %s records found for'%A, domainname)
         if A == 'AAAA' and bytes is str:
           # work around pydns inconsistency plus python2 bytes/str ambiguity
-          return [ipaddress.Bytes(ip) for ip in r]
+          return [Bytes(ip) for ip in r]
         return r
 
     def validated_ptrs(self):
@@ -1617,6 +1617,22 @@
               % (sender, self.c)
         raise ValueError("invalid SPF result for header comment: "+res)
 
+# We need to distinguish between the string and packed-bytes representations
+# of an IP address.  For example, b'0::1' is the IPv4 address 48.58.58.49,
+# while '0::1' is an IPv6 address.
+#
+# In Python 3, the native 'bytes' type already provides this functionality,
+# so we use it directly.  For earlier implementations where bytes is not a
+# distinct type, we create a subclass of str to serve as a tag.
+try:
+    if bytes is str:
+        raise TypeError("bytes is not a distinct type")
+    Bytes = bytes
+except (NameError, TypeError):
+    class Bytes(str):
+        def __repr__(self):
+            return 'Bytes(%s)' % str.__repr__(self)
+
 def split_email(s, h):
     """Given a sender email s and a HELO domain h, create a valid tuple
     (l, d) local-part and domain-part.
----------------------

Does the above do anything?

Comment 32 Bojan Smojver 2015-12-11 02:34:35 UTC
OK, one last try before I give up:
-------------------
diff -ruN pyspf-2.0.11-v/spf.py pyspf-2.0.11/spf.py
--- pyspf-2.0.11-v/spf.py	2014-12-06 03:20:07.000000000 +1100
+++ pyspf-2.0.11/spf.py	2015-12-11 13:31:46.102844734 +1100
@@ -1201,9 +1201,6 @@
         if self.strict > 1 and len(r) == 0:
             raise AmbiguityWarning(
                     'No %s records found for'%A, domainname)
-        if A == 'AAAA' and bytes is str:
-          # work around pydns inconsistency plus python2 bytes/str ambiguity
-          return [ipaddress.Bytes(ip) for ip in r]
         return r
 
     def validated_ptrs(self):
-------------------

Essentially, don't touch what's returned, but just pass it on. Absolutely no idea whether this will screw things up even more.


Note You need to log in before you can comment on or make changes to this bug.