Bug 1319880 - python34-3.4.3-4 ships /usr/bin/python3
Summary: python34-3.4.3-4 ships /usr/bin/python3
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: python34
Version: epel7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Petr Viktorin (pviktori)
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-21 18:01 UTC by Troels Arvin
Modified: 2017-01-10 16:15 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-10 16:15:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Troels Arvin 2016-03-21 18:01:54 UTC
Base system: RHEL 7.2 x86_64 with all patches.

After updating from python34-3.4.3-2 to python34-3.4.3-4, a script stopped working:

$ tail -f -n 500000 /var/log/messages | python3.4 /usr/local/bin/syslog-filters/myscript
Traceback (most recent call last):
  File "/usr/local/bin/syslog-filters/myscript", line 612, in <module>
    exclude_pat_compiled = re.compile(exclude_regex_str)
  File "/usr/lib64/python3.4/re.py", line 223, in compile
    return _compile(pattern, flags)
  File "/usr/lib64/python3.4/re.py", line 294, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/lib64/python3.4/sre_compile.py", line 579, in compile
    "sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups

The Python script has a rather long regular expression:
exclusions = [
  " \\.NET_Runtime: 1026: Application: ", # generalized on 2015-05-27
  " ports: .* port .+ is now on-line", # updated 2015-05-27
  " 0056. ports:  port .* PD", # 2015-06-24
  " tftp:.* Transfer completed", # 2015-05-21; generalized on 2015-05-22
(...)
]
exclude_regex_str = '|'.join(exclusions)
exclude_pat_compiled = re.compile(exclude_regex_str)
(...)

Comment 1 Petr Viktorin (pviktori) 2016-03-22 09:32:47 UTC
Are you sure it's caused by the version update? AFAIK Python has had this limit since 2.5.

$ python
Python 2.7.10 (default, Sep  8 2015, 17:20:17) 
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> re.compile('(a)'*100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/re.py", line 194, in compile
    return _compile(pattern, flags)
  File "/usr/lib64/python2.7/re.py", line 249, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/lib64/python2.7/sre_compile.py", line 583, in compile
    "sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups


The examples you give are not enough to reproduce. They don't have any named groups (i.e. sub-regex enclosed in parentheses).

If you don't need named groups, a workaround would be to replace ( by \( and ) by \) in "exclusions".

Comment 2 Troels Arvin 2016-03-22 10:30:20 UTC
I know: I hit the limit with Python 2.7; that's why I installed python34 in the first place.

When I downgrade to python34-3.4.3-2, my script works again.

Comment 3 Petr Viktorin (pviktori) 2016-03-22 11:50:06 UTC
Python 3 is not supposed to remove this limitation. You might be hitting an edge case here but I believe that adding one more named group will make the script stop working even with the old release.


Alternatively, you can install use the "regex" library (python3-regex) instead of "re". It's designed to be backwards-compatible, but uses a reworked engine that removes this limitation.

Comment 4 Robert Kuska 2016-03-22 12:13:30 UTC
(fyi python-3.5 removes this limitation)

Comment 5 Troels Arvin 2016-03-22 13:33:37 UTC
Sorry: My mistake. Some paths-stuff meant that I was actually using a locally (non-RPM) compiled Python 3.5 when my script was working.

The difference is probably that the python34-3.4.3-3 or python34-3.4.3-4 added /bin/python3 which was not present in python34-3.4.3-2. Thus, after the update, my locally compiled Python 3.5 was no longer executed when "python3" was executed.

Comment 6 Orion Poplawski 2016-03-22 15:33:52 UTC
Actually, I think that's perhaps a problem - I'm not sure we were meant to ship /usr/bin/python3 with these packages.

However, the conditional is:

# pybasever without the dot:
%global pyshortver 34

# is this the EPEL 7 main Python 3?
%if "%python3_pkgversion" == "%pyshortver"
%global main_python3 1

I'm surprised this didn't take effect on the original build.  Perhaps the macros weren't in the buildroot at that point?  What is the expected behavior here?

Comment 7 Felix Schwarz 2017-01-10 10:24:59 UTC
I guess the ship sailed by now as python34 in EPEL7 contains /usr/bin/python3 for almost a year and by now for sure there are other users depending on this.

My 2ยข: I think python34 should indeed provide /usr/bin/python3 similar to Python 2.7 providing /usr/bin/python2 so the changes looks good and this bug should be closed. (If you have a self-compiled Python 3 ensure that it does not install anything in /usr/bin/.)


Note You need to log in before you can comment on or make changes to this bug.