Created attachment 1260884 [details] Patch to update epel7 package to v0.5.6. Description of problem: python-idstools 0.5.4 currently in EPEL has a bug causing some code paths to crash on Python 3.4. As the tools (ie: /usr/bin/idstools-rulecat) are only installed with the python34 variant of the package, some of these tools (like idstools-rulecat) are unusable. 0.5.6 fixes this issue. How reproducible: Reproducible for anyone trying to run idstools-rulecat and has Suricata installed. Steps to Reproduce: 1. Install suricata (yum install suricata). 2. Install idstools (yum install python34-idstools) 3. Run idstools-rulecat (idstools-rulecat). Actual results: [root@9d3d1f76ead0 ~]# idstools-rulecat Traceback (most recent call last): File "/usr/bin/idstools-rulecat", line 12, in <module> sys.exit(main()) File "/usr/lib/python3.4/site-packages/idstools/scripts/rulecat.py", line 725, in main args.url.append(resolve_etopen_url(args.suricata)) File "/usr/lib/python3.4/site-packages/idstools/scripts/rulecat.py", line 642, in resolve_etopen_url suricata_version = idstools.suricata.get_version(suricata_path) File "/usr/lib/python3.4/site-packages/idstools/suricata.py", line 61, in get_version m = re.search("version ((\d+)\.(\d+)\.?(\d+|\w+)?)", output.strip()) File "/usr/lib64/python3.4/re.py", line 170, in search return _compile(pattern, flags).search(string) TypeError: can't use a string pattern on a bytes-like object Expected results: A successful run of idstools-rulecat. Additional info: Also, its a little frustrating that "python-idstools" resolves to "python2-idstools" which is the package that doesn't actually provide the tools, just the library.
Hi, that's the result of my reading of https://fedoraproject.org/wiki/Packaging:Python "For other executables, the general rule is: If only one executable is to be shipped, then it owns its own slot and should use /usr/bin/python3 from Fedora 22 on." I decided to ship only python3 executables (scripts), but python2 can be run with python2 `which idstools-rulecat` If I read this policy correctly this may be a way of pushing python3 instead of python2. See also this thread https://lists.fedoraproject.org/pipermail/packaging/2014-December/010360.html The packaging rules have become complicated, especially if one wants to package for both Fedora and EPEL using the same spec file.
Discussion here https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org/thread/54G6DJFFFCEFFZNX4KCERUPZQOYXFRAR/
Fair enough. So from that I see that the executables need to go in the Python 3 package. But the more generic "python-<packagename>" resolves to the system Python version which is currently version 2 across the board... Sigh, thats a bit of UX issue I suppose. Oh well, I'll be happy when its updated so the executable works with Python 3. Thanks for the explanation.
I don't reproduce your error after `yum install -y python34-idstools`: $ strace idstools-u2json 2>&1 | grep 'bin/python' readlink("/usr/bin/python3.4", 0x7ffde8806060, 4096) = -1 EINVAL (Invalid argument) lstat("/usr/bin/python3.4", {st_mode=S_IFREG|0755, st_size=11312, ...}) = 0 read(3, "#!/usr/bin/python3.4 -s\n#\n# Laun"..., 214) = 214 read(3, "#!/usr/bin/python3.4 -s\n#\n# Laun"..., 4096) = 236 read(3, "#!/usr/bin/python3.4 -s\n#\n# Laun"..., 4096) = 236 $ python --version Python 2.7.5 idstools scripts come with python3.4 shebang so they should be run with python3 by default: $ grep python `which idstools-rulecat` #!/usr/bin/python3.4 -s Can you verify this? About the release - should I wait until 0.5.7?
Did you install Suricata from epel? The error occurs when Suricata is on the path, as idstools-rulecat will automatically try and parse the version. idstools-u2json is unaffected. I have no immediate plans for a 0.5.7.
I think we need 0.5.7. It seems that the problem is related to re in python3: # python3 Python 3.4.5 (default, Nov 9 2016, 16:24:59) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> import re >>> output = subprocess.check_output(["suricata", "-V"]) >>> re.search("version ((\d+)\.(\d+)\.?(\d+|\w+)?)", output.strip()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.4/re.py", line 170, in search return _compile(pattern, flags).search(string) TypeError: can't use a string pattern on a bytes-like object >>> re.search(b"version ((\d+)\.(\d+)\.?(\d+|\w+)?)", output.strip()) <_sre.SRE_Match object; span=(17, 30), match=b'version 3.2.1'> >>> # python2 Python 2.7.5 (default, Nov 6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> import re >>> output = subprocess.check_output(["suricata", "-V"]) >>> re.search(b"version ((\d+)\.(\d+)\.?(\d+|\w+)?)", output.strip()) <_sre.SRE_Match object at 0x7f52b7089b58>
That doesn't look like 0.5.6, the regex in 0.5.6 is: m = re.search("((\d+)\.(\d+)(\.(\d+))?(\w+)?)", str(buf).strip()) https://github.com/jasonish/py-idstools/blob/0.5.6/idstools/suricata.py#L51
OK, I see - the 0.5.6 actually fixes the re problem.
python-idstools-0.5.6-5.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-68959e2eb0
python-idstools-0.5.6-5.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-15810df738
python-idstools-0.5.6-5.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-56837cefd5
python-idstools-0.5.6-5.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-0e9bb68871
python-idstools-0.5.6-5.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-56837cefd5
python-idstools-0.5.6-5.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-15810df738
python-idstools-0.5.6-5.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-0e9bb68871
python-idstools-0.5.6-5.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-68959e2eb0
python-idstools-0.5.6-5.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
python-idstools-0.5.6-5.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
python-idstools-0.5.6-5.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
python-idstools-0.5.6-5.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.