Bug 672043 - abrt.pth breaks rpmbuild of python
Summary: abrt.pth breaks rpmbuild of python
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: abrt
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jiri Moskovcak
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 676661 (view as bug list)
Depends On:
Blocks: 707950
TreeView+ depends on / blocked
 
Reported: 2011-01-23 13:21 UTC by Paul Bolle
Modified: 2015-02-01 22:53 UTC (History)
11 users (show)

Fixed In Version: abrt-2.0.3-1.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 672483 (view as bug list)
Environment:
Last Closed: 2011-06-24 03:24:22 UTC


Attachments (Terms of Use)

Description Paul Bolle 2011-01-23 13:21:27 UTC
Description of problem:
abrt.pth breaks rpmbuild of python

Version-Release number of selected component (if applicable):
1.1.14-3

How reproducible:
Always

Steps to Reproduce:
1. rpmbuild -bb $SPECS/python.spec
2.
3.
  
Actual results:
[...]
gcc -pthread -shared   Modules/xxsubtype.o   -o Modules/xxsubtype_d.so
Traceback (most recent call last):
  File "/home/[...]/Rpmbuild/BUILD/Python-2.7.1/Lib/site.py", line 553, in <module>
    main()
  File "/home/[...]/Rpmbuild/BUILD/Python-2.7.1/Lib/site.py", line 536, in main
    known_paths = addsitepackages(known_paths)
  File "/home/[...]/Rpmbuild/BUILD/Python-2.7.1/Lib/site.py", line 315, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/home/[...]/Rpmbuild/BUILD/Python-2.7.1/Lib/site.py", line 192, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/home/[...]/Rpmbuild/BUILD/Python-2.7.1/Lib/site.py", line 162, in addpackage
    exec line
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/abrt_exception_handler.py", line 27, in <module>
    import syslog
ImportError: No module named syslog
[15673 refs]
make: *** [sharedmods] Error 1
error: Bad exit status from /home/[...]/Rpmbuild/TMP/rpm-tmp.x75Dze (%build)


RPM build errors:
    Bad exit status from /home/[...]/Rpmbuild/TMP/rpm-tmp.x75Dze (%build)


Expected results:
NOTA

Additional info:
0) Removing the abrt-addon-python package solves this problem.

1) I don't really understand what's going on here. This seems to be the first time I ran into a .pth file. I'm guessing that syslog (a built-in module) isn't yet available in this stage of the python built.

Comment 1 Jiri Moskovcak 2011-01-24 15:36:01 UTC
(In reply to comment #0)

> Additional info:
> 0) Removing the abrt-addon-python package solves this problem.
> 

- abrt-addon-python should depend on python, because it doesn't make sense to install this without python, need to fix this in spec

> 1) I don't really understand what's going on here. This seems to be the first
> time I ran into a .pth file. I'm guessing that syslog (a built-in module) isn't
> yet available in this stage of the python built.

- .pth is meant to be run every time python interpreter is run and this seems to me like python is using some bootstrapping to build/install it's files and as you say, the syslog module is not available

Comment 2 Paul Bolle 2011-01-24 17:32:29 UTC
(In reply to comment #1)
> - abrt-addon-python should depend on python, because it doesn't make sense to
> install this without python, need to fix this in spec

It already does. See:
rpm -q --requires abrt-addon-python
[...]  
python(abi) = 2.7
[...]

Just to be clear, the issue that I ran into is that if abrt-addon-python is installed (and therefore python is installed too) building python with rpmbuild isn't possible.

Comment 3 Dave Malcolm 2011-01-24 23:54:01 UTC
I spent some time looking at this; I'm seeing it both in rawhide (2.7) and on RHEL6 (2.6)

The site.py module looks for ".pth" files in various directories, when it finds a .pth file:
       For each line in the file, either combine it with sitedir to a path
       and add that to known_paths, or execute it if it starts with 'import '.

abrt-addon-python has this file:
  /usr/lib/python2.7/site-packages/abrt.pth
containing:
  import abrt_exception_handler

It's happening in "make sharedmods", which invokes the freshly-built python binary on the setup.py

In particular, the "syslog" module is built as part of this step.

Verbosely invoking the freshly-built python with "import site" from within the build directory:

  ./python -v -c "import site"

shows a series of imports, all being done from within the build tree.  This is followed by an import of "abrt_exception_handler", which (on RHEL6) goes to
  # /usr/lib/python2.6/site-packages/abrt_exception_handler.pyc matches /usr/lib/python2.6/site-packages/abrt_exception_handler.py
i.e. it uses the installed python packages tree.

Looking at sys.path (builddir here was "/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6"):
  ./python -c "import sys; print sys.path"

we get:
  'import site' failed; use -v for traceback
  ['', '/usr/lib64/python26.zip', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Lib', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Lib/plat-linux2', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Lib/lib-tk', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Lib/lib-old', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Modules', '/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/build/lib.linux-x86_64-2.6', '/home/david/.local/lib/python2.6/site-packages', '/usr/lib64/python2.6/site-packages', '/usr/lib64/python2.6/site-packages/gst-0.10', '/usr/lib64/python2.6/site-packages/gtk-2.0', '/usr/lib64/python2.6/site-packages/webkit-1.0', '/usr/lib/python2.6/site-packages']

So it contains the "site-packages" of the installed tree, but not from the builtin packages of the installed tree (and hence the installed Python's "syslog.so" isn't found).

I tried adding "-S" to invocations of ./python, to suppress the implicit "import site" during startup, but it still fails, because:
   Lib/distutils/command/build_ext.py
is importing it explicitly:
  File "/home/david/coding/cvs-dist/python/RHEL-6/Python-2.6.6/Lib/distutils/command/build_ext.py", line 13, in <module>
    from site import USER_BASE, USER_SITE

On my RHEL 6 box:
  $ find /usr/lib*/python2.6 -name "*.pth"
  /usr/lib64/python2.6/site-packages/pygst.pth
  /usr/lib64/python2.6/site-packages/pygtk.pth
  /usr/lib64/python2.6/site-packages/webkit-1.0.pth
  /usr/lib/python2.6/site-packages/abrt.pth
and only abrt.pth has an "import line; the others merely list package names.

sys.exec_prefix and sys.prefix are coming out as "/usr/".

They are calculated by Modules/getpath.c:calculate_path

    if (!(pfound = search_for_prefix(argv0_path, home))) {

and:
  (gdb) p pfound
  $24 = -1
which is the return value from search_for_prefix to signify that we're in the build directory (as indeed we are).

This means that calculate_path() falls back to using:
  642        strncpy(prefix, PREFIX, MAXPATHLEN);
and this is set up in the Makefile via:

    Modules/getpath.o: $(srcdir)/Modules/getpath.c Makefile
  	$(CC) -c $(PY_CFLAGS) -DPYTHONPATH='"$(PYTHONPATH)"' \
		-DPREFIX='"$(prefix)"' \
		-DEXEC_PREFIX='"$(exec_prefix)"' \
		-DVERSION='"$(VERSION)"' \
		-DVPATH='"$(VPATH)"' \
		-o $@ $(srcdir)/Modules/getpath.c

This reflects this comment in getpath.c:
     * If we're loading relative to the build directory,
     * return the compiled-in defaults instead.
which seems to have been the case since at least this commit from 1997
  http://svn.python.org/view?view=rev&revision=7775


So as it stands, Python expects when built from its build location to look in /usr/ for python libs, and executes .pth files found in those locations.  ABRT supplies one, and python uses it.

I don't think this is cleanly fixable from the Python side.

One workaround is to not have the abrt-addon-python package installed when building python (does rpm have a BuildConflicts: specfile tag?)

Another may be to move the "import syslog" into the callsites that use it, rather than in module scope.

Comment 4 Paul Bolle 2011-01-25 09:02:07 UTC
(In reply to comment #3)
> One workaround is to not have the abrt-addon-python package installed when
> building python (does rpm have a BuildConflicts: specfile tag?)

Yes. For example, adding
    BuildConflicts: abrt-addon-python

will result in
    rpmbuild -bp $SPECS/python.spec
    error: Failed build dependencies:
            abrt-addon-python conflicts with python-2.7.1-5.fc15.i686

if abrt-addon-python is installed.

(Maybe the error should have been "[...] buildconflicts with [...]", but that's off topic.)

Comment 5 Jiri Moskovcak 2011-01-25 09:34:42 UTC
or we can just catch this exception in abrt hook..

Comment 6 Dave Malcolm 2011-02-10 17:28:33 UTC
*** Bug 676661 has been marked as a duplicate of this bug. ***

Comment 8 Jiri Moskovcak 2011-06-10 10:47:47 UTC
Fixed in git, thanks Dave, for the hints, I moved import syslog from the module scope and it seems to fix that problem even though python fails to build on my machine with error:

2 skips unexpected on linux2:
    test_tk test_ttk_guionly
[1130032 refs]
make: *** [test] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.4WULQM (%check)

but that's probably not because of ABRT.

Comment 9 Fedora Update System 2011-06-21 13:44:21 UTC
abrt-2.0.3-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/abrt-2.0.3-1.fc15

Comment 10 Fedora Update System 2011-06-21 23:55:18 UTC
Package abrt-2.0.3-1.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing abrt-2.0.3-1.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/abrt-2.0.3-1.fc15
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2011-06-24 03:22:17 UTC
abrt-2.0.3-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.