Bug 436581 - RFE: Enable PyXPCOM extension in xulrunner
RFE: Enable PyXPCOM extension in xulrunner
Product: Fedora
Classification: Fedora
Component: xulrunner (Show other bugs)
All Linux
low Severity medium
: ---
: ---
Assigned To: Christopher Aillon
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
Blocks: FedoraOLPCDelta
  Show dependency treegraph
Reported: 2008-03-07 20:39 EST by Peter Robinson
Modified: 2018-04-11 08:35 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-10-12 12:06:23 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Build python/xpcom extension (2.72 KB, patch)
2008-03-12 14:05 EDT, Marco Pesenti Gritti
no flags Details | Diff
Full patch (5.33 KB, patch)
2008-03-12 21:28 EDT, Marco Pesenti Gritti
no flags Details | Diff

  None (edit)
Description Peter Robinson 2008-03-07 20:39:11 EST
ActiveState has just released their Komodo editor as open source under the
openkomodo name at openkomodo.com but it needs the PyXPCOM extension to be
enabled to be able to build on Fedora. It would be cool to be able to build it
without having to recompile xulrunner (although there looks to be more work to
be able to fully do that).
Comment 1 Matěj Cepl 2008-03-08 14:46:37 EST
nothing to triage
Comment 2 Marco Pesenti Gritti 2008-03-12 06:56:43 EDT
Working on this, I need it for Sugar.
Comment 3 Marco Pesenti Gritti 2008-03-12 14:01:13 EDT
Opened a bug about a build issue on x86_64:

Comment 4 Marco Pesenti Gritti 2008-03-12 14:05:23 EDT
Created attachment 297819 [details]
Build python/xpcom extension

OK to apply this and do a build?
Comment 5 Peter Robinson 2008-03-12 21:08:45 EDT
There's also other breakage on x86_64 once this patch is applied. This was
against the 0.38 build but I'm not sure if that's because I'm compiling that of
Fedora 8.

c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor
-Wcast-align -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar
-pthread -pipe  -DNDEBUG -DTRIMMED -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
-DPYTHON_SO=\"libpython2.5.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o
libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o
nsPyDOMISupports.o nsPyTimeout.o     -lpthread
-Wl,-rpath,/usr/lib64/xulrunner-1.9pre  -Wl,-rpath-link,../../../../dist/bin 
../../../../dist/lib/libxpcomglue_s.a -L../../../../dist/bin -lxpcom
-L../../../../dist/bin -lxpcom -lxul  -L../../../../dist/bin
-L../../../../dist/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl 
-Wl,-Bsymbolic -ldl -lm  -L/usr/lib64 -lpython2.5 -lutil -L../../../../dist/lib
-lpyxpcom  -L../../../../dist/bin -lmozjs  
/usr/bin/ld: nsPyContext.o: relocation R_X86_64_PC32 against
`PyMarshal_ReadObjectFromString' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
gmake[7]: *** [libpydom.so] Error 1
gmake[7]: Leaving directory
gmake[6]: *** [libs] Error 2
gmake[6]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions/python/dom'
gmake[5]: *** [libs] Error 2
gmake[5]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions/python'
gmake[4]: *** [libs] Error 2
gmake[4]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions'
Comment 6 Marco Pesenti Gritti 2008-03-12 21:28:56 EDT
Created attachment 297875 [details]
Full patch
Comment 7 Marco Pesenti Gritti 2008-03-12 21:30:49 EDT
Peter, I attached the full patch (including spec changes etc). The error you are
seeing is in pydom, which I'm not building at the moment. Would be nice to
figure out that too though...
Comment 8 Marco Pesenti Gritti 2008-03-13 19:52:46 EDT
The pydom error is probably another instance of this one:
In fact compiling with ac_cv_visibility_pragma=no works.
Comment 9 Jon Stanley 2008-04-23 16:29:33 EDT
Adding FutureFeature keyword to RFE's.
Comment 10 Michael Stone 2008-05-06 19:15:25 EDT
A few extra comments pertaining to building PyXPCOM on top of
xulrunner-1.9-0.60.beta5.f9 in fedora-9-i386 mock chroots:

(Analysis partly due to Christopher Aillon and Scott Ananian):

http://developer.mozilla.org/en/docs/Building_PyXPCOM suggests adding 

ac_add_options --enable-extensions=python,default

to xulrunner-mozconfig and adding python-devel as a BuildRequires in order to
build PyXPCOM. This causes the xulrunner build system to attempt to build PyDOM
which dies with linker errors like 

c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor
-Wcast-align -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar
-pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fno-reorder-functions 
-DPYTHON_SO=\"libpython2.5.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o
libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o
nsPyDOMISupports.o nsPyTimeout.o     -lpthread
-Wl,-rpath-link,../../../../dist/bin  ../../../../dist/lib/libxpcomglue_s.a
-L../../../../dist/bin -lxpcom -L../../../../dist/bin -lxpcom -lxul 
-L../../../../dist/bin -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl 
-Wl,-Bsymbolic -ldl -lm  -L/usr/lib -lpython2.5 -lutil -L../../../../dist/lib
-lpyxpcom  -L../../../../dist/bin -lmozjs
nsPyContext.o: In function `nsPythonContext::Deserialize(nsIObjectInputStream*,
nsPyContext.cpp:(.text+0xc79): undefined reference to
nsPyContext.o: In function `nsPythonContext::Serialize(nsIObjectOutputStream*,
nsPyContext.cpp:(.text+0xd58): undefined reference to
/usr/bin/ld: libpydom.so: hidden symbol `PyMarshal_WriteObjectToString' isn't
/usr/bin/ld: final link failed: Nonrepresentable section on output

To examine the failed build, rebuild with MOCKARGS="--no-cleanup-after" and
enter the buildroot. After changing to the extensions/python/dom/src folder in
the build tree, and checking that all include and linker flags are in order,

objdump --demangle -t nsPyContext.o | grep Marshal 

reports that both PyMarshal_WriteObjectToString and
PyMarshal_ReadObjectFromString are undefined and, contrary to our desires, hidden.

The reason that they are hidden seems to be that xulrunner does not wrap
appropriate gcc visibility pragmas around /usr/include/python2.5/marshal.h in
dist/include/system_wrappers/marshal.h (as is done for Python.h and most other
system headers).

I am also concerned that extensions/python/xpcom/src/PyXPCOM.h includes
<Python.h> instead of "Python.h" and therefore may be using the unwrapped
version but I did not verify the effects of this choice with the '-E'
compilation flag.

Private communication with Marco suggests that it may be possible to work around
these problems by avoiding PyDOM by building with 

--enable-extensions=default,python/xpcom   instead of

Finally, Cristopher suggested on the basis of the evidence described above that
it's time to start talking/filing-bugs with the upstream maintainers.
Comment 11 Michael Stone 2008-05-06 19:23:30 EDT
Future readers should also consider reviewing
https://bugzilla.mozilla.org/show_bug.cgi?id=273336 for background material on
why symbol visibility is being manipulated so pervasively throughout the
xulrunner codebase.
Comment 12 Peter Robinson 2008-06-17 08:54:01 EDT
The upstream build bug for x86_64 is fixed. What else needs to be fixed to get
this enabled? I know that the openkomodo guys also have some other patches for
pyxpcom in their repos but I don't know what the state of them getting them
merged upstream is, or whether they are relevant/useful for olpc.
Comment 13 Peter Robinson 2008-08-17 08:55:42 EDT
There's also these fixes/enhancement bugs for pyxpcom upstream. Not sure if there relevant for the OLPC use of xulrunner.

Comment 14 Simon Schampijer 2008-10-07 15:54:57 EDT
Has been enabled http://koji.fedoraproject.org/koji/buildinfo?buildID=65566 and I have been testing it with hulahop/browse-activity as working. I think we can close this one.
Comment 15 Peter Robinson 2008-10-12 12:06:23 EDT
Fixed in xulrunner
Comment 16 Gour 2009-07-09 07:55:28 EDT

Attempt to build xulrunner-1.9.1 with 

ac_add_options --enable-extensions=default,python

fails on x86_64.

Here is the log:

rm -f libpydom.so
c++  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fno-reorder-functions  -DPYTHON_SO=\"libpython2.6.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o nsPyDOMISupports.o nsPyTimeout.o     -lpthread -Wl,-rpath,/usr/lib/xulrunner-1.9.1  -Wl,-rpath-link,/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -Wl,-rpath-link,/usr/lib  /var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/lib/libxpcomglue_s.a -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lxpcom -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lxpcom -lxul  -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl  -Wl,--version-script -Wl,../../../../build/unix/gnu-ld-scripts/components-version-script -Wl,-Bsymbolic -lasound -ldl -lm  -L/usr/lib -lpython2.6 -lutil -L../../../../dist/lib -lpyxpcom  -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lmozjs  
nsPyContext.o: In function `nsPythonContext::Deserialize(nsIObjectInputStream*, nsScriptObjectHolder&)':
nsPyContext.cpp:(.text+0xb5d): undefined reference to `PyMarshal_ReadObjectFromString'
nsPyContext.o: In function `nsPythonContext::Serialize(nsIObjectOutputStream*, void*)':
nsPyContext.cpp:(.text+0xc1d): undefined reference to `PyMarshal_WriteObjectToString'
/usr/bin/ld: libpydom.so: hidden symbol `PyMarshal_WriteObjectToString' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make[7]: *** [libpydom.so] Error 1
make[7]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python/dom/src'
make[6]: *** [libs] Error 2
make[6]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python/dom'
make[5]: *** [libs] Error 2
make[5]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python'
make[4]: *** [libs] Error 2
make[4]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions'
make[3]: *** [libs_tier_app] Error 2
make[3]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make[2]: *** [tier_app] Error 2
make[2]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make: *** [build] Error 2

Comment 17 Martin Stransky 2009-11-24 08:50:33 EST
It's because system_wrappers are missing marshal.h file. There is a fix here

diff -r 4b0ff648d2ee config/system-headers
--- a/config/system-headers	Tue Oct 13 12:27:24 2009 -0700
+++ b/config/system-headers	Tue Nov 24 14:49:28 2009 +0100
@@ -607,6 +607,7 @@
Comment 18 Martin Stransky 2009-11-24 08:54:46 EST
Upstream bug - https://bugzilla.mozilla.org/show_bug.cgi?id=461652

Note You need to log in before you can comment on or make changes to this bug.