Bug 436581 - RFE: Enable PyXPCOM extension in xulrunner
Summary: RFE: Enable PyXPCOM extension in xulrunner
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: xulrunner
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Christopher Aillon
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords: FutureFeature
Depends On:
Blocks: FedoraOLPCDelta
TreeView+ depends on / blocked
 
Reported: 2008-03-08 01:39 UTC by Peter Robinson
Modified: 2018-04-11 12:35 UTC (History)
9 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2008-10-12 16:06:23 UTC


Attachments (Terms of Use)
Build python/xpcom extension (2.72 KB, patch)
2008-03-12 18:05 UTC, Marco Pesenti Gritti
no flags Details | Diff
Full patch (5.33 KB, patch)
2008-03-13 01:28 UTC, Marco Pesenti Gritti
no flags Details | Diff

Description Peter Robinson 2008-03-08 01:39:11 UTC
ActiveState has just released their Komodo editor as open source under the
openkomodo name at openkomodo.com but it needs the PyXPCOM extension to be
enabled to be able to build on Fedora. It would be cool to be able to build it
without having to recompile xulrunner (although there looks to be more work to
be able to fully do that).

Comment 1 Matěj Cepl 2008-03-08 19:46:37 UTC
nothing to triage

Comment 2 Marco Pesenti Gritti 2008-03-12 10:56:43 UTC
Working on this, I need it for Sugar.

Comment 3 Marco Pesenti Gritti 2008-03-12 18:01:13 UTC
Opened a bug about a build issue on x86_64:

https://bugzilla.mozilla.org/show_bug.cgi?id=422463

Comment 4 Marco Pesenti Gritti 2008-03-12 18:05:23 UTC
Created attachment 297819 [details]
Build python/xpcom extension

OK to apply this and do a build?

Comment 5 Peter Robinson 2008-03-13 01:08:45 UTC
There's also other breakage on x86_64 once this patch is applied. This was
against the 0.38 build but I'm not sure if that's because I'm compiling that of
Fedora 8.

c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor
-Wcast-align -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar
-pthread -pipe  -DNDEBUG -DTRIMMED -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
-DPYTHON_SO=\"libpython2.5.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o
libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o
nsPyDOMISupports.o nsPyTimeout.o     -lpthread
-Wl,-rpath,/usr/lib64/xulrunner-1.9pre  -Wl,-rpath-link,../../../../dist/bin 
../../../../dist/lib/libxpcomglue_s.a -L../../../../dist/bin -lxpcom
-L../../../../dist/bin -lxpcom -lxul  -L../../../../dist/bin
-L../../../../dist/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl 
-Wl,--version-script
-Wl,../../../../build/unix/gnu-ld-scripts/components-version-script
-Wl,-Bsymbolic -ldl -lm  -L/usr/lib64 -lpython2.5 -lutil -L../../../../dist/lib
-lpyxpcom  -L../../../../dist/bin -lmozjs  
/usr/bin/ld: nsPyContext.o: relocation R_X86_64_PC32 against
`PyMarshal_ReadObjectFromString' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
gmake[7]: *** [libpydom.so] Error 1
gmake[7]: Leaving directory
`/usr/src/redhat/BUILD/mozilla/extensions/python/dom/src'
gmake[6]: *** [libs] Error 2
gmake[6]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions/python/dom'
gmake[5]: *** [libs] Error 2
gmake[5]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions/python'
gmake[4]: *** [libs] Error 2
gmake[4]: Leaving directory `/usr/src/redhat/BUILD/mozilla/extensions'


Comment 6 Marco Pesenti Gritti 2008-03-13 01:28:56 UTC
Created attachment 297875 [details]
Full patch

Comment 7 Marco Pesenti Gritti 2008-03-13 01:30:49 UTC
Peter, I attached the full patch (including spec changes etc). The error you are
seeing is in pydom, which I'm not building at the moment. Would be nice to
figure out that too though...

Comment 8 Marco Pesenti Gritti 2008-03-13 23:52:46 UTC
The pydom error is probably another instance of this one:
http://benjamin.smedbergs.us/blog/2005-10-27/gcc-40-workaround/
In fact compiling with ac_cv_visibility_pragma=no works.


Comment 9 Jon Stanley 2008-04-23 20:29:33 UTC
Adding FutureFeature keyword to RFE's.

Comment 10 Michael Stone 2008-05-06 23:15:25 UTC
A few extra comments pertaining to building PyXPCOM on top of
xulrunner-1.9-0.60.beta5.f9 in fedora-9-i386 mock chroots:

(Analysis partly due to Christopher Aillon and Scott Ananian):

http://developer.mozilla.org/en/docs/Building_PyXPCOM suggests adding 

ac_add_options --enable-extensions=python,default

to xulrunner-mozconfig and adding python-devel as a BuildRequires in order to
build PyXPCOM. This causes the xulrunner build system to attempt to build PyDOM
which dies with linker errors like 

c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor
-Wcast-align -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar
-pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fno-reorder-functions 
-DPYTHON_SO=\"libpython2.5.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o
libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o
nsPyDOMISupports.o nsPyTimeout.o     -lpthread
-Wl,-rpath-link,../../../../dist/bin  ../../../../dist/lib/libxpcomglue_s.a
-L../../../../dist/bin -lxpcom -L../../../../dist/bin -lxpcom -lxul 
-L../../../../dist/bin -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl 
-Wl,--version-script
-Wl,../../../../build/unix/gnu-ld-scripts/components-version-script
-Wl,-Bsymbolic -ldl -lm  -L/usr/lib -lpython2.5 -lutil -L../../../../dist/lib
-lpyxpcom  -L../../../../dist/bin -lmozjs
nsPyContext.o: In function `nsPythonContext::Deserialize(nsIObjectInputStream*,
nsScriptObjectHolder&)':
nsPyContext.cpp:(.text+0xc79): undefined reference to
`PyMarshal_ReadObjectFromString'
nsPyContext.o: In function `nsPythonContext::Serialize(nsIObjectOutputStream*,
void*)':
nsPyContext.cpp:(.text+0xd58): undefined reference to
`PyMarshal_WriteObjectToString'
/usr/bin/ld: libpydom.so: hidden symbol `PyMarshal_WriteObjectToString' isn't
defined
/usr/bin/ld: final link failed: Nonrepresentable section on output

To examine the failed build, rebuild with MOCKARGS="--no-cleanup-after" and
enter the buildroot. After changing to the extensions/python/dom/src folder in
the build tree, and checking that all include and linker flags are in order,

objdump --demangle -t nsPyContext.o | grep Marshal 

reports that both PyMarshal_WriteObjectToString and
PyMarshal_ReadObjectFromString are undefined and, contrary to our desires, hidden.

The reason that they are hidden seems to be that xulrunner does not wrap
appropriate gcc visibility pragmas around /usr/include/python2.5/marshal.h in
dist/include/system_wrappers/marshal.h (as is done for Python.h and most other
system headers).

I am also concerned that extensions/python/xpcom/src/PyXPCOM.h includes
<Python.h> instead of "Python.h" and therefore may be using the unwrapped
version but I did not verify the effects of this choice with the '-E'
compilation flag.

Private communication with Marco suggests that it may be possible to work around
these problems by avoiding PyDOM by building with 

--enable-extensions=default,python/xpcom   instead of
--enable-extensions=default,python

Finally, Cristopher suggested on the basis of the evidence described above that
it's time to start talking/filing-bugs with the upstream maintainers.


Comment 11 Michael Stone 2008-05-06 23:23:30 UTC
Future readers should also consider reviewing
https://bugzilla.mozilla.org/show_bug.cgi?id=273336 for background material on
why symbol visibility is being manipulated so pervasively throughout the
xulrunner codebase.

Comment 12 Peter Robinson 2008-06-17 12:54:01 UTC
The upstream build bug for x86_64 is fixed. What else needs to be fixed to get
this enabled? I know that the openkomodo guys also have some other patches for
pyxpcom in their repos but I don't know what the state of them getting them
merged upstream is, or whether they are relevant/useful for olpc.

Comment 13 Peter Robinson 2008-08-17 12:55:42 UTC
There's also these fixes/enhancement bugs for pyxpcom upstream. Not sure if there relevant for the OLPC use of xulrunner.

https://bugzilla.mozilla.org/show_bug.cgi?id=374851
https://bugzilla.mozilla.org/show_bug.cgi?id=374852
https://bugzilla.mozilla.org/show_bug.cgi?id=375318
https://bugzilla.mozilla.org/show_bug.cgi?id=450784

Comment 14 Simon Schampijer 2008-10-07 19:54:57 UTC
Has been enabled http://koji.fedoraproject.org/koji/buildinfo?buildID=65566 and I have been testing it with hulahop/browse-activity as working. I think we can close this one.

Comment 15 Peter Robinson 2008-10-12 16:06:23 UTC
Fixed in xulrunner 1.9.0.2-3

Comment 16 Gour 2009-07-09 11:55:28 UTC
Hi!

Attempt to build xulrunner-1.9.1 with 

ac_add_options --enable-extensions=default,python

fails on x86_64.

Here is the log:

[...]
rm -f libpydom.so
c++  -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-long-long -pedantic -fno-strict-aliasing -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -Os -freorder-blocks -fno-reorder-functions  -DPYTHON_SO=\"libpython2.6.so\" -fPIC -shared -Wl,-z,defs -Wl,-h,libpydom.so -o libpydom.so  nsPyArgArray.o nsPyContext.o nsPyRuntime.o nsPyDOMModule.o nsPyDOMISupports.o nsPyTimeout.o     -lpthread -Wl,-rpath,/usr/lib/xulrunner-1.9.1  -Wl,-rpath-link,/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -Wl,-rpath-link,/usr/lib  /var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/lib/libxpcomglue_s.a -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lxpcom -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lxpcom -lxul  -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl  -Wl,--version-script -Wl,../../../../build/unix/gnu-ld-scripts/components-version-script -Wl,-Bsymbolic -lasound -ldl -lm  -L/usr/lib -lpython2.6 -lutil -L../../../../dist/lib -lpyxpcom  -L/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/dist/bin -lmozjs  
nsPyContext.o: In function `nsPythonContext::Deserialize(nsIObjectInputStream*, nsScriptObjectHolder&)':
nsPyContext.cpp:(.text+0xb5d): undefined reference to `PyMarshal_ReadObjectFromString'
nsPyContext.o: In function `nsPythonContext::Serialize(nsIObjectOutputStream*, void*)':
nsPyContext.cpp:(.text+0xc1d): undefined reference to `PyMarshal_WriteObjectToString'
/usr/bin/ld: libpydom.so: hidden symbol `PyMarshal_WriteObjectToString' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make[7]: *** [libpydom.so] Error 1
make[7]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python/dom/src'
make[6]: *** [libs] Error 2
make[6]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python/dom'
make[5]: *** [libs] Error 2
make[5]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions/python'
make[4]: *** [libs] Error 2
make[4]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1/extensions'
make[3]: *** [libs_tier_app] Error 2
make[3]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make[2]: *** [tier_app] Error 2
make[2]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/var/abs/local/x11/xulrunner/src/mozilla-1.9.1'
make: *** [build] Error 2



Sincerely,
Gour

Comment 17 Martin Stransky 2009-11-24 13:50:33 UTC
It's because system_wrappers are missing marshal.h file. There is a fix here

diff -r 4b0ff648d2ee config/system-headers
--- a/config/system-headers	Tue Oct 13 12:27:24 2009 -0700
+++ b/config/system-headers	Tue Nov 24 14:49:28 2009 +0100
@@ -607,6 +607,7 @@
 pthread.h
 pwd.h
 Python.h
+marshal.h
 QDOffscreen.h
 Quickdraw.h
 QuickDraw.h

Comment 18 Martin Stransky 2009-11-24 13:54:46 UTC
Upstream bug - https://bugzilla.mozilla.org/show_bug.cgi?id=461652


Note You need to log in before you can comment on or make changes to this bug.