There is a few bundled libraries ( besides the one that you already removed, bundling qt is just insane ) :

- linenoise ( likely modified )

- coffeescript

There is the directory ghostdriver, that bundle and modify code from https://github.com/detro/ghostdriver

there is a copy of  qcommandline too.


And given the mess that the tarball is from a license points of view, I think it may be easier for us to clean it. 
Pro :
- smaller tarball ( smaller srpm )
- less noise if someone decide to use coverty or anything on the code
- easier to review from a license and bundling point of view
Cons :
- more work for you as you need to write a cleaning script and use it

(In reply to comment #1)
> There is a few bundled libraries ( besides the one that you already removed,
> bundling qt is just insane ) :
> 
> - linenoise ( likely modified )

I missed this one. Why do all these folks think that bundling all this junk in their source is a good approach, instead of just linking against it? They have even written scripts to automate the bundling process :-(

I might try porting phantomjs to readline instead. linenoise looks annoying, the author states it is intended to be bundled... But at least phantomjs isn't carrying any modifications to linenoise that I can find.

> - coffeescript

This one might be tricky... there is an old review request for coffeescript in bug 732216, which was closed due to the problems with nodejs, but now that nodejs is going to be landing in Fedora perhaps it can be revisited. I'm not sure if coffeescript needs nodejs at runtime either. At any rate, phantomjs uses only the pure js parts from it... so I'm not sure how that will work if coffeescript is packaged as an npm package or whatever.

> There is the directory ghostdriver, that bundle and modify code from
> https://github.com/detro/ghostdriver

I think this one should stay. Ghostdriver support is linked into the phantomjs binary, and ghostdriver is not usable without phantomjs. There is very close cooperation between the authors of phantomjs and ghostdriver, and in fact I suspect in the very near future ghostdriver will just be folded into phantomjs completely.

> there is a copy of  qcommandline too.

I'm not sure what to do about this one. qcommandline has been declared dead by its original author, and the phantomjs bundled version has lots of modifications so it is essentially a fork. I saw some talk about switching phantomjs over to a better-supported alternative in the future, not sure if anyone is working on that though.

> And given the mess that the tarball is from a license points of view, I
> think it may be easier for us to clean it. 
> Pro :
> - smaller tarball ( smaller srpm )
> - less noise if someone decide to use coverty or anything on the code
> - easier to review from a license and bundling point of view
> Cons :
> - more work for you as you need to write a cleaning script and use it

Agreed that cleaning the tarball is worthwhile. I will work on a script for it.

For ghostdriver, i would prefer to have the FESCO approval. ( even if I agree with you that it doesn't make sense and could be bundled, if something is a integral part of the software, why is it developped in a separate git in the first place, especially since the name of the developper of ghostdriver also appear in the source code of phantomjs )


Also, you remove breakpad from the list of bundle, but nothing replace it, I assume that's because we use abrt in Fedora ?

After looking at the source code, I see they put everything in the binary using QT ressources. As this is a little bit ugly ( ie, this consume more memory, and force us to bundle coffeescript in the binary instead of using another copy ), I assume we cannot simply ship the file out of the main binary ? ( as this would let us solve the bundle of coffeescript with a simple symlink )

Folks,

Thanks for taking the time to look into packaging this officially!

First of all, let me say than rather than "why is this junk bundled?" assumption, why not discussing it in a suitable channel? We have a mailing-list for the discussion and I'm sure a lot of other people (esp. potential packagers) will benefit a lot from the feedback. I believe such a constructive input will be appreciated a lot. 

Most of your concerns was not a deliberate or philosophical choice, it's pure practical as in "we don't have any contributor for packaging this stuff so let's just do it this way". There's only 24 hours/day and this is the best we can do given the limited time and other compromises.

Some more details of third party libraries. We're very open to have it built using the system libraries whenever they are available (see also http://code.google.com/p/phantomjs/issues/detail?id=415, though nobody got the time to solve it). This is likely perfect for e.g. mongoose. As for readline vs linenoise, IIRC it was because readline didn't work for us very well (again, just start a question on our mailing-list). QCommandLine needs to be replaced, there's nobody contributing that yet.

I'm not sure CoffeeScript can be decoupled easily. The reason is the module support. If you just run one script, surely an external coffee-to-js will do the job, but once modules are used (via requirejs), the hooks need to be right inside PhantomJS itself.

As for Qt itself, there is some background covered in http://ariya.ofilabs.com/2012/03/the-evolution-of-phantomjs-build-workflow.html. If you switch to system Qt, some functionalities might be crippled (including remote debugging, file upload, web font, JavaScript stack trace) not including potential crashes. So far there is no sign of Digia producing Qt 4.9 which may or may not fix those issues. Also, we want to support pure headless (sans X11) which gets really tricky to be implemented with stock Qt.

Last but not least, we want to include a fresher QtWebKit, see http://code.google.com/p/phantomjs/issues/detail?id=31 for some details. Not sure how this can be solved without our current solution so far.


I'll be happy to address more concerns or feel free to start a thread on our mailing-list.

Thank you!

(In reply to comment #5)

Thanks for joining the conversation here, Ariya.

My comment #2 about "bundling junk" was rude and uncivil, I apologise for that. I was frustrated after spending the morning trying to package linenoise. Actually my frustration was mainly with linenoise, for actively encouraging embedding in other programs and not building into a shared library.

I do understand that there are many good reasons for you to bundle these third party dependencies, but Fedora has very strong policies against bundled dependencies (also for very good reasons). It's possible to apply for exemptions from FESCO but I doubt any of the reasons here would be sufficient, except perhaps for coffee-script due to the module loading issues (and ghostdriver, which I don't consider to be a third-party dependency at all).

I'm also aware there will be some missing functionality with stock Qt. Right now I have 20 failures from the test suite, I think the most serious problem is file uploads, I am still hoping to find a solution for that. The test suite was also triggering a crash in qtwebkit but I filed that as bug 891464 and it is already fixed.

I am very keen to see phantomjs shipped in Fedora because I think it's an excellent tool, but the only way that will happen is if I can find a way to get it building against packaged versions of all its dependencies.

(In reply to comment #3)
> Also, you remove breakpad from the list of bundle, but nothing replace it, I
> assume that's because we use abrt in Fedora ?

Yes, exactly. No need for us to handle SIGSEGV inside phantomjs.

Linenoise is packaged, its review is bug 892988. I have here a (trivial) patch for phantomjs to unbundle it, not worth posting a new SRPM yet though.

QCommandline is the next one I will need to tackle... Since it seems nobody is willing to fork and maintain it (certainly I am not), I will look into porting phantomjs to something else instead.

There doesn't seem to be any consensus on what QCommandLine should be replaced with, so I rewrote the phantomjs patches for it as backwards-compatible enhancements and submitted them upstream last week. However I now see the upstream bug tracker has gone AWOL (returning 404s for everything), not sure if that is a bad sign...

Anyway, QCommandLine review is filed as bug 894954 (including the patches for phantomjs) and I've written a patch to unbundle it.

I think the only one left now is coffeescript...

Hi Dan,

I'm glad that you are making progress with this.


I'm still curious on how to solve remote debugging, web font, JavaScript stack trace, while still using a system-installed Qt. Are you going to have a different version of Qt package for this?

Also, the headless feature of PhantomJS is quite popular for CI system. Are you going to skip this as well?

The possible replacement for QCommandLine is QApplicationArgument:
http://code.google.com/p/phantomjs/issues/detail?id=746

(In reply to comment #10)
> I'm still curious on how to solve remote debugging, web font, JavaScript
> stack trace, while still using a system-installed Qt. Are you going to have
> a different version of Qt package for this?
> 
> Also, the headless feature of PhantomJS is quite popular for CI system. Are
> you going to skip this as well?

I can't ship a fork of Qt and QtWebKit in Fedora just for PhantomJS. If the Qt patches that PhantomJS is relying on have been accepted and applied upstream, and if they backport to 4.8 without breaking ABI/API, then I can request the maintainer of qt (or qtwebkit) apply them in the Fedora package.

Otherwise, the only choices are to:
- find a way to implement the feature in PhantomJS against stock Qt (I am still hoping to do this for file uploads, since I really need those);
- leave the feature patched out, and include a README.Fedora file explaining the limitations of the packaged version (I am afraid I might have to do this for stack traces and headless support); or
- stall this review until such time as PhantomJS works against stock Qt (perhaps when Qt5 is released?)

we have phantomjs-1.9.0 
in https://code.google.com/p/phantomjs/downloads/list

http://www.serjux.com/phantomjs/phantomjs.spec 
http://www.serjux.com/phantomjs/phantomjs-1.9.0-1.fc18.src.rpm

I had to changed this 2 patches, to build the package:
http://www.serjux.com/phantomjs/0004-unbundle-qt.patch
http://www.serjux.com/phantomjs/0002-unbundle-mongoose.patch

Hi , I try do more work on this phantomjs, but not finish so we should add :

+BuildRequires:  qcommandline-devel
+BuildRequires:  linenoise-devel
+BuildRequires:  coffee-script

+# remove source bundle
+rm -rf src/qt 
+rm -rf src/qcommandline
+rm -rf src/linenoise
+rm -rf src/ghostdriver
+rm -rf src/breakpad
+rm -rf src/coffee-script
+#rm -rf src/mongoose 

and patch with backup help on update the patch for new releases 
+%patch1 -p1 -b .giflib
+%patch2 -p1 -b .mongoose
+%patch3 -p1 -b .breakpad
+%patch4 -p1 -b .qt
+%patch5 -p1 -b .gifwriter

I upload my state of art to http://www.serjux.com/phantomjs/stateofart/ 

maybe just next week I continue

Thanks for the suggestions, Sergio. I'm publishing my patches in the "fedora" branch of my phantomjs fork on Github, in case this makes it easier for you to work with them:

https://github.com/danc86/phantomjs/commits/fedora

I've updated everything for 1.9.0, the new spec and SRPM are here:

http://fedorapeople.org/~dcallagh/phantomjs/phantomjs.spec
http://fedorapeople.org/~dcallagh/phantomjs/phantomjs-1.9.0-1.fc20.src.rpm

There are no more bundled dependencies left, but there are some test failures which I will need to investigate further:

http://fedorapeople.org/~dcallagh/phantomjs/test-failures-1.9.0-1.txt

Most of those are due to missing enhancements in stock Qt. And there is no headless support for the same reason.

I just stumbled upon phantomjs. How's this going ahead? Are you just waiting for a reviewer?

(In reply to Volker Fröhlich from comment #16)
> I just stumbled upon phantomjs. How's this going ahead? Are you just waiting
> for a reviewer?

No this is unfortunately not ready. The current status is that phantomjs builds successfully, but with reduced functionality and many test failures (see comment 15).

It *is* still useful in its current state (for example as a WebDriver implementation when used with Xvfb) but it is not really proper phantomjs so not suitable to ship in Fedora as is.

If you'd like to help, you can work from the fedora branch of my GitHub fork:

git clone https://github.com/danc86/phantomjs.git
cd phantomjs/
git checkout fedora
wget http://fedorapeople.org/~dcallagh/phantomjs/phantomjs.spec
sudo yum-builddep phantomjs.spec
qmake-qt4
make

Or if you'd like to use the phantomjs package in its current state, you can grab the SRPM and build it yourself:

wget http://fedorapeople.org/~dcallagh/phantomjs/phantomjs-1.9.0-1.fc20.src.rpm
mock -r fedora-19-x86_64 --rebuild phantomjs-1.9.0-1.fc20.src.rpm

heads up!

i wanted to inform that i updated 1.9.0 -> 1.9.6 some of the patches:
https://github.com/pld-linux/phantomjs

(In reply to Elan Ruusamäe from comment #18)
> heads up!
> 
> i wanted to inform that i updated 1.9.0 -> 1.9.6 some of the patches:
> https://github.com/pld-linux/phantomjs

your specs is an updated of Dan Callaghan ? if so what is the patch between his and yours spec ? I can get the other patches

works on Fedora 20 ? F21/rawhide ?

I've updated my spec for 1.9.7:

http://fedorapeople.org/~dcallagh/phantomjs/phantomjs.spec
http://fedorapeople.org/~dcallagh/phantomjs/phantomjs-1.9.7-1.fc20.src.rpm

Nothing much has changed though, it builds and runs but some features are missing or broken when built against stock Qt4:

http://fedorapeople.org/~dcallagh/phantomjs/test-failures-1.9.7-1.txt

I'm working on building the Qt5 port:

https://github.com/Vitallium/phantomjs-qt5

since more of phantomjs's Qt patches are (hopefully) upstreamed in Qt 5.3.0 which is now in Fedora.

I did  fedora-review -b 891461  , 
and have one warning that worth mention 
 
phantomjs.x86_64: W: no-manual-page-for-binary phantomjs

So for me the package is Approved 

but when running  phantomjs 

I got this error :
Error: Cannot find module '/usr/lib/node_modules/coffee-script'

(In reply to Sergio Monteiro Basto from comment #21)
> I got this error :
> Error: Cannot find module '/usr/lib/node_modules/coffee-script'

That's a mistake in the Requires, it needs coffee-script (in addition to coffee-script-common).

I have to test it more seriously, but after install coffee-script  I got : 
Error: Cannot find module 'module'

(In reply to Sergio Monteiro Basto from comment #23)
> I have to test it more seriously, but after install coffee-script  I got : 
> Error: Cannot find module 'module'

same here, strace shows this:

stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module.js", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module.json", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module.coffee", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module.litcoffee", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module.coffee.md", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/package.json", 0xbfe5a770) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/index.js", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/index.json", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/index.coffee", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/index.litcoffee", 0xbfe5a780) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/node_modules/coffee-script/lib/coffee-script/node_modules/module/index.coffee.md", 0xbfe5a780) = -1 ENOENT (No such file or directory)


I have no clue in which package to find these files?

It's coffee-script.

You should not only drop the bundled coffeescript, but also remove all references. 

And, please use macro %qmake_qt4.

That "Error: Cannot find module 'module'" does not seem to be so serious, further testing however reveals that everything using eg page.open(){...} hangs forever befor getting into the body :(

With --debug=yes it seems that it hangs consistently after completely loading the page and printing
  2014-08-06T22:57:43 [DEBUG] WebPage - setupFrame "" 
message.

Version 2.x of phantomjs will be completely without builtin coffeescript support so maybe that would be an easier target?

Any progress here?

Sorry, I stopped working on this package (and following phantomjs development) a while ago. Once I started hitting segfaults where the stack trace was just a huge mountain of generated code from Webkit JSCore, I kinda gave up. To me slimerjs seems like a more promising alternative.

If anyone wants to take over this review they are most welcome. I will do my best to help.

Just an update on state of Phantomjs. Recently a PR(https://github.com/ariya/phantomjs/pull/14386) for using a system Qt was merged. So I think when next release version is released we should now be able to package this without bigger problems.

There is now version 2.1.1 available from upstream. Is the 2.x series API compatible with the 1.x series?

Dan, do you still have interest in this package? How can we continue the review?

I haven't looked at phantomjs in a long time. My (now very outdated) unbundling patches are still at:

https://github.com/danc86/phantomjs/compare/1.9...fedora

in case they are still helpful. If anyone else wants to take this package review and update it for phantomjs 2.x, please feel free.

last test are dated from 2014, maybe the review should start from scratch

Maybe I could take this request and look into the open topics but not before next week.
Wondering if we need two packages phantomjs1 and phantomjs (version 2.x). There are some inofficial efforts to bring OBS into EPEL7 and OBS seems to need phantomjs in version 1.x series, propably because of some API breakage.

No, my port of OBS for Fedora uses PhantomJS 2:
https://obs.infoserver.lv/project/show/obs:server:fedora:2.7
(this is OBS 2.7; 2.8 port is still in progress, will be ready soon).

BTW, this OBS server already runs on Fedora 25.

Oleg, thanks for your work and the hint.
k0da from OpenSUSE community has another project with packages for CentOS 7: Unfortunately, build.opensuse.org is currently in maintanance mode, so I can not find the project URL.

Oleg, you may also be interested in case of OBS 2.8 for Fedora in a too old rubygem-concurrent-ruby, see bug #1426969.

(In reply to Oleg Girko from comment #36)
> No, my port of OBS for Fedora uses PhantomJS 2:
> https://obs.infoserver.lv/project/show/obs:server:fedora:2.7
> (this is OBS 2.7; 2.8 port is still in progress, will be ready soon).
> 
> BTW, this OBS server already runs on Fedora 25.

Oleg, how can this work without coffeescript and with qt5? Does OBS use javascript only? Underlying browser emulation / engine might be mostly irrelevant, in concern of the API enhancement of Qt and WebKit.

From the homepage:

Updated Qt and WebKit

This release, and the upcoming 2.x series, is different that PhantomJS 1.x since it switches the core implementation to use Qt 5 to get a more modern base platform, as well as to have an updated WebKit module.

Removal of CoffeeScript support

PhantomJS 2 can not run scripts written in CoffeeScript anymore (see issue 12410). As a workaround, CoffeeScript users can still compile their scripts to JavaScript first before executing it with PhantomJS.

(In reply to Raphael Groner from comment #39)
> Oleg, how can this work without coffeescript and with qt5?

Yes, somehow it works. It causes some problems with webui tests in OBS 2.7, mostly related to timing, but I think, most of the problems were fixed in 2.8.

> Does OBS use javascript only?

Yes, OBS uses just basic JavaScript.

Taken from:
https://github.com/ariya/phantomjs/issues/15344

Archiving the project: suspending the development #15344

Due to the lack of active contribution, I am going to archive this project soon.

At some point in the future, if we pick up the development again (such as #15341, #15342, #15343), the project will be unarchived.

With that, all the earlier plans regarding PhantomJS 2.5 (from @Vitallium) or 2.1.x (from @pixiuPL) will be abandoned effective immediately. Consequently, the source and binary packages for the above abandoned version will be removed to avoid any confusions. PhantomJS version 2.1.1 will remain the last known stable release until further notice.

To keep the source repository in a sane situation:

    the master branch will be preserved under the new bleeding-edge branch.
    after that, the master branch will be restored back to the approximate state of v2.1.1

That way, if anyone wants to improve PhantomJS for their own usages (internal fork, in-house patches), the master branch can still serve as a good baseline. It can still be built from source and it will still work on macOS, Linux, and Windows.


I assume, we should close this review request, as the project upstream is dead.

The upstream repo is now archived. As the original submitter has stopped using PhantomJS and likely has no need for this package, I'm going to close this Review Request.

Too sad but it has to happen.

In assumption because xWebKit itself is nearly dead, use xWebEngine instead, at least in Qt world.