Bug 484566

These are the current .jars:

[limb@fawkes gallery2]$ find . -name '*.jar'
./modules/uploadapplet/applets/GalleryRemoteAppletMini.jar
./modules/uploadapplet/applets/applet_img.jar
./modules/uploadapplet/applets/GalleryRemoteHTTPClient.jar
./modules/slideshowapplet/applets/GalleryRemoteAppletMini.jar
./modules/slideshowapplet/applets/applet_img.jar
./modules/slideshowapplet/applets/GalleryRemoteHTTPClient.jar
./modules/remote/applets/metadata-extractor-2.1.jar
./modules/remote/applets/GalleryRemote.jar
./modules/remote/applets/img.jar
./modules/core/classes/GalleryStorage/g2_db2.jar
./modules/panorama/java/Metamorphose.jar

How do I determine if the source is present for one?  It's been years since I did any java development at all.

Blocking FE-LEGAL

Not only does the source have to be present but the jar must also be built from the source when building the RPM, see http://fedoraproject.org/wiki/Packaging/Guidelines#No_inclusion_of_pre-built_binaries_or_libraries

As mentioned earlier, the gallery source distribution doesn't seem to contain the source files for the applets, but there does seem to be something in the gallery svn repo e.g. http://gallery.svn.sourceforge.net/viewvc/gallery/trunk/gallery_remote/com/gallery/GalleryRemote/
But do check the licences, e.g. http://gallery.svn.sourceforge.net/viewvc/gallery/trunk/java/Metamorphose/Copyright?view=markup doesn't seem to be acceptable for fedora )-:

I know the source must be present, I just didn't know what the source to a .jar looked like. I have deduced from the svn that it would be .java. :)

There are no .java files anywhere in the whole gallery2 tarball. I'll initiate a conversation with upstream.   Hopefully they will release a new tarball with the source included, if not, should I create a tarball with all the sources?  And drop the panorama plugin if it can't be relicensed?

Upstream bug:

https://sourceforge.net/tracker/index.php?func=detail&aid=2585568&group_id=7130&atid=107130

Upstream response:

-----------------------------
>Comment By: Chris Kelly (ckdake)
Date: 2009-02-10 11:14

Message:
Hello-

We are pretty much done putting effort into Gallery 2 except for security
bugfixes at this point, so changing out packing process likely won't
happen. 

1. The .java files are available in our svn repository, and we have never
included them in the .zip/.tgz downloads. Feel free to repackage things as
needed or use your build process to build the jars!

2. Feel free to just not include the panorama plugin as I'm fairly sure a
small percentage of our users use it.

-Chris
----------------------------

So, I'll work on the .java bits, and dropping panorama, as well as correcting:

https://bugzilla.redhat.com/show_bug.cgi?id=483523

Any update here?

I've been too busy with other Fedora, $_DAYJOB and real life bits to make progress on this.  I'm by no means a Java coder.  Any suggestions for someone to ping for assistance?  I suppose as a stopgap I could try pulling and .jars and see what breaks, but that's somewhat ugly. . .

My suggestion would be to use the gallery source from svn (which does seem to contain all source) instead of using the release tar-balls (which more have the character of binary packages).

So the first task would be to figure out what (if any) tag gallery2 uses for their releases. Unfortunately I've been too deep in other-life issues as well (hence I haven't been complaining) but if I get some time on my hands I'll give it a go.

That's the kicker, there isn't a tag per se as I understand it, they have a buildsystem that uses an old Sun Java version for compatibility.  OpenJDK supports the older output versions, but it's more than a simple sed on the build.xml.  I was hoping for a listing of the sources used for each .jar, but never really got that, see the tracker bug linked in #4.

gallery2-2.3-5.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/gallery2-2.3-5.fc10

New build uses sources to build jars at rpm build time.  Please test.  Tom, does this look OK to you?

Why are you building the jars during %prep, as opposed to %build?
Also, please document how you generated the gallery-jar-sources-2.3.tar.gz.

Aside from those issues, this seems to resolve the legal concerns.

Testing a fixed version now.  Essentially, I unpacked the shipped jars in an organized manner, created a script to build the correct jars from the correct files, and tarred up the whole thing.  The upstream build process was too cumbersome, and in fact overkill to generate these jars, as it also build some client tools we don't ship.

gallery2-2.3-5.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update gallery2'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-3675

Reopening, your new build doesn't fix the issue at all, you're just repacking the binary .class files into the .jar files, not building them from source. That's basically worthless and it's definitely not compliant with the guidelines.

Hint: unpacked JARs are NOT sources (except if they're source JARs). A JAR is just a ZIP file. Most JARs are binary only. A .class file is NOT source code, it's compiled bytecode. You have to rebuild ALL .class files from the corresponding .java files.

Sorry, I'm utterly new to Java, and my examination of one or two of the .class files showed ASCII text encapulating, of all things, PHP, and assumed they were all of that ilk.

How do I determine which .java files resulted in which .class files?

The names of the .java files normally match the ones of the .class file, so you should find those fairly easily, but given that the .class files were built with some ancient Java, there's really no way to be sure you're using the same version (rebuilding the .java files is likely to generate subtly different .class files which then fail a byte-by-byte comparison). It looks like even upstream isn't sure what they're shipping:

> Unfortunately, there is no way to be 100% sure that the .java files you get
> are the ones that were used to build the jars in the package that we
> distribute.

They really deserve a beating for this blatant disregard of their own license (the GPL).

They also say:

> However, the newest of all the .java files in svn should (in theory) compile to
> jars that work with the most recent version of Gallery 2. 

which looks to me like the best you can do: check out the plugins from SVN trunk, build them (and I think you do need to use their build system, you can't just use a simple script, the reason your simple script "works" is because it's just repacking an unpacked JAR, which is a trivial operation, but not what is required) and ship the result.

I've actually gotten some advice about building compatibly with newer java, but only on a file by file basis, not with their system, which I believe expects the proprietary 1.4 JDK, IIRC.

I've contacted them requesting assistance with this.  I'm not sure how the buildsystem works, and I hope we don't need the whole trunk checkout to do a build, as it's over 600 MB. . .

I've brought the source handling issue up before, and been told that work on 2.3 is essentially over, and 3.0 will be better in this regard. <crosses fingers>

I'm digging through the svn checkout and found a PHP script that seems to be the way to run builds.  The catch is it starts by doing a svn checkout.  I could in theory modify it to use a pre-checked out tree.  By default it doesn't rebuild java bits, from what I can tell.  When I trace through to where it should be building them, however(gallery2/lib/tools/bin/makeManifest.php) it looks like it's just tarring them up and doing checksums.

I'm either totally misreading this, or the java bits are already built in svn.

They're indeed included in the main SVN as binary JARs, you have to track down the SVN for the individual links, e.g. the link posted in comment #2:
http://gallery.svn.sourceforge.net/viewvc/gallery/trunk/gallery_remote/com/gallery/GalleryRemote/

Yes, they're scattered all over the place, upstream is making it a PITA to actually build their software from source. :-( (And they're not the only Java project doing that, unfortunately. That's why portable binaries suck, they make people believe source code is not useful.)

Oops, I mean: "track down the SVN for the individual *plugins*".

Sent to gallery-devel.net:

Hi, Jon Ciesla, Fedora maintainer for the gallery2 package.  We've run into a snag*, and I need some assistance.  The gallery-2.3-full.zip contains several prebuilt .jar files, which in turn contain prebuild .class files.  In fedora, we build from source, and so I am faced with either dropping gallery from Fedora (which I *really* don't want to do) or find a way to build from .java sources.

Is it documented anywhere either in the G2 wiki or in the tarball how to build the entire project from source?  I've done svn checkout of trunk, and I'm not sure which subdirectories need to be present at build or even how to do so. I'd like to not have to use the whole 600MB+ trunk checkout if I can avoid it.

I've found trunk/packaging/gallery2/build.php, which appears to do an svn checkout among other things, but I can't find where the .class files contained within the .jar files are build.

I should also point out, in Gallery bug 2585568, that the GPL stipulate that when binaries are shipped, the source must also be made available.  I'm sure it is, and I just haven't found it, but others might question the projects adherence to the GPL.

Any assistance would be greatly appreciated.

Thanks, 

Jon

*https://bugzilla.redhat.com/show_bug.cgi?id=484566

Thread on gallery-devel list (apologies for top-posting):

I understand your frustration.  I'm one of the more uptight GNUites I 
know, so I see it all the time.  ;) 

Hopefully one of the java folk will chime in.  In the meantime:

1. I used to do that as well, but find it easier to do mass deployments 
with packages.  To each their own, and I like that distros that include 
packages give admins the option.

2. I can live with that, under the circumstances.  I'll drop the remote, 
slideshowapplet, and uploadapplet module subpackages, and all the .jar 
files.  That will fix our legal issue.

3. See #2.

Re: the roadmap, am I missing something, or is there no 3.0 milestone?  
At any rate, it looks like late May/early June might be a reasonable 
estimate for 3.0, no?

When G3 is out, I'll package it to replace G2.

Thanks for the information, and the assistance.

Chris Kelly wrote:
> > My frustration derives from some of the comments in the fedora bug
> > tracker, so sorry about that!
> >
> > The number of people that know the Java side of G2 is very small.
> > Perhaps one of them will chime in here, but to compare it to the way G2
> > is packaged in Fedora currently, most of the work on that was a long
> > time ago.
> >
> > Per your options:
> >
> > 1. G2 is a PHP webapp so getting it from a vendor instead of through a
> > packaging system isn't too bad. I have bunches of Gentoo, CentOS, and
> > Debian systems all running PHP applications, and I've never liked using
> > the distro's package management system to install them.
> >
> > 2. G2 will work without the Java components, just the fancy
> > drag-and-drop uploader won't be there and it won't work with DB2.  There
> > are plenty of other upload methods available, so this shouldn't be that
> > big of a deal.
> >
> > 3.  I'd agree that it's not worth the time. Either of the above are a
> > much better idea.
> >
> > G3 Beta 1 should be next and should be soon.  You can see what is left here:
> >
> > http://apps.sourceforge.net/trac/gallery/roadmap
> >
> > I think work is finishing up on the G2 importer which should be in Beta
> > 1 in a functional form.  Albums/Images/Users should all transfer over,
> > but since the feature set of G3 is a lot smaller, there will be things
> > that don't transfer over (like all kinds of sizes for resizes,
> > permissions, etc)
> >
> > -Chris
> >
> > Jon Ciesla wrote:
> >   
>> >> Some, yes, that bug was closed automatically, and I was asked not to
>> >> pursue such topics in the bug tracker in 2750571.
>> >>
>> >> At any rate, I would agree that it seems like quite a lot of work to
>> >> build the java components.  I am a complete stranger to Java, and by
>> >> "build the ImageTools jar", I meant re-compose it from it's extracted
>> >> contents, which is not nearly as far as I need to be.
>> >>
>> >> My point is not that you're disregarding the letter of the GPL, since
>> >> the source is publicly available, just that some might misinterpret the
>> >> effort required to obtain the full source as disregarding the spirit of
>> >> the GPL, since though available, it's not as if one can say "download
>> >> tarball X from URL Y".  I'd love to see the process documented
>> >> somewhere, but that's neither here nor there.
>> >>
>> >> I understand that with G3 being imminent, G2 is effectively dead.  The
>> >> catch is, I don't know when G3 is coming, and G2 is already in Fedora
>> >> and has been for some time.  The packages currently available for
>> >> gallery in Fedora contain prebuilt jars.  Gallery should not have been
>> >> packaged this way in the first place, but that was done more than 3
>> >> years ago and is pretty much water under the bridge.
>> >>
>> >> The situation I find myself in as the new maintainer is that the package
>> >> as it stands violates Fedora's legal guidelines, and has a large user
>> >> base.  This leaves me with the following choices:
>> >>      1. Drop G2 from Fedora and package G3 (which presumably will not
>> >> have this problem) at it's release.  This would be very disappointing. 
>> >> Perhaps (and understandably) not for you, but for Fedora's G2 users,
>> >> myself included.
>> >>    2. Push a version of G2 minus the .jar files.  This would almost
>> >> certainly break G2, correct?
>> >>    3. Build the java bits from source.  Laborious, as it turns out, and
>> >> possibly not worth the time if G3 is coming soon.
>> >>
>> >> So, I guess, unless someone else knows of a simpler way to grab and
>> >> build the source, I'd like to know an ETA for G3, how the upgrade
>> >> process looks from G2 to G3, and if a jarless G2 would work at all.  G3
>> >> Alpha 3 is nearly a  month old, and based on the timing of Alpha 2, I'd
>> >> guess either Alpha 4, or a Beta of some type might be next, but that's
>> >> subject to my ignorance of your schedule.  :) 
>> >>
>> >> Chris Kelly wrote:
>> >>     
>>> >>> Didn't you cover some of this in the bug thread?
>>> >>>
>>> >>> https://sourceforge.net/tracker/?func=detail&aid=2585568&group_id=7130&atid=107130
>>> >>>
>>> >>>
>>> >>> "With some wrangling, I've been able to build the ImageTools jar"
>>> >>>
>>> >>> And we agreed that you could pull out the db2 stuff.
>>> >>>
>>> >>> It unfortunate that the Java things in Gallery 2 are such a mess, but
>>> >>> Gallery 3 is java-free and won't have this problem, and it wouldn't hurt
>>> >>> my feelings if G2 was not available in Fedora.  We're not putting any
>>> >>> more effort into Gallery 2.
>>> >>>
>>> >>> And per Kevin's Comment #18:  My "100% sure" comment was guessing how
>>> >>> much time you'd want to put in. you could certainly look at timestamps
>>> >>> of releases to see what applets were in svn at that time, then look at
>>> >>> when those applets where checked in, then get the code that was in trunk
>>> >>> for them at that time, check out that code, look at the build scripts
>>> >>> for that time, build them, etc, but that would take some time.  Still
>>> >>> not sure how us not making it easy to build a few small components of G2
>>> >>> from source, when it is possible if you work on it, is a "blatant
>>> >>> disregard" of the GPL.
>>> >>>
>>> >>> Just trying to figure out what more you're asking for. Thanks!
>>> >>>
>>> >>> -Chris
>>> >>>
>>> >>>
>>> >>> Jon Ciesla wrote:

I light of the above, I'm working on a build with no .jar files, and slightly reduced functionality.

gallery2-2.3-7.fc12 built in rawhide.  Spot, Kevin, thoughts?

gallery2-2.3-7.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/gallery2-2.3-7.fc11

gallery2-2.3-7.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/gallery2-2.3-7.fc10

Moving to -testing for 10 and 11.

gallery2-2.3-7.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update gallery2'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-3754

gallery2-2.3-7.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

I tried manualy installing the remote & upload applets, however uninstalling and deleting them again didn't get rid of the following error when running http://localhost/gallery2/upgrade:

[Sun Jun 07 13:46:59 2009] [error] [client 203.162.3.161] PHP Fatal error:  Class 'GalleryCoreApi' not found in /usr/share/gallery2/modules/core/classes/GallerySession.class on line 29

[root@augustus modules]# rpm -q gallery2
gallery2-2.3-12.fc10.noarch
[root@augustus modules]# rpm -V `rpm -qa gallery2`
S.5....T  c /etc/gallery2/config.php
SM5....T  c /usr/share/gallery2/.htaccess
[root@augustus modules]#

Is this a bug? or have I screwed something up?

Have you seen gallery2-2.3-module-cleanup.README?

Text:

A recent update of gallery2 removed 3 modules, which, if these had not been
 deactivated within gallery2 beforehand, caused errors in upgraded gallery2
installations.

To remove the remnants of the recently removed uploadapplet, 
slideshowapplet and remote modules, do the following:

In your gallery2 database:
delete from g2_PluginMap where g_pluginId = 'uploadapplet';
delete from g2_PluginPackageMap where g_pluginId = 'uploadapplet';
delete from g2_PluginParameterMap where g_pluginId = 'uploadapplet';
delete from g2_PluginMap where g_pluginId = 'slideshowapplet';
delete from g2_PluginPackageMap where g_pluginId = 'slideshowapplet';
delete from g2_PluginParameterMap where g_pluginId = 'slideshowapplet';
delete from g2_PluginMap where g_pluginId = 'remote';
delete from g2_PluginPackageMap where g_pluginId = 'remote';
delete from g2_PluginParameterMap where g_pluginId = 'remote';

Then, on the host with gallery2 installed, in your g2data directory:
rm -f g2data/cache/module/_all/0/0/*
rm -rf g2data/cache/module/uploadapplet
rm -rf g2data/cache/module/slideshowapplet
rm -rf g2data/cache/module/remote

This will restore any functionality not directly affected by the .jar 
removals, such as adding new items.

Doing the cleanup didn't fix it.

However, I added the following line to /usr/share/gallery2/upgrade/index.php, and it works fine now:
require_once($g2Base . 'modules/core/classes/GalleryCoreApi.class');

Are others able to laod the upgrade page?  Or is it just me that has this problem?

I haven't needed to, as I did back when I first rolled out 2.3.  Are you upgrading from 2.2.5?

This seems to have broken non-java use of the Remote API (e.g. the Digikam Gallery2 plugin we ship).

Is there not a reason to remove the .jar's specifically and keep the PHP?

Um, no, probably just that I was not able to fully suss out the interrelationship of the jars and the rest of the code.  Since adding the jars back in is a non-starter, should I just drop the Remote plugin?

That's basically what you're doing now and what's breaking Digikam syncing.

So I should drop the PHP portions and the entire subpackage, so the situation is clearer to the casual observer?

No! That's exactly what you're doing now and what the complaint is about.

What Bill McGonigle is asking you for is to READD a -remote subpackage with the PHP remote API which is used by Digikam and ONLY remove the .jar files, which are client-side applets and which are NOT needed to sync Gallery2 using Digikam. You're removing the whole remote plugin, which is mostly PHP code and which is required for Digikam to work, when you should be removing only the client-side applets.

And in the long run, you should talk to the Java folks so they can help you get this fixed properly, by tracking down the actual source code for those .jar files and building that from source. But this is not related to the request at hand, which is to just ship the PHP portions of the remote plugin so Digikam can sync again.

Right, so, to clarify, Gallery Remote is a server-side PHP-based feature without licensing problems.  It also happens to ship with a .jar sample client implementation, but most users (that I'm aware of anyway) use -remote with thick clients (Digikam, iPhoto, etc.).  I seem to recall some folks also use Remote for application integration (blogs, etc.).  

That included sample client has licensing and/or build problems, granted.  So, for now anyway, removing the .jar makes perfect sense, but whacking all of -remote is too big a hammer to drive that nail.

Despite having used Remote for years, I've never used the .jar, so I'm not sure - are there any links to the .jar in the Gallery GUI?  I'd be happy to help with a patch to hide those if that's the case.

I see, sorry, I misunderstood. Can you file a bug indicating which branches you need this done for?  I'll get on it in the meantime.

It should be fixed on all supported branches. We really want Digikam to work.

Sounds reasonable.  I just wasn't aware that it did without the jar.  I don't see any reference to it in the PHP code.

As far as the jars are concerned by the way, I initiated what became a lengthy discussion with upstream about this.  I tried to obtain the source for them, find out what the build process was etc.  What I found was that the build process is extremely complex, and that no one could tell me what SVN revision the jars in the release had been built from, or how to build them from source, since the jars are checked into SVN already build.  There's an ant build.xml, but no documentation on how to use it.  After banging my head against the wall for weeks, upstream basically said, look, we're focusing on Gallery3 at this point and don't really care to put much effort into resolving this for Gallery2.  They also said this wouldn't be an issue in Gallery3 which would be out soon.  This was nearly a year ago and it's still not out.

That said, if you want to give it a whirl, be my guest, but I think it will be somewhat quixotic.

Any movement on fixing this, i.e. shipping the PHP parts of Remote without the JARs so Digikam's Gallery2 upload works?

Created attachment 422721 [details]
gallery2-remote file listing

I forgot about this since I downgraded and locked yum... :P

Attaching the old file listing.  There only seems to be one reference to the jars, in a .inc file, GalleryRemoteWebStart.inc.  I think if we omit the jars and the .inc file and update the MANIFEST file we might do OK.  I didn't see an explicit reference to the .inc file anywhere, and it looks like it probably does extension through inheritance with extant files?  But I don't really understand the Gallery extension mechanism and would defer to those who do.

$ rpm -q --filesbypkg gallery2-remote | cut -f 12 -d ' ' | grep -v jar | xargs grep jar
/usr/share/gallery2/modules/remote/GalleryRemoteWebStart.inc:    <jar href="applets/GalleryRemote.jar"/>
/usr/share/gallery2/modules/remote/GalleryRemoteWebStart.inc:    <jar href="applets/img.jar"/>
/usr/share/gallery2/modules/remote/GalleryRemoteWebStart.inc:    <jar href="applets/metadata-extractor-2.1.jar"/>
/usr/share/gallery2/modules/remote/MANIFEST:modules/remote/applets/GalleryRemote.jar    6890469 6890469 705806  705806
/usr/share/gallery2/modules/remote/MANIFEST:modules/remote/applets/img.jar      4087949072      4087949072      19225   19225
/usr/share/gallery2/modules/remote/MANIFEST:modules/remote/applets/metadata-extractor-2.1.jar   1088430076      1088430076      69726   69726
/usr/share/gallery2/modules/remote/po/eu.po:"igotzeko nahiko da arrastratu eta jaregin egitearekin, irudiak igo baino lehen biratu eta "

$ rpm -q --filesbypkg gallery2-remote | cut -f 12 -d ' ' | grep -v jar | xargs grep GalleryRemoteWebStart.inc

I guess in the process of waiting for a bug to be filed, I forgot about it completely.  Real Life has been quite full of late.  I'll see if I can take a crack at it this week.

I have a local build done with remote added back in, still jarless, with the webstart.inc removed and the MANIFEST patched.  I don't use the remote module, if I put the SRPM up somewhere would someone like to test, or should I just build for F-13 updates-testing and rawhide?

Thanks, Jon.  I'd be happy to test for you.  My vm is still f11 for a short time, so the SRPM would be perfect.

http://zanoni.jcomserv.net/fedora/gallery2/

Let me know how it goes.  If it's good, I'll get out there.