Bug 520505

perl-JSON-2.15-5.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/perl-JSON-2.15-5.fc11

(In reply to comment #0)
> [...] a spurious dependency [...] coming from somewhere in t/* in docs.
> IMO the best fix here [...] would be to _not_ package test suite code
> at all.  

Sorry, but I cannot agree.  The best fix is to skip all %doc files when generating dependencies.  There are often some examples in the docs, and it is a mistake to base package requirements on these as well.
I hope to manage to get this to rpm during F-13 development.

> IMO the best fix here (and something that should be done by default in
> practically all perl packages) would be to _not_ package test suite code
> at all.

There are often good reasons to pack the testsuite as documentation; it is well explained by Chris Weyl in http://www.redhat.com/archives/fedora-perl-devel-list/2009-August/msg00351.html

The reported spurious dependencies are remeved in perl-JSON-2.15-5.fc12.

(In reply to comment #2)

> There are often good reasons to pack the testsuite as documentation;

I think this is the first time I've heard someone actually agree with Chris on this.  On the other hand, I know several people (including myself) have repeatedly stated their opinions that it's pointless or even actively harmful when done as a general packaging practice.  Well, I guess we'll end up needing a guideline for this sometime.

(In reply to comment #4)
> (In reply to comment #2)
> 
> > There are often good reasons to pack the testsuite as documentation;
> 
> I think this is the first time I've heard someone actually agree with Chris on
> this.  On the other hand, I know several people (including myself) have
> repeatedly stated their opinions that it's pointless or even actively harmful
> when done as a general packaging practice.  Well, I guess we'll end up needing
> a guideline for this sometime.  

Not to fan the flames, as it were, but I'm curious as to what you mean by "actively harmful"?  If possible, some concrete examples would be helpful to understand the claimed harm.

In the clarification of why it is harmful to add the test suite to the docs, please abstract from the fact that rpm currently searches for dependencies in %doc files.  That's a bug that need to be fixed.  (In package review, it is ensured that docs are not required for correct function; consequently, requirements of cocds cannot be necessary for proper function either.)

As said, it's been discussed before.  Bad API usage/coding examples; package size bloat; installing stuff upstream hasn't intended to be installed; potential for spurious dependency bloat (sorry that can't be left out as evidenced by this bug) - or if you eliminate these deps, the tests can't be run without manually managing them at which point it'd be better to download and use the source rpms for this purpose (they have the proper deps, Makefile.PL's etc infrastructure upstream intended for the test suite); kind of encourages bad practices such as mentioned by Chris in his mail referred to in comment 2 (installing packages in system locations and _then_ after the fact thinking about running test suites) etc.

Yes, there are exceptions and _sometimes_ packaging the test suites or parts of them might be a good idea.  It seems that test suites are packaged without much thought in Fedora packages these days though.  If it was a good idea in general, why wouldn't it be done in

As said, it's been discussed before.  Test suites often (usually?) contain undocumented and bad API usage/coding examples; package size bloat; installing stuff upstream hasn't intended to be installed; potential for spurious dependency bloat (sorry that can't be left out as evidenced by this bug) - or if you eliminate these deps, the tests can't be run without manually managing them at which point it'd be better to download and use the source rpms for this purpose (they have the proper deps, Makefile.PL's etc infrastructure upstream intended for the test suite and have less risk of littering system locations with generated test leftover data); kind of encourages bad practices such as mentioned by Chris in his mail referred to in comment 2 (installing packages in system locations and _then_ after the fact thinking about running test suites); if upstream docs aren't good enough it'd be better to let upstream know about that and ask them to improve things so more people would benefit; etc.

Yes, there are exceptions and _sometimes_ packaging the test suites or parts of them might be beneficial.  But I strongly think those cases are a rare exception and it seems that test suites are packaged without much thought in Fedora packages these days and FWIW I wouldn't personally consider approving any package that doesn't document a good rationale for including that stuff in the particular case of that package in question.  If it was a good idea in general, why wouldn't it be done in other packages besides perl ones, and why isn't it done even in all perl packages, and why isn't there a general guideline that encourages shipping that stuff in place or being pushed by people, and all that already in place for lots and lots of years of packaging history?

(In reply to comment #8)
> Yes, there are exceptions and _sometimes_ packaging the test suites or parts of
> them might be beneficial. [...] document a good rationale for including that stuff

Sounds fair, tests should not be packed by default.

But I'd like to comment some of your reasons:

> if upstream docs aren't good enough it'd be better to let upstream know about
> that and ask them to improve things so more people would benefit; etc.

Sure, it is necessary to report the problem, if upstream does not know or underestiomates its importance.  But then, packing the tests is a good temporary aid until sufficient documentation becomes available.

> [...] kind of encourages bad practices such as
> mentioned by Chris in his mail referred to in comment 2 (installing packages in
> system locations and _then_ after the fact thinking about running test suites);

Generally, tests can be run on uninstalled software, or after instalation.  Ideally, both kind of tests are performed; this can often be done by rerunning the same test suite.  (GNU Coding Standards define two make targets: "check" and "installcheck".)  rpm's %check does test uninnstalled software, but the spec files have no way to describe tests on installed program.

There are situations when the tests cannot be run from the spec file, e.g., because they require GUI or because they require a running database server.

You seem to imply that enabling install time checking somehow discourages proper use of build time testing.  I cannot agree with that.

If the tests cannot be (fully) performed at build time, it is useful to pack them, so that the user has chance to run them before deploying the module.

Again, this reason justifies packing tests only as a temporary aid, because http://fedoraproject.org/wiki/Features/FullyTestablePerl would render this type of reasoning irrelevant.

To sum up, if tests are packed, comment should contain a reason, e.g.:
- the tests show example usage, supplementing incomplete documentation
- the tests cannot be fully performed at build time because of xyxz

(In reply to comment #8)
> As said, it's been discussed before.  

So, again, do you have any real examples of "actual harm" specific to including the tests as documentation, or is this all handwaving and FUD?  If I remember correctly, there weren't any real examples the last time this came up, either...  And if I'm being accused of causing Fedora "actual harm", I'd at least like to see the evidence against me.


Dependency bloat (from %_docdir contents) is a long standing rpm issue that no one seems willing to tackle; by policy we don't allow deps in %_docdir so Fedora rpm shouldn't even look in there.  And frankly, I don't think it's unreasonable to say that I've done more recently to help eliminate %_docdir and other spurious deps by writing up and proposing the filtering system now in place than any other effort I've recently seen.  (Corrections welcome.)  If rpm conformed to Fedora policy, %_docdir deps wouldn't ever be an issue with any package.

Catalyst test suites often contain mini test-apps that help demonstrate how something actually works.  MooseX::AttributeHelpers was for ages poorly documented; the tests were the only real way to help figure out how they worked.  MooseX::Workers sounded really quite cool didn't make any sense until I read through the test cases.  If one does a basic google search[1] for 'site:search.cpan.org AND ("see the test suite" OR "see the tests")':  MP3::M3U::Parser ("See the tests in the distribution for example codes"), Parallel::Forker ("For more examples, see the tests"), MooseX::Types ("See the tests directory for a start on this", w.r.t. a specific coding scenario), WWW::Netflix::API ("Also see the "TEST SUITE" source code for more examples"), etc, etc.

As to "undocumented api"...  We all know the conventions, we all get to make our own decisions, and we all know we're shooting ourselves in the foot (or head) if we use internal methods.  If this were a real argument, we should probably not allow any users to see the source code at all.  Or have root. Or a keyboard. :)

I think the ability to run the test suite post-install is pretty important, but we're installing them in %_docdir.  ATM the intention is to have them as documentation; if someone uses them beyond that that's their own doing.  And I hardly think running a test suite "post-install" is a "bad practice"; modern Perl apps often have a fairly extensive dependency tree...  It's nice to be able to validate that updating perl-Sub-Name, say, isn't going to somehow break Class::MOP and thereby your Catalyst app.  Unless we start breaking them out into -tests subpackages, we don't need to consider this here, however, as they stand quite nicely on their own as docs.

Soo...  maybe the "undocumented" / "bad API" theories held some weight back in 1999, but this is 2009.  Modern Perl test suites are an entirely different animal.  I'm more than open to reconsidering the merits of this, but I haven't seen any of the "actual harm" claimed to be specific to including the tests in %_docdir.

[1] http://www.google.com/#hl=en&q=site%3Asearch.cpan.org+AND+("see+the+test+suite"+OR+"see+the+tests")&aq=f&aqi=&oq=&fp=3aa7f458acaa2672

(In reply to comment #10)
> is this all handwaving and FUD?

Thank you for so kindly considering differing opinions.

(In reply to comment #11)
> (In reply to comment #10)
> > is this all handwaving and FUD?
> 
> Thank you for so kindly considering differing opinions.  

As I said, I'm more than open to different opinions and the merits of these approaches.  You claimed I'd done "actual harm" to Fedora, effectively saying that your opinions are fact and empirically quantifiable.  I welcome and seek feedback, as this isn't dicta but rather an evolving process; but simply slamming the practice, citing "actual harm", and then claiming that "..a guideline will be needed" isn't very constructive.

How about, instead of bashing, we sit down and try to work out how to best do this -- that is, including the test suite in some form that satisfies both of us and benefits the community as a whole.

I'd like to see the evidence of any concrete harm done specific to including the test suites in %_docdir, but I haven't seen any yet, despite repeated requests.  If there is actually any harm caused here, I'd like to correct it.

(In reply to comment #12)

> You claimed I'd done "actual harm" to Fedora

No I didn't, please read comment 4 again (I guess that's what you're referring to).  But let's not get caught up with this or other word games.

> How about, instead of bashing, we sit down and try to work out how to best do
> this

I have plenty on my plate so no thanks for now.  But on a brief look, FullyTestablePerl looks like a good idea that would pretty much eliminate my concerns.  And Stepan's suggestion at end of comment 9 (+ some real thought from the packagers whether to ship the tests) would be an improvement in the meantime.

> I'd like to see the evidence of any concrete harm done specific to including
> the test suites in %_docdir, but I haven't seen any yet

The reasons given in comment 8 demonstrate why I think this stuff is harmful.  IMO it makes affected packages worse than they would be without those issues, which I think qualifies this practice as harmful.  You don't agree with it, but the "evidence" is there, and one (very insignificant) manifestation of it is this very bug.  Dismissing it as handwaving, FUD or nonexistent is a bit extreme.

I think it's about time we let this bug rest in peace and continue the discussion in another more appropriate medium in case someone still has something to say.

(In reply to comment #13)
> (In reply to comment #12)
> 
> > You claimed I'd done "actual harm" to Fedora
> 
> No I didn't, please read comment 4 again (I guess that's what you're referring
> to).  But let's not get caught up with this or other word games.

I don't think anyone is playing word games here.  I take it you're retracting your claim?

> > How about, instead of bashing, we sit down and try to work out how to best do
> > this
> 
> I have plenty on my plate so no thanks for now.  But on a brief look,
> FullyTestablePerl looks like a good idea that would pretty much eliminate my
> concerns.  And Stepan's suggestion at end of comment 9 (+ some real thought
> from the packagers whether to ship the tests) would be an improvement in the
> meantime.

Good. With perl now shipping /etc/rpm/macros.perl we have a good vehicle for delivery of any helper macros.

> > I'd like to see the evidence of any concrete harm done specific to including
> > the test suites in %_docdir, but I haven't seen any yet
> 
> The reasons given in comment 8 demonstrate why I think this stuff is harmful. 
> IMO it makes affected packages worse than they would be without those issues,
> which I think qualifies this practice as harmful.  You don't agree with it, but
> the "evidence" is there, and one (very insignificant) manifestation of it is
> this very bug.  Dismissing it as handwaving, FUD or nonexistent is a bit
> extreme.

If there's other "evidence", that is, something "actively harmful", I'd appreciate it if you could point it out, so I could at least own up to it and fix it.

> I think it's about time we let this bug rest in peace and continue the
> discussion in another more appropriate medium in case someone still has
> something to say.  

Fine by me :)

(In reply to comment #14)
> I take it you're retracting your claim?

I don't know what you're referring to.  This is nothing personal, and I haven't claimed you having done actual harm to Fedora; I simply don't know if you consistently follow this practice I find harmful and I'm not after anyone.  Comment 4 *is* what I think and I'm not retracting that (note "when done as a general packaging practice", and see also the 2nd paragraph of comment 8).  But I'm done with this discussion here.

perl-JSON-2.15-5.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.