Bug 537138

Summary: Bioperl doesn't bootstrap on other platforms, circular dependencies
Product: [Fedora] Fedora Reporter: Yaakov Nemoy <loupgaroublond>
Component: perl-bioperl-runAssignee: Alex Lancaster <alex>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 11CC: alex, bloch, kasal, perl-devel
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-06-28 15:27:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yaakov Nemoy 2009-11-12 16:26:21 UTC
Description of problem: perl-bioperl and perl-bioperl-run have circular build dependencies effectively

While this situation is not explicit it's based on runtime only requirements enforced by the packages. It makes it hard to bootstrap this on a system that doesn't already have an earlier version of bioperl.

perl-bioperl has a runtime requirement on a dependency that is fulfilled by perl-bioperl-run. perl-bioperl-run has a build time dependency on something provided by perl-bioperl. Without a perl-bioperl-run package, it's impossible to install bioperl to build a newer bioperl-run. This presents a problem when trying to bootstrap this is centos.

Version-Release number of selected component (if applicable):
perl-bioperl 1.6.1 (based of the 1.6.0 spec file with only a version change)
perl-bioperl-run 1.6.1-1

How reproducible:
I'm guessing it is based on theory, i haven't tried to reproduce it on a second machine.

Steps to Reproduce:
1. Build a perl-bioperl package on enterprise linux 5
2. Add it to a repo your system can access.
3. Try building a perl-bioperl-run package starting with rpmbuild -bs on the .spec file.
4. Try installing perl-bioperl to fill the BR
  
Actual results:

[root@bofh016 SPECS]# rpmbuild -bs perl-bioperl-run.spec
error: Failed build dependencies:
        perl(Bio::Root::Version) >= 1.5.9 is needed by perl-bioperl-run-1.6.1-1.noarch

[root@bofh016 SPECS]# yum install perl-bioperl-1.6.1-2.el5.noarch
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * addons: archive.cs.uu.nl
 * base: archive.cs.uu.nl
 * extras: archive.cs.uu.nl
 * updates: archive.cs.uu.nl
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package perl-bioperl.noarch 0:1.6.1-2.el5 set to be updated
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtPars) for package: perl-bioperl
--> Processing Dependency: perl(List::MoreUtils) for package: perl-bioperl
--> Processing Dependency: perl(Test::Warn) for package: perl-bioperl
--> Processing Dependency: perl(Test::Exception) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Alignment::Clustalw) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Molphy::ProtML) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtDist) for package: perl-bioperl
--> Processing Dependency: perl(Bio::ASN1::EntrezGene) for package: perl-bioperl
--> Processing Dependency: perl(XML::Simple) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::Neighbor) for package: perl-bioperl
--> Running transaction check
---> Package perl-List-MoreUtils.x86_64 0:0.22-2.el5 set to be updated
---> Package perl-Test-Exception.noarch 0:0.26-2.el5 set to be updated
--> Processing Dependency: perl(Sub::Uplevel) for package: perl-Test-Exception
---> Package perl-Test-Warn.noarch 0:0.11-2.el5 set to be updated
---> Package perl-XML-Simple.noarch 0:2.14-4.fc6 set to be updated
---> Package perl-bioperl.noarch 0:1.6.1-2.el5 set to be updated
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtPars) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Alignment::Clustalw) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Molphy::ProtML) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtDist) for package: perl-bioperl
--> Processing Dependency: perl(Bio::ASN1::EntrezGene) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::Neighbor) for package: perl-bioperl
--> Running transaction check
---> Package perl-Sub-Uplevel.noarch 0:0.18-2.el5 set to be updated
---> Package perl-bioperl.noarch 0:1.6.1-2.el5 set to be updated
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtPars) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Alignment::Clustalw) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Molphy::ProtML) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtDist) for package: perl-bioperl
--> Processing Dependency: perl(Bio::ASN1::EntrezGene) for package: perl-bioperl
--> Processing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::Neighbor) for package: perl-bioperl
--> Finished Dependency Resolution
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::Neighbor) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtPars) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtDist) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::Tools::Run::Alignment::Clustalw) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::ASN1::EntrezGene) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
perl-bioperl-1.6.1-2.el5.noarch from centos-5-genomics has depsolving problems
  --> Missing Dependency: perl(Bio::Tools::Run::Phylo::Molphy::ProtML) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::ASN1::EntrezGene) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::Tools::Run::Alignment::Clustalw) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::Neighbor) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::Tools::Run::Phylo::Molphy::ProtML) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtDist) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
Error: Missing Dependency: perl(Bio::Tools::Run::Phylo::Phylip::ProtPars) is needed by package perl-bioperl-1.6.1-2.el5.noarch (centos-5-genomics)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest

Expected results:

A source package

Additional info:

I'm going to push a fedora package to our private repo to make this work, but i'm wondering how this could be solved for EPEL in the future. Ideally we would like to run bioperl out of EPEL directly rather than running our own private builds.

Comment 1 Stepan Kasal 2009-11-12 18:44:55 UTC
Thanks for the report.

I think that it makes sense to filter the perl-bioperl requires so that perl-bioperl-run is not required by them.  (Otherwise the two packages could probably be merged.)

I'm going to fix this in devel; I doubt it is worth to issue a F11 update for this bug.

Comment 2 Alex Lancaster 2009-11-12 20:42:36 UTC
If you look at the .spec file:

http://cvs.fedoraproject.org/viewvc/rpms/perl-bioperl/devel/perl-
bioperl.spec?revision=1.23&view=markup

you'll notice that we have:

#cat << \EOF > %{_builddir}/BioPerl-%{version}/%{name}-req
##!/bin/sh
#%{__perl_requires} $* |\
#   sed -e '/perl(Bio::Tools::Run::Alignment::Clustalw)/d; /perl(Bio::Tools::Run::GenericParameters)/d; /perl(Bio::Tools::Run::Phylo::Molphy::ProtML)/d; /perl(Bio::Tools::Run::Phylo::Phylip::Neighbor)/d; /perl(Bio::Tools::Run::Phylo::Phylip::ProtDist)/d; /perl(Bio::Tools::Run::Phylo::Phylip::ProtPars)/d; /perl(Bio::Tools::Run::RemoteBlast)/d'
#EOF
 
#%%define __perl_requires %{_builddir}/BioPerl-%{version}/%{name}-req
#chmod +x %{__perl_requires}

to handle the bootstrapping problem.  So simply enable those sections of the spec on first build, then build perl-bioperl-run against them, then comment them out again.  Not pretty but it works (more requires may need to be filtered since new requires may have been added since the first bootstrap).  

(In reply to comment #1)
> Thanks for the report.
> 
> I think that it makes sense to filter the perl-bioperl requires so that
> perl-bioperl-run is not required by them. 

Right and that's what the above does, although I'm not sure if that's a good idea to permanently filter them.

> (Otherwise the two packages could  probably be merged.)

No that should not be done, because they are two tarballs maintained separately with different release schedules upstream.  I have already discussed this problem with upstream and if anything they are moving in the direction of more modularisation of bioperl into separate packages than less.

> I'm going to fix this in devel; I doubt it is worth to issue a F11 update for
> this bug.

Comment 3 Yaakov Nemoy 2009-11-12 20:53:57 UTC
Sure, if you do it in devel, i'm just going to backport it for our environment anyways. Once we have this working, we can always use any patches i figure out to deploy the same to el5 if you're interested.  

I'll give the commenting trick a stab at work tomorrow morning. I tried forcing the install of the fedora package for perl-bioperl-run but it's got a dependency on perl 5.10 which doesn't help the situation much. I was debating doing the same anyways.

Comment 4 Alex Lancaster 2009-11-12 21:02:56 UTC
(In reply to comment #3)
> Sure, if you do it in devel, i'm just going to backport it for our environment
> anyways. Once we have this working, we can always use any patches i figure out
> to deploy the same to el5 if you're interested.  

Thanks!  

PS.  I'm really glad somebody is interested in getting this ported to EPEL as others have been asking me whether they could get bioperl packaged on RHEL/EPEL/Centos, but I don't have the time, nor do I run any RHEL/Centos machines.  Will this mean you will submit/maintain the package for EPEL?

Also are all the other perl deps (e.g. perl-Graph etc.) now available for EPEL?  Creating and maintaining all those extra EPEL branches was another reason I wasn't able to take on the task myself (although I am willing to be a co-maintainer)

> I'll give the commenting trick a stab at work tomorrow morning. I tried forcing
> the install of the fedora package for perl-bioperl-run but it's got a
> dependency on perl 5.10 which doesn't help the situation much. I was debating
> doing the same anyways.  

OK, no patch has been yet committed for devel (which means F-13 as F-12 is now taking updates), although any improvements on my hack in the spec file that you make will be happily filtered back to Fedora.

Comment 5 Alex Lancaster 2009-11-12 21:06:01 UTC
You might also note that all active branches (rawhide, F-12, F-11) except F-10 which will be EOLed soon have moved to bioperl 1.6.1 in CVS and builds are either now in rawhide or pending for F-12 and F-11, so you may want to make your diffs/patches against that .spec not the 1.6.0 spec for perl-bioperl.

Comment 6 Yaakov Nemoy 2009-11-12 21:23:53 UTC
Alot of packages are not, i spent half the day methodically rebuilding a bunch of packages that were missing or needed newer versions. I haven't figured out if all of them are epel, though i think some of them, including libxml2 are core packages and it might not be possible to update them without difficulty.

At $dayjob we've decided to stop using cpan to install modules and to focus our efforts on building RPMs where possible. As long as i can do it within the infrastructure of Fedora/EPEL, this will be ok, and i am more than willing to take up any branches that we need to support EPEL so long i'm working at this job. Presumably if i leave, this duty will be picked up by my successor.  Comaintanership is always welcome.

I'll have a look at the CVS packages tommorow then. All i did was update the Version field anyways, and i'm not 100% sure the package is going to work until after i run it on our test systems and ask the researchers to break it thoroughly.

Comment 7 Alex Lancaster 2009-11-12 21:42:15 UTC
(In reply to comment #6)

> I'll have a look at the CVS packages tommorow then. All i did was update the
> Version field anyways, and i'm not 100% sure the package is going to work until
> after i run it on our test systems and ask the researchers to break it
> thoroughly.  

Right, and a lot of the Fedora perl- packages that I maintain in order to get bioperl working need to be updated to the latest CPAN versions, but I've gotten a bit behind lately.  Also harder to keep track of this since Chris Weyl's nice upstream tracker for perl packages went offline:

http://perl.biggerontheinside.net/

Comment 8 Alex Lancaster 2009-11-12 21:44:39 UTC
If you happen to update the .spec for EPEL/RHEL for any Perl packages that are out of date with respect to upstream that I own/maintain, see my list here:

https://fedoraproject.org/wiki/User:Alexlan

please send me the patches to the .spec and source files and I'll do new updates, or just check them into CVS devel branch (if you have access) and I'll do the builds for rawhide and the rest of the branches.

Comment 9 Alex Lancaster 2009-11-12 21:49:02 UTC
(In reply to comment #6)
> Alot of packages are not, i spent half the day methodically rebuilding a bunch
> of packages that were missing or needed newer versions. I haven't figured out
> if all of them are epel, though i think some of them, including libxml2 are
> core packages and it might not be possible to update them without difficulty.

Does libxml2 need to be updated for running bioperl?  Is it that old?

> At $dayjob we've decided to stop using cpan to install modules and to focus our
> efforts on building RPMs where possible. As long as i can do it within the
> infrastructure of Fedora/EPEL, this will be ok, and i am more than willing to
> take up any branches that we need to support EPEL so long i'm working at this
> job. Presumably if i leave, this duty will be picked up by my successor. 

Great!

> Comaintanership is always welcome.

Right, and actually I request that I be added as a co-maintainer of any EPEL branches of packages I own in Fedora proper, see (FAS username "alexlan"):

https://fedoraproject.org/wiki/EPEL/ContributorStatusNo

Comment 10 Yaakov Nemoy 2009-11-12 21:54:20 UTC
I did have to update libxml2. I haven't tested it on all our machines to see if it breaks stuff. It definitely bears looking in to. The reason why i updated it is because the RPM required it, not because of any tests of fails. I was planing on experimenting with a rebuild with a modified RPM to see if it would still work.

Comment 11 Yaakov Nemoy 2009-11-13 12:51:16 UTC
Follows is a list of the packages we had to rebuild to make this work. In almost all cases we had to do a simple rebuild for epel. 'umc' means that we applied our own patches. In this particular case, one of the tests was failing and i need to attach an appropriate bug report. I googled the error and it appears it might be a regression from a bug that was fixed in an earlier version, but i'm not 100% what it is.

On that note, is there anything you fundamentally need todo to the spec file to once and for all solve the bootstrap problem? If not, uncommenting and then commenting code seems to work, and i feel this bug can be closed. Comments?

 perl-bioperl                 noarch  1.6.1-1.el5      centos-5-genomics  6.2 M
 libxml2                      x86_64  2.7.6-1.el5      centos-5-genomics  854 k
 libxml2-python               x86_64  2.7.6-1.el5      centos-5-genomics  519 k
 libxslt                      x86_64  1.1.26-1.el5     centos-5-genomics  546 k
 perl-Ace                     noarch  1.92-2.el5       centos-5-genomics  310 k
 perl-Bio-ASN1-EntrezGene     noarch  1.091-7.el5      centos-5-genomics   41 k
 perl-Convert-Binary-C        x86_64  0.74-1.el5       centos-5-genomics  294 k
 perl-Data-Stag               noarch  0.11-2.el5       centos-5-genomics  185 k
 perl-Math-Derivative         noarch  0.01-3.el5       centos-5-genomics  7.3 k
 perl-Math-Spline             noarch  0.01-2.el5       centos-5-genomics  8.4 k
 perl-PostScript              noarch  0.06-2.el5       centos-5-genomics   31 k
 perl-SVG                     noarch  2.44-1.el5       centos-5-genomics   82 k
 perl-SVG-Graph               noarch  0.02-1.el5       centos-5-genomics   96 k
 perl-Set-Scalar              noarch  1.23-2.el5       centos-5-genomics   38 k
 perl-XML-DOM-XPath           noarch  0.14-1.el5       centos-5-genomics   11 k
 perl-XML-LibXSLT             x86_64  1.68-3.el5       centos-5-genomics   54 k
 perl-XML-SAX                 noarch  0.96-2.el5       centos-5-genomics   78 k
 perl-XML-XPathEngine         noarch  0.11-1.el5       centos-5-genomics   40 k
 perl-umc-XML-LibXML          x86_64  1:1.69-3.el5     centos-5-genomics  375 k

Comment 12 Stepan Kasal 2009-11-17 16:52:54 UTC
Hello Alex,

(In reply to comment #2)
> although I'm not sure if that's a good
> idea to permanently filter them.

You are right, this circular dependency makes sense, let's keep the filtering commented out in the spec.

> [...] Not pretty but it works

I took the liberty to convert this to the new filtering rpm macros; with that, it looks pretty elegant.  (Tested, but then commented out again.)

> (more requires may need to be filtered
> since new requires may have been added since the first bootstrap).

Actually, it seems that filtering out Bio::Tools::Run::* is enough; in any case, the comment shows the way.

I think this bug could be closed as a notabug now.

Comment 13 Bug Zapper 2010-04-28 11:17:58 UTC
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 14 Bug Zapper 2010-06-28 15:27:17 UTC
Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.