Bug 591564

+1

It's highly irritating to have to jump through hoops in order to get an updated CGI.pm (for example) installed that :
a) Is in the @INC path
b) Doesn't conflict clobber with existing files / rpms
c) Won't get clobbered when the Perl RPM is updated

If the Perl RPM package contained only Perl, and no CPAN modules, then each module could be packaged individually, making upgrade (and dependency) paths much cleaner.

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

(In reply to comment #0)
> Description of problem:
> 
> With RHEL5, several of us attempting to package third-party software have had
> issues with the perl
> RPM including some basic modules, e.g., CGI.pm.
> 
> The specific example I am working with is sympa, available at www.sympa.org
> 
> like most any software written in perl, sympa has vast CPAN requirements.
> 
> in particular, sympa wants CGI.pm version 3.35, but RHEL5's perl RPM provides
> version 3.15.
> 
> If perl-cgi were broken out into its own RPM, we could roll our own RPM with
> the latest version, have it as a dependency, and those who want to install our
> software, but not muck with the base OS perl, can easily get the piece they
> want.    

Sub-packages in perl package were created on users request. That's the way how we choose which will be in sub-package. 

Do you want only CGI? (upstream http://search.cpan.org/~lds/CGI.pm-3.49/)
I can hardly broke perl into sub-packages for every module included in perl because it takes a lot of time to find all dependency problems between sub-packages.

(In reply to comment #5)

> Sub-packages in perl package were created on users request. That's the way how
> we choose which will be in sub-package. 
> 
> Do you want only CGI? (upstream http://search.cpan.org/~lds/CGI.pm-3.49/)
> I can hardly broke perl into sub-packages for every module included in perl
> because it takes a lot of time to find all dependency problems between
> sub-packages.    

While i can't speak for Dan Pritts, it is my opinion that, yes, all of the modules should be taken out of the Perl RPM and packaged separately.  If that's not realistic, well, i suppose it can't be done, but it'd be the optimal scenario.  The CGI module is just a convenient example, since it's the one that (imho) people run into most often.

Basically what it comes down to is this : one of the most important aspects of running an RHEL system is maintaining package coherency, and if you end up using interactive CPAN (for example) to upgrade Perl modules, then you break that coherency.  Not only that, but then you have to live in fear of a Perl package upgrade, since it might break everything you have set up - that's clearly not a sustainable situation.

(In reply to comment #6)

> While i can't speak for Dan Pritts, it is my opinion that, yes, all of the
> modules should be taken out of the Perl RPM and packaged separately.  If that's
> not realistic, well, i suppose it can't be done, but it'd be the optimal
> scenario.  The CGI module is just a convenient example, since it's the one that
> (imho) people run into most often.
> 

IMHO it would be un-maintainable. Also it's late for bigger changes in rpm in beta. No one from Fedora developers and users didn't ask for such granularity, so I left it as it's now. One or two sub-packages should be possible to add. So all of you are asking only perl-CGI at the moment?

> Basically what it comes down to is this : one of the most important aspects of
> running an RHEL system is maintaining package coherency, and if you end up
> using interactive CPAN (for example) to upgrade Perl modules, then you break
> that coherency.  Not only that, but then you have to live in fear of a Perl
> package upgrade, since it might break everything you have set up - that's
> clearly not a sustainable situation.    

I understand your problems, but every update of internal perl module is also inconvenient for us. Modules are sometimes completely rewritten and it's hard or impossible to update them after a few years.

Well, i suppose that unrolling CGI would be better than nothing. :)

Thanks for taking the time to consider our requests

(In reply to comment #7)
> IMHO it would be un-maintainable. 

I understand that it's more work, but I think debian does something along these lines.

> Also it's late for bigger changes in rpm in
> beta. No one from Fedora developers and users didn't ask for such granularity,

Fedora's limited life cycle means that you have to upgrade everything pretty regularly, so
you pick up bug fixes and new features from upstream pretty regularly.

RHEL's longer life cycle means that does not happen, only critical bugs are backported.
Which is fine; we like RHEL's package stability.  Sometimes though, we need to make our
own updates to portions of the distribution, and take the risks involved.

> I understand your problems, but every update of internal perl module is also
> inconvenient for us. Modules are sometimes completely rewritten and it's hard
> or impossible to update them after a few years.    

Yes, we understand that you would not release feature updates to these RPMs.  

> so I left it as it's now. One or two sub-packages should be possible to add. So
> all of you are asking only perl-CGI at the moment?

In the particular case of sympa on RHEL5, we need to replace Time::HiRes also.

Perhaps a better way to approach a partial unroll though is to figure out what the 
change rate of the bundled modules is, and unroll the ones with a higher change rate.  

I don't have a good feel for that, but a quick look of the ext/ directory of the perl
tarball suggests that IO::COmpress and Time::Piece are relatively quickly developed.
I ran out of steam when i went to look at the lib/ directory of the tarball, which
probably has the bulk of the modules.

Dan, do you have any clue?

Finally, before we go too far down this road, maybe we should rethink the best way to achieve
our goal.  I would prefer just unrolling everything but I understand that is not very likely.

Dan wrote a blog post (which i can't find right now) about  rolling RPMs that fit into the site_perl area; 
the only real problem he has had is conflicting man pages.  Maybe we can work out a MANPATH 
for perl to fix this problem?

Using site_perl is fine for local use but not appropriate IMO for a third-party repository to distribute.

There is also a vendor_perl area, which sure seems like it should be reserved for Red Hat,
although it is not clear to me that Red Hat actually uses it for anything.  It would definitely not fix
all the concerns, but a statement from Red Hat that you are not and will not use vendor_perl,
and the provision of some sort of perl MANPATH, would help a lot.

The aforementioned blog post :
http://www.dark.ca/2010/04/08/cpan-rpms-in-rhel-centos/

It seems to me that vendor_perl would actually be the perfect place for Red Hat to put their modules, frankly.  The @INC order goes site > vendor > perl, which makes a lot of sense assuming that's the way it gets used.

As for the man pages ; frankly stated, those conflicts wouldn't exist if the modules weren't included in the base Perl RPM to begin with...

> It seems to me that vendor_perl would actually be the perfect place for Red Hat
> to put their modules, frankly.  The @INC order goes site > vendor > perl, which
> makes a lot of sense assuming that's the way it gets used.

Perhaps adding a fourth directory in the compiled-in @INC.  thirdparty or something like that,

site > thirdparty > vendor > perl

> As for the man pages ; frankly stated, those conflicts wouldn't exist if the
> modules weren't included in the base Perl RPM to begin with...    

Clearly.  Just trying to think creatively about how to solve our concerns in a way that is likely to actually happen.

(In reply to comment #10)

> There is also a vendor_perl area, which sure seems like it should be reserved
> for Red Hat,
> although it is not clear to me that Red Hat actually uses it for anything.  It
> would definitely not fix
> all the concerns, but a statement from Red Hat that you are not and will not
> use vendor_perl,
> and the provision of some sort of perl MANPATH, would help a lot.    

After chewing on this for a while, i'd have to agree that an official statement from Red Hat concerning how and where Perl modules (and man pages, etc..) are packaged and installed would be a great idea.  This would allow the community at large to respond to the issue of rolling our own updates in a coherent, standardised way.

For example, if the community knows that module XYZ will always be installed in a given location, and the supporting binaries in another (and that this standard will be rigorously adhered to), then a third-party update to said module could be generated and distributed as necessary without having to live in fear of an eventual vendor update.

(In reply to comment #10)
> 
> In the particular case of sympa on RHEL5, we need to replace Time::HiRes also.
> 
> Perhaps a better way to approach a partial unroll though is to figure out what
> the 
> change rate of the bundled modules is, and unroll the ones with a higher change
> rate.  
> 
> I don't have a good feel for that, but a quick look of the ext/ directory of
> the perl
> tarball suggests that IO::COmpress and Time::Piece are relatively quickly
> developed.
> I ran out of steam when i went to look at the lib/ directory of the tarball,
> which
> probably has the bulk of the modules.
> 
Time::Piece is already a sub-package. Creating Time::HiRes sub-package was easy, but IO::Compress is complicated. Some modules were merged into this one and it changed provides/requires. In Fedora it was solved by creating dual-lived modules (one in main perl package, second in separate cvs), but that's probably not what we want in RHEL. Also this process was started few months ago and it wasn't tested much.

At the moment the @INC are:
site:    /usr/local/lib64/perl5
site:    /usr/local/share/perl5
duplicate:    /usr/local/share/perl5
vendor:    /usr/lib64/perl5
vendor:    /usr/share/perl5
duplicate:    /usr/share/perl5
perl:    /usr/lib64/perl5
perl:    /usr/share/perl5
compatibility directories:   
/usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.10.0
    /usr/lib64/perl5/vendor_perl/5.10.0/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.10.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/site_perl

I assume installation of cpan modules should go to "site". "vendor" and "perl" were merged, because it was believed that shorter paths will be clear and easy to maintain. We are installing RPMs into /usr/lib|share/perl5. Man pages are installed in the same place as all man pages into /usr/share/man/man?/. 

It would be probable change "vendor" back to /usr/lib|share/perl5/vendor or add some third party directory. But I'm not sure if everyone will be happy with this solution. Don't you add your own directory into INC?

btw I'll remove duplicates of paths.

(In reply to comment #11)
> The aforementioned blog post :
> http://www.dark.ca/2010/04/08/cpan-rpms-in-rhel-centos/
> 
> It seems to me that vendor_perl would actually be the perfect place for Red Hat
> to put their modules, frankly.  The @INC order goes site > vendor > perl, which
> makes a lot of sense assuming that's the way it gets used.
> 
> As for the man pages ; frankly stated, those conflicts wouldn't exist if the
> modules weren't included in the base Perl RPM to begin with...    

Not everyone would agree with you...
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2009-08/msg00406.html

In short in Fedora-10 was shipped perl as interpreter and not as whole tar-ball, which didn't make upstream happy. Perl should have the same rpm as the upstream tar-ball or it shouldn't have the same name... Also some admins use perl as it is and they would probably disagree with splitting modules.

The main part of spec file was written for Fedora and discussed with community of users on mailing list perl-sig (perl-devel.org). Also new release of perl-5.12 is discussed there. IMHO this is the place when you can have quick impact on how will be rpms done. Although quite late for RHEL-6.

(In reply to comment #12)
> > It seems to me that vendor_perl would actually be the perfect place for Red Hat
> > to put their modules, frankly.  The @INC order goes site > vendor > perl, which
> > makes a lot of sense assuming that's the way it gets used.
> 
> Perhaps adding a fourth directory in the compiled-in @INC.  thirdparty or
> something like that,
> 
> site > thirdparty > vendor > perl
> 
If you have cpan module in site, your rpm in thirdparty and RHEL rpm in perl, then you have still a problem. Wouldn't be really better to create your own directory if you want install your rpms? Also if you are using your rpms, then you can install them into site, because you are not using cpan.

(In reply to comment #17)
> > Perhaps adding a fourth directory in the compiled-in @INC.  thirdparty or
> > something like that,
> > 
> > site > thirdparty > vendor > perl
> > 
> If you have cpan module in site, your rpm in thirdparty and RHEL rpm in perl,
> then you have still a problem. Wouldn't be really better to create your own
> directory if you want install your rpms? Also if you are using your rpms, then
> you can install them into site, because you are not using cpan.    

Bottom line, third-party repositories need a reasonable way to provide software, including replacing parts of the OS, without mucking up too much of the RHEL base.

If this is done by unrolling all the modules out of the perl RPM, so third parties can replace them individually, great.  

If this is done by telling third parties they can use vendor_perl, and that Red Hat won't be using vendor_perl,  that may not be as good but i think it should work fine.

third-party repos should NOT be using site_perl, that as you say is for the local site, whether they use CPAN or roll their own RPMs.

I'm sure this is a pain in your butt, but please remember that the third-party repositories add a tremendous amount of value to the RHEL ecosystem, and ask Red Hat for very little in return.  I am primarily a (paying customer) user/sysadmin of RHEL, not a third-party packager, and RPMForge has been a godsend.

thanks for your help!

> The main part of spec file was written for Fedora and discussed with community
> of users on mailing list perl-sig (perl-devel.org). Also

I will join this list.  

It's an interesting disconnect - I know that Red Hat considers Fedora the proving ground for RHEL, but at a fundamental level, due to Fedora's short life-cycle, it does not address the real-world concerns of users who want a stable OS (RHEL) but might need to update a little bit here or there to get "the latest stuff."

(In reply to comment #15)
> It would be probable change "vendor" back to /usr/lib|share/perl5/vendor or add
> some third party directory. But I'm not sure if everyone will be happy with
> this solution. Don't you add your own directory into INC?

I assume you mean adding my own directory into @INC in my own perl code, not some system-wide mechanism for adding INC directories that I am unaware of.

The answer is I often do, because i'm forced to.  It often means modifying a bunch of code that i've downloaded somewhere, which as you understand is a maintenance headache.  modifying the PERL5LIB environment variable is not always an appropriate solution (e.g., it is difficult to get environment variables passed to FastCGI scripts - it may be possible but i couldn't figure it out).

(In reply to comment #19)
> > The main part of spec file was written for Fedora and discussed with community
> > of users on mailing list perl-sig (perl-devel.org). Also
> 
> I will join this list.  
> 
> It's an interesting disconnect - I know that Red Hat considers Fedora the
> proving ground for RHEL, but at a fundamental level, due to Fedora's short
> life-cycle, it does not address the real-world concerns of users who want a
> stable OS (RHEL) but might need to update a little bit here or there to get
> "the latest stuff."    

It's not optimal solution, but specfile is written for a long time and testing takes also some time. Sure, you could file a bug as future feature, but then it should contain direct specification and communication in this case seems to me better.

I create CGI and Time::HiRes. I hadn't time for more. Also path for 3rd party rpms is in vendor. We don't use it. I'll try to write some documentation about it.

In case something is missing, let me know.

Thank you for your ongoing interest in this matter.  It's nice to know that legitimate user suggestions can (and do) affect the distribution.

Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.