Bug 441110

And has there been an attempt to actually get these changes into upstream RPM?

AFAIK, this is now upstream already. Refer

https://lists.dulug.duke.edu/pipermail/rpm-maint/2008-April/000872.html

That's lzma compressed sources (ie Source0: foo.tar.lzma or whatnot), not lzma
payloads.

Support for LZMA-payloads is certainly going to go into upstream sooner or
later, but it's not there yet. I'm waiting for the next major version of
lzma-utils with a usable library to be released, I'm not too keen on carrying an
lzma-decoder within rpm (which is how Mandriva does it)

Suse seems to be doing the right thing (imo) here: they've added the lzma
payload support by developing against current alpha-version of liblzma. The
patch looks fairly complete and sane, I'm expecting to forward-port it to
rpm.org HEAD soonish.

Support for LZMA payloads is now in rpm.org HEAD, forward-ported by Jindrich.

Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Back to Rawhide.

Compression and decompressing scales almost perfectly on multi core cpu's, so
perhaps it would make sense to first make liblzma multi threaded before changing
to lzma for rpm?

The new rpm in rawhide has support for LZMA payloads, but only when built
against lzma-utils >= lzma-4.42.2alpha, whereas rawhide is still on 4.32.x which
is the stable lzma branch.

From this point on, enabling lzma compression of payloads is a distro, not
rpm-level, if and when question (alpha release/snapshot of lzma libs could be
separately packaged for rpm needs, which is what SuSE did).

FWIW, I don't see this as being useful to add until any release that we support
upgrading from also supports LZMA payloads.

Panu - how backportable is the support?

(In reply to comment #11)
> FWIW, I don't see this as being useful to add until any release that we support
> upgrading from also supports LZMA payloads.
FWIW: Fedora's rpm lacking lzma payloads disqualifies Fedora's mock from being
usable as a build host for SUSE-packages.

The LZMA payload support doesn't need much in the way of backporting as SuSE
developed it on top of their patched 4.4.2 rpm and we forward-ported in to
rpm.org HEAD. 

Even so, my suggestion would be considering moving to LZMA payloads when F11
development starts:
- With a little bit of luck, a stable version of new lzma utils might be out by
then. If not, we need to consider if we want to base such a critical piece on an
unreleased version.
- LZMA support for F10 rpm can be easily enabled at any time by just rebuilding,
only depends on the above case. So the reasonable upgrade path would be there,
or at least could be easily arranged.

Sad, but true. apparently, not much progress on this front.

FC10's rpm, yum and mock are as unusable for building suse-packages as FC9's rpm, yum and mock had been.

Suse made a gamble with unstable lzma-utils and they're going to pay for it. lzma-utils upstream just recently changed the file format (again), and it wont even read the files created by the intermediate alpha version Suse uses:

[pmatilai@localhost tmp]$ lzma --version|head -1
lzma (LZMA Utils) 4.42.3alpha
[pmatilai@localhost tmp]$ lzma rpm-4.6.0-rc1.tar 
[pmatilai@localhost tmp]$ ~/repos/lzma-utils/src/lzma/lzma --version|head -1
lzma (LZMA Utils) 4.999.6alpha
[pmatilai@localhost tmp]$ ~/repos/lzma-utils/src/lzma/lzma --test rpm-4.6.0-rc1.tar.lzma 
/home/pmatilai/repos/lzma-utils/src/lzma/lzma: rpm-4.6.0-rc1.tar.lzma: File format not recognized

Better wait until a stable release...

This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

setting it back

It seems the xz-enabled RH-rpm is able process SUSE rpms with lzma payload.

No idea yet, if it processes them correctly.

(In reply to comment #18)
> It seems the xz-enabled RH-rpm is able process SUSE rpms with lzma payload.
> 
> No idea yet, if it processes them correctly.  

It's supposed to be compatible with Suse (and Mandriva), both of which use the "old" lzma-alone format for payload compression. File separate bugs if you find issues.

All of rawhide has been built with XZ (LZMA successor format) payload compression now, considering this done.

(In reply to comment #19)
> (In reply to comment #18)
> > It seems the xz-enabled RH-rpm is able process SUSE rpms with lzma payload.
> > 
> > No idea yet, if it processes them correctly.  
> 
> It's supposed to be compatible with Suse (and Mandriva), both of which use the
> "old" lzma-alone format for payload compression. File separate bugs if you find
> issues.
Well, I am facing bizarre dependency issues when trying to use suse repositories.

Either their EVR comparison operations work entirely different than Red Hat's, their repos are incompatible to Red Hat's yum or their yum repos are entirely messed up.

E.g.
mock -r suse-11.0-x86_64 init
...
 # /usr/bin/yum --installroot /var/lib/mock/suse-11.0-x86_64/root/  update
perl-5.10.0-37.8.x86_64 from installed has depsolving problems
  --> Missing Dependency: perl-base = 5.10.0 is needed by package perl-5.10.0-37.8.x86_64 (installed)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.
Error: Missing Dependency: perl-base = 5.10.0 is needed by package perl-5.10.0-37.8.x86_64 (installed)


# rpm -q --provides -p /var/cache/mock/suse-11.1-x86_64/yum_cache/base/packages/perl-base-5.10.0-62.16.x86_64.rpm | grep perl-base
perl-base = 5.10.0-62.16

# rpm -q --requires -p /var/cache/mock/suse-11.1-x86_64/yum_cache/update/packages/perl-5.10.0-62.18.1.x86_64.rpm | grep perl-base
perl-base = 5.10.0

So, I am inclined to think suse might be treating 
Requires: %version as %version >= %version-"" (release=empty),
while Red Hat seems to be treating 
%version as exact string match.

> So, I am inclined to think suse might be treating 
> Requires: %version as %version >= %version-"" (release=empty),

There, >= would make no sense if the spec says '='.

RPM in Red Hat/Fedora satisfies "Requires: %name = %version" with exact %version but any %release. 

[Further, since %release is not part of %version, something like 5.10.0-62.16 is not "higher than" 5.10.0 - but that's a mistake only done by packaging newbies]

(In reply to comment #21)
> RPM in Red Hat/Fedora satisfies "Requires: %name = %version" with exact
> %version but any %release. 
I am observing exactly the converse:

Red Hat rpm/yum/mock fails with dependency resolution issue on SuSE repos because SUSE-rpms provide "V-R", while some of their packages require "V" (without %release)

Example (here suse-11.1):

# rpm -q --provides -p /var/cache/mock/suse-11.1-x86_64-rtems/yum_cache/base/packages/perl-base-5.10.0-62.16.x86_64.rpm | grep perl-base
perl-base = 5.10.0-62.16

# rpm -q --requires -p /var/cache/mock/suse-11.1-x86_64-rtems/yum_cache/update/packages/perl-5.10.0-62.18.1.x86_64.rpm | grep perl-base
perl-base = 5.10.0

Fedora 11's mock/yum/rpm is not able to resolve this issue and fails.

FWI: I am also observing that the number of such issue increase with SUSE releases, i.e. suse-10.3 has less such issues than 11.0, and 11.0 less such issues than 11.1 (11.1 under Fedora 11 mock/rpm/yum is 1st class disaster).

> Example (here suse-11.1):

That example only demonstrates that their RPM version comparison is expected to work exactly like Fedora's. [Or else they could not require "perl-base = 5.10.0" and expect perl-base = 5.10.0-62.16 to satisfy the dep.]


> Fedora 11's mock/yum/rpm is not able to resolve this issue and fails.

So, can you create a reduced [or minimal] test-case to show that?

Does F11's RPM alone fail for you when trying to install exemplary openSUSE packages with such dependencies?

Or is it the combination of yum+rpm or just mock+yum+rpm? Or perhaps it's even important for the packages to be built with suse's RPM?

[...]

I can show that F11's RPM handles such dependencies as expected (they've come up during broken deps and bad Obsoletes/Provides reports regularly):

$ sudo rpm -ivh subpkg-1.0-2.fc11.i586.rpm 
error: Failed dependencies:
        goodpkg = 1.0 is needed by subpkg-1.0-2.fc11.i586
        something = 2.0 is needed by subpkg-1.0-2.fc11.i586

$ rpm -qp --provides goodpkg-1.0-2.fc11.i586.rpm 
something = 2.0-4
goodpkg = 1.0-2.fc11
goodpkg(x86-32) = 1.0-2.fc11

$ sudo rpm -ivh subpkg-1.0-2.fc11.i586.rpm goodpkg-1.0-2.fc11.i586.rpm 
Preparing...                ########################################### [100%]
   1:goodpkg                ########################################### [ 50%]
   2:subpkg                 ########################################### [100%]

Created attachment 356918 [details]
suse-10.3-x86_64 mock setup

(In reply to comment #23)

> > Fedora 11's mock/yum/rpm is not able to resolve this issue and fails.
> 
> So, can you create a reduced [or minimal] test-case to show that?

Yes. Install the tarball above on Fedora 11/x86_64:
cd /; tar xjvf suse-mock-demo.tar.bz2

Then try the following:
# mock -r suse-10.3-x86_64 --init
# mock -r suse-10.3-x86_64 --update

Result:

# mock -r suse-10.3-x86_64 --update
INFO: mock.py version 0.9.16 starting...
State Changed: init plugins
State Changed: start
ERROR: Command failed: 
 # /usr/bin/yum --installroot /var/lib/mock/suse-10.3-x86_64/root/  update
perl-5.8.8-76.4.x86_64 from installed has depsolving problems
  --> Missing Dependency: perl-base = 5.8.8 is needed by package perl-5.8.8-76.4.x86_64 (installed)
Error: Missing Dependency: perl-base = 5.8.8 is needed by package perl-5.8.8-76.4.x86_64 (installed)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.


# rpm -q --provides -p /var/cache/mock/suse-10.3-x86_64/yum_cache/base/packages/perl-base-5.8.8-76.x86_64.rpm  | grep perl-base
perl-base = 5.8.8-76

# rpm -q --requires -p /var/cache/mock/suse-10.3-x86_64/yum_cache/update/packages/perl-5.8.8-76.4.x86_64.rpm | grep perl-base
perl-base = 5.8.8

> Does F11's RPM alone fail for you when trying to install exemplary openSUSE
> packages with such dependencies?
How to test this? You normally can't install openSUSE rpms on RH for various reasons (Fedora's rpm lacking lzma compression until recently was one "big road block").

> Or is it the combination of yum+rpm or just mock+yum+rpm?
My test case is "building rpms targetting for openSUSE using Fedora 11's mock"

> Or perhaps it's even
> important for the packages to be built with suse's RPM?
The issues I am referring to are with openSUSE packages during setting up a mock environment, i.e. mostly only genuine openSUSE packages are involved.

Unrelated to the payload.

We should move this elsewhere. fedora-devel-list e.g. or Yum upstream.

openSUSE 10.3 metadata is pretty much incompatible here with F11 also for i586. And it smells a lot as if the old "no Epoch => zero Epoch" issue is involved. For example, I see dependency rpm:entry tags without "epoch" attributes. The missing Epoch ("None" in Python) reveals a bug in repoclosure ( http://yum.baseurl.org/ticket/193 ) which causes %version to be missing in versioned deps. And even if fixed, I get tons of broken deps, such as "python >= 0:2.5" or "splashy = 0.3.3" while splashy-0.3.3-45.2 clearly is in the repo and provides the default "splashy = 0.3.3-45.2". But still, Yum whatProvides does not resolve it. It wouldn't surprise me if transparently added Epoch 0 (somewhere) causes all this.

Yup this doesn't belong here... but just FWIW, rpm itself finds no problems in the package set (whereas I can easily reproduce the "mock --update" failure):
[root@localhost ~]# rpm -Va --nofiles --root /var/lib/mock/suse-10.3-x86_64/root/
[root@localhost ~]#

(In reply to comment #27)
> Yup this doesn't belong here... 
... yes, meanwhile we know.

Michael seems to be right, it's suse's update repos' metadata which seems to be incompatible to RH's yum (RH's yum seems to be able to process the "distribution" repos).