Bug 739701

So my first observation is that, if the debugging output is correct, the repository is broken, it's providing libgconfmm-2.6.so.1()(64bit) in a package which has no business providing it (a noarch package providing a shared library, and which is an .el6 package in a F15 repository, huh?). There are also 2 google-* packages also abusively providing that shared library.

Now why zif is failing on this is another question, which I'll have to leave to Richard Hughes to answer. (I'm not a developer of zif.)

Yes, the repo looks pretty messed up. I'll have a proper look tomorrow and try to reproduce and see what we should do.

(In reply to comment #1)
Kevin,  I don't think the debugging information from zif is correct.

On the same system
repoquery -q --repoid=openshift-express --whatprovides "libgconfmm-2.6.so.1()(64bit)"
returns nothing.  

Whereas
repoquery -q --repoid=fedora --whatprovides "libgconfmm-2.6.so.1()(64bit)" 
returns:

gconfmm26-0:2.28.2-2.fc15.x86_64


the openshift-express repo provides exactly one package rhc. There's no indication whatsoever from the yum based tooling that I understand how to use that the libgconfmm provides you point is referenced in the openshift-express repository metadata. And I've looked at the primary.xml in 
/var/cache/zif/x86_64/15/openshift-express/

and there's no reference to any unexpected provides which do not match the rhc package.

-jef

Created attachment 523925 [details]
Problem again this time with upstream adobe repo enabled

zif -v install paprefs     


this time with the adobe-linux-i386 repo enabled. Again a failure doing a dep resolution. Fails with:

The transaction failed: Failed to filter newest: cannot compare gconfmm26;2.28.2-2.fc15;x86_64;fedora with adobe-release-i386;1.0-1;noarch;adobe-linux-i386

repoquery -q --provides -a --repoid adobe-linux-i386 |grep gconfmm
returns nothing.

here is the repo definition as installed by adobe-release-i386-1.0-1.noarch

[adobe-linux-i386]
name=Adobe Systems Incorporated
baseurl=http://linuxdownload.adobe.com/linux/i386/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux


Lightning striking twice?

Ok testing this with F16 beta2. I did the following:

1) Installed a minimal system
2) yum groupinstall kde --exclude=digikam --exclude=kipi*
3) make 4 repo files in /etc/yum.repos.d
[openshift-express]
name=Openshift-express
baseurl=https://openshift.redhat.com/app/repo/rpms/15/$basearch/
failovermethod=priority
skip_if_unavailable=1
gpgkey=https://openshift.redhat.com/app/repo/RPM-GPG-KEY-redhat-beta
ggpkey=https://openshift.redhat.com/app/repo/RPM-GPG-KEY-redhat-release
enabled=1
gpgcheck=1

[adobe-linux-i386]
name=Adobe Systems Incorporated
baseurl=http://linuxdownload.adobe.com/linux/i386/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/i386
enabled=1
gpgcheck=1

[google-talkplugin]
name=google-talkplugin
baseurl=http://dl.google.com/linux/talkplugin/rpm/stable/i386
enabled=1
gpgcheck=1

[root@mae ~]# zif repo-list
                              [==============================] (100%)   
adobe-linux-i386          enabled   Adobe Systems Incorporated
fedora                    enabled   Fedora 16 - i386
fedora-debuginfo          disabled  Fedora 16 - i386 - Debug
fedora-source             disabled  Fedora 16 - Source
google-chrome             enabled   google-chrome
google-talkplugin         enabled   google-talkplugin
openshift-express         enabled   Openshift-express
updates                   disabled  Fedora 16 - i386 - Updates
updates-debuginfo         disabled  Fedora 16 - i386 - Updates - Debug
updates-source            disabled  Fedora 16 - Updates Source
updates-testing           enabled   Fedora 16 - i386 - Test Updates
updates-testing-debuginfo disabled  Fedora 16 - i386 - Test Updates Debug
updates-testing-source    disabled  Fedora 16 - Test Updates Source

'zif list | grep openshift' does not show it. If I do yum list | grep openshift I see rhc.noarch In fact I don't see anything from the other repos in a zif list.

Next I do a zif install google-talkplugin. A zif list shows it as installed, but  afterwords any zif install command gives me errors:

(zif:8003): Zif-CRITICAL **: zif_object_array_add: assertion `object != NULL' failed
The transaction failed: Failed to filter newest: cannot compare google-talkplugin;2.3.2.0-1;i386;google-talkplugin with rubygem-parseconfig;0.5.2-4.fc15;noarch;fedora

Hopefully this helps debug what may be the errors are.

(In reply to comment #5)

This unlistable openshift repository content confirmed but in a different way 

zif list |grep rhc   does not show rhc package as being available

but 

zif install rhc  installs the package as expected.

That is highly unexpected behavior.


similarily zif list |grep adobe returns nothing

but

zif update  includes the following:

flash-plugin-10.3.183.10-release.i386 (adobe-linux-i386)

but

zif update flash-plugin  tells me there is no update ready.


This is seriously borked. 

Thanks Stephen for taking the time to confirm some of what I am seeing. This has been driving me crazy over the last couple of days. The internal inconsistency in the repodata handling between the different functions in zif make it impossible to do any comprehensive testing against real-world repos.

-jef

(In reply to comment #5)
> Ok testing this with F16 beta2. I did the following:

Cool, thanks for the detailed instructions, I appreciate it.

> 'zif list | grep openshift' does not show it. If I do yum list | grep openshift
> I see rhc.noarch In fact I don't see anything from the other repos in a zif
> list.

This was a bug in ZifRepoMdPrimaryXml. Most of my repos are new-style sqlite-based repos and so I didn't spot this bug until now. Basically:

commit 97b58553756fda953d280c363f168958aac5f65d
Author: Richard Hughes <richard>
Date:   Fri Sep 23 10:33:16 2011 +0100

    Ensure the ZifMdPrimaryXml is loaded before we get the package list
    
    Fixes the other half of https://bugzilla.redhat.com/show_bug.cgi?id=739701

> (In reply to comment #5)
> This unlistable openshift repository content confirmed but in a different way 
> zif list |grep rhc   does not show rhc package as being available

This was caused by a regression when I turned on the native resolve flag in the command line client:

commit fd164226e54a92c9115b8b6f6317e5df3e825eb8
Author: Richard Hughes <richard>
Date:   Wed Sep 14 14:33:05 2011 +0100

    Prefer the native arch when resolving if the ZIF_STORE_RESOLVE_FLAG_PREFER_NATIVE flag is set

This caused all resolves to a noarch package to fail. You can reproduce the failure using only the fedora repos using:

"zif resolve perl-Log-Message-Simple"

What we basically want to encode in the matching logic is that noarch matches any arch. This is what I've done this morning with:

commit 31ea01238dd83c734f57456fe4e0cda6b71fdf4f
Author: Richard Hughes <richard>
Date:   Fri Sep 23 09:41:39 2011 +0100

    Allow Resolve(zif.i386) to match zif.noarch
    
    Resolves half of https://bugzilla.redhat.com/show_bug.cgi?id=739701

Can you guys please check that verifies the problem by checking git master, or my snapshot repo please. Thanks!

This should fix things once and for all:

commit 4c5ee94c5c15224e5b5405095a94703cb6aaee7f
Author: Richard Hughes <richard>
Date:   Fri Sep 23 18:57:04 2011 +0100

    Use the results from all repos when calculating the 'best' provide
    
    Before we were searching each repo individually, and finishing when a repo
    provided something that looked good enough.
    This doesn't work when repos contain packages with broken provides, and we have
    to consider all the packages at once.
    
    Resolves https://bugzilla.redhat.com/show_bug.cgi?id=739701

Okay, using git master I now get:

$ zif install rhc
Transaction summary:
  Installing:
  1.	rhc-0.75.9-1.el6_1.noarch (openshift-express)
  Installing for dependencies:
  1.	ruby-1.8.7.352-1.fc16.x86_64 (fedora)
  2.	ruby-irb-1.8.7.352-1.fc16.noarch (fedora)
  3.	ruby-rdoc-1.8.7.352-1.fc16.noarch (fedora)
  4.	rubygem-json-1.4.6-3.fc15.x86_64 (fedora)
  5.	rubygem-parseconfig-0.5.2-4.fc15.noarch (fedora)
  6.	rubygems-1.8.10-1.fc16.noarch (updates-testing)
Total download size: 1.8 MB
Run transaction? [y/N] 

Can you please confirm this works for you guys please. Thanks.

Richard,

When you say "broken provides" what are you referring to in the openshift-express case?  I'm still not sure I understand what is "broken" with regard to the rhc package itself or the openshift-express repodata.   Or is there some subtle brokenness in the fedora repository provided rubygem packageset that is getting pulled in as a dependency?

-jef

Richard,

I pulled git and did a local build of zif just now on an F15 system.

I see different behavior but its still not entirely what I expected.

It looks like zif is failing to report the full list of packages to be downloaded at the Y/N review step.

Here's how I reproduced the issue.

Prep the system

yum remove rhc "rubygems*"

confirm that no rubygems* packages are installed on system with rpm -q

rpm -q rubygems

no package installed

./zif install rhc


Transaction summary:
  Installing:
  1.	rhc-0.75.9-1.el6_1.noarch (openshift-express)
  Installing for dependencies:
  1.	rubygem-json-1.4.6-3.fc15.x86_64 (fedora)
Total download size: 0.7 Mb

Note that the required rubygems package is not listed... but it is installed as part of the zif transaction.

rpm -q rubygems

package installed but was not listed in the zif transaction review!

On install Zif should be selecting rubygems packages as it is the only thing which provides one of the rubygem-json requirements.

So zif seems to be doing the depsolving its just not providing the correct "Installing for dependencies" information nor correct "download size" at the Y/N review point. 


-jef

git build of zif  is still is showing oddness on update.

./zif update flash-plugin
No packages found (after filter)

./zif update
Transaction summary:
  Updating the system:
  1.	device-mapper-1.02.63-4.fc15.x86_64 (updates-testing)
  2.	device-mapper-event-1.02.63-4.fc15.x86_64 (updates-testing)
  3.	device-mapper-event-libs-1.02.63-4.fc15.x86_64 (updates-testing)
  4.	device-mapper-libs-1.02.63-4.fc15.x86_64 (updates-testing)
  5.	espeak-1.45.05-3.fc15.x86_64 (updates-testing)
  6.	flash-plugin-10.3.183.10-release.i386 (adobe-linux-i386)
  7.	gnupg2-2.0.18-1.fc15.x86_64 (updates-testing)
  8.	google-musicmanager-beta-1.0.18.6104-0.x86_64 (google-musicmanager)
  9.	lvm2-2.02.84-4.fc15.x86_64 (updates-testing)
  10.	lvm2-libs-2.02.84-4.fc15.x86_64 (updates-testing)
  11.	m4-1.4.16-2.fc15.x86_64 (updates-testing)


Richard, do I need to file a separate bug for that?

-jef

(In reply to comment #10)
Okay so the transaction summary issue seems to be an unrelated bug with how the hashbar is being written.  It appears that the "Completed" hashbar is overwriting the transaction summary display and effectively hiding a couple of lines of the transaction.

Enabling -v is an effective workaround as it disables the unfortunately garbled effect of the default hashbar display.

So as of now the zif install rhc  case apepars to have been fixed. 

flash-plugin update case awaits. Unfortunately this one is going to be harder to replicate. You would need to have an older flash-plugin already installed on system to trigger the scenario.  

I could make my current rpmdb available to you so you could try to replicate the problem.
 

-jef

The flash-plugin case is actually quite simple to explain: it's a 32-bit-only package, your system is 64-bit. Saying just "flash-plugin" with no arch suffix matches only x86_64 and noarch packages, you have to write "flash-plugin.i386" explicitly.

Kevin,

Interesting thought. but sadly doesn't appear to resolve the issue. 

./zif -v -n update flash-plugin.i386  does not work.

Nor does ./zif -v -n install adobeair.i386  work

Nor for that matter does ./zif -v -n install libxslt.i686  work

libxslt,i686 is provided in the Fedora repository and is installable via yum.

I even tried changing multilib_policy to "all" from "best" with no change.

Kevin, can you  please attempt to install libxslt.i686 on a 64bit system as provided by the standard fedore repositories (any branch)? 


For completeness verbose output using my local build of git zif pulled today.

./zif -v -n update flash-plugin.i386

TI:12:28:00	Zif version 0.2.5
TI:12:28:00	using config /usr/local/etc/zif/zif.conf
TI:12:28:00	loading config file /usr/local/etc/zif/zif.conf
TI:12:28:00	no override file specified
Action: Updating
Percentage: 1
TI:12:28:00	abandoning cache
Allow cancel: FALSE
Action: Loading installed
Detail: /
TI:12:28:00	not using yumdb lookup as disabled
TI:12:28:00	using rpmdb at /
TI:12:28:02	using history lookup
TI:12:28:02	trying to open database '/var/lib/zif/history.db'
Allow cancel: TRUE
Percentage: 4
TI:12:28:02	done, so cancelling action loading-rpmdb
Percentage: 5
TI:12:28:02	setting releasever '15'
Percentage: 6
No packages found (after filter)



-jef

(In reply to comment #12)
> (In reply to comment #10)
> Okay so the transaction summary issue seems to be an unrelated bug with how the
> hashbar is being written.  It appears that the "Completed" hashbar is
> overwriting the transaction summary display and effectively hiding a couple of
> lines of the transaction.

commit 5709b5a13cabc125d761dd81be7e12475c2d3f33
Author: Richard Hughes <richard>
Date:   Sat Sep 24 09:33:43 2011 +0100

    Don't overwrite the last two lines of additional deps

(In reply to comment #14)
> Nor does ./zif -v -n install adobeair.i386  work
> No packages found (after filter)

There's a logic bug in zif_package_array_filter_best_arch() -- I'll fix this on Monday. This should only affect installing using the Zif CLI, using git master and PackageKit should work fine. Thanks.

(In reply to comment #15)

small suggestion, you might want to add some logic in git master make check to test the zif cmdline tool that is built from git master as well as the library API against the same finite set of test transactions. 

-jef

Okay, can somebody check with master or one of my snapshot srpms please. This seems to work for me now. Thanks.

I am able to install and see the rhc command with the snapshot rpm for F15 rebuilt for F16.

Using git master rebuild locally I think I can confirm the fix.


/usr/local/bin/zif install adobeair.i386

succeeds

with alure-1.0-5.fc15.i686  installed on system
/usr/local/bin/zif update alure
succeeds 

and picks up alure-1.2-1.fc15.i686 (updates) 


/usr/local/bin/zif install adobeair

fails 

Just noting it for completeness not suggesting this is intended or not as per
the design on what zif is supposed to do in this case. It is different than yum
cmdline behavior and thus I note it as zif still advertises itself in the
manpage as yum compatible. Such deviation with regard to 32bit package handling
may break local admin scripted action. If the design of the zif cmdline tool is
meant to deliberately diverge with regard to 32bit handling on 64bit systems,
this should be documented and communicated, possibly in the manpage or at least
a passing reference in the manpage to a longer incompatibility document for
admins to review.


-jef

(In reply to comment #19)
> /usr/local/bin/zif install adobeair
> fails 

This is unintentional. I've fixed this with:

commit f3cec94a361b95537dd2452c9b49e3fe8809ea47
Author: Richard Hughes <richard>
Date:   Tue Sep 27 10:22:34 2011 +0100

    When specifying resolving with prefer-native, re-search non-native if there are no package results
    
    Resolves https://bugzilla.redhat.com/show_bug.cgi?id=739701 for good. :)

Can you please verify again please. Thanks!

Looks like the 32bit adobeair install works now and in a compatible way with yum.

Here's the summary for completeness.

time zif -n install adobeair
Transaction summary:
  Installing:
  1.	adobeair-2.6.0-19170.i386 (adobe-linux-i386)
  Installing for dependencies:
  1.	libxml2-2.7.8-6.fc15.i686 (fedora)
  2.	libxslt-1.1.26-8.fc15.i686 (fedora)
Automatically declined action

real	0m5.215s
user	0m4.641s
sys	0m0.456s


that looks as I expect on a 64bit system and matches yum's transaction.

time yum --assumeno  install adobeair
Setting up Install Process

================================================================================
 Package         Arch        Version              Repository               Size
================================================================================
Installing:
 adobeair        i386        2.6.0-19170          adobe-linux-i386         20 M
Installing for dependencies:
 libxml2         i686        2.7.8-6.fc15         fedora                  795 k
 libxslt         i686        1.1.26-8.fc15        fedora                  226 k

Transaction Summary
================================================================================
Install  1 Package (+2 Dependent packages)

Exiting on user Command
Your transaction was saved, rerun it with:
 yum load-transaction /tmp/yum_save_tx.2011-09-27.09-23.BfKu2y.yumtx

real	0m1.454s
user	0m1.272s
sys	0m0.170s

(In reply to comment #21)
> Looks like the 32bit adobeair install works now and in a compatible way with
> yum.

Yay, thanks all. Please reopen if there's anything that I've missed.

They'll be a new release on the 3rd October, and I'll push it to F16 then.