As far as I know it isn't possible to see that data from the rpm-python API.

I looked at it in F16, and as far as I could see the information is available to python through the RPMSENSE_* bitmap in requirement flags.

The main point seems to be that especially requirements with only RPMSENSE_SCRIPT_PRE or RPMSENSE_SCRIPT_POST no longer are relevant when the package has been installed - and it seems like rpm ignores them. It would however be convenient if python-land didn't have to re-invent the interpretation of these flags.

(Slightly related: I just found out that an extra -v option to rpm -qR will make it show the type of each dependency and not just the name and version. That is very handy for debugging this. It would be nice if -v was mentioned in the rpm man page under -qR.)

Yes, the information is (and has been "forever") available to python through the RPMSENSE_* bitmap in dependency flags tag(s). The semantics of these is kinda exported in the C API (look for the _(INSTALL|ERASE)_ONLY_MASK related defines in http://rpm.org/api/4.9.0/rpmds_8h_source.html), but not documented or exported to python as such, although the RPMSENSE_* symbols are available to python.

Guess I should go do something about it...

As a consequence of this inconsistency I got this on yum update systemd:

Running Transaction Check
ERROR with transaction check vs depsolve:
authconfig is needed by systemd-33-2.fc16.x86_64
Please report this error in http://yum.baseurl.org/report
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
systemd-33-1.fc16.x86_64 has missing requires of authconfig

BTW rather than try to duplicate the logic dependency flag interpretation etc, package-cleanup would be better off using rpm transactionset methods to do the job of finding broken dependencies similar to how rpm itself does on verification. That way it'd become consistent, and remain that way, with the reports given by 'rpm -Va --nofiles' regardless of the rpm version:

With rpm >= 4.9.1, package-cleanup is not only reporting false positives on some of the scriptlet dependencies, it's also missing certain file dependency problems that rpm now detects and reports (file state is taken into account, so "wrong colored" and non-installed files cannot satisfy dependencies), eg:

[root@localhost RPMS]# rpm -Uvh /home/pmatilai/rpmbuild/RPMS/noarch/deptest-hello-1.0-1.noarch.rpm hello-2.0-1.*
Preparing...                ########################################### [100%]
   1:hello                  ########################################### [ 33%]
   2:deptest-hello          ########################################### [ 67%]
   3:hello                  ########################################### [100%]
[root@localhost RPMS]# rpm -e --test hello.x86_64
error: Failed dependencies:
	/usr/bin/hello is needed by (installed) deptest-hello-1.0-1.noarch
[root@localhost RPMS]# rpm -e --test hello.i686
[root@localhost RPMS]# rpm -e --nodeps hello.x86_64
[root@localhost RPMS]# rpm -Va --nofiles
Unsatisfied dependencies for deptest-hello-1.0-1.noarch:
	/usr/bin/hello is needed by (installed) deptest-hello-1.0-1.noarch
[root@localhost RPMS]# package-cleanup --problems
Loaded plugins: presto, refresh-packagekit
No Problems Found
[root@localhost RPMS]#

Slightly off-topic but also slightly relevant:
(In reply to comment #5)
> [root@localhost RPMS]# rpm -e --test hello.x86_64
> error: Failed dependencies:
>  /usr/bin/hello is needed by (installed) deptest-hello-1.0-1.noarch
...
> [root@localhost RPMS]# rpm -Va --nofiles
> Unsatisfied dependencies for deptest-hello-1.0-1.noarch:
>  /usr/bin/hello is needed by (installed) deptest-hello-1.0-1.noarch

It would be nice if it also told what kind of dependency was missing. That would make it much easier to debug and fix the issue.


Note also that the incorrect dependency checks not only show up in package-cleanup but also in 'yum check' and in all regular yum operations where it goes like this:
...
Running Transaction
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
systemd-33-2.fc16.x86_64 has missing requires of authconfig
...

Panu, what's the code like to use the rpm TS to do this work?

The basic idea is to feed installed headers into a transaction set and run ts.check() on it to figure out the problems. Rpm verify does this one header at a time, emptying the transaction after each but rpmtsEmpty() is not exported to python in any released rpm versions (doh) so for now python needs to do it all at once. Here's a minimal quick-and-dirty version that should work with pretty much any rpm version old and new:

import rpm
ts = rpm.ts()
for h in ts.dbMatch():
    ts.addInstall(h, None, 'i')
ts.check()
for p in ts.problems():
    print p

Besides missing requires, this would return possible conflict- and obsolete-problems too. Downside is the rather ugly rpm problem strings (although package-cleanup output is not particularly pretty either :)

Various alternatives exist, including using ts.check() return or callback to collect the missing requires but IIRC both methods miss the arch of the package so it'll make for ambiguous output on multilib systems. If not limited by rpm 4.4.x compatibility, in rpm >= 4.8.0 the problems are custom objects so you can format the output to your liking from the object properties, and rpm >= 4.9.0 the problems are collected in the originating transaction element which opens some extra possibilities.

Hmm, actually rpm < 4.9.0 actually suffers from the same problem as the bug at hand: it will report false missing dependencies for Requires(pre,post) deps. So doing it the rpm way is only really beneficial on recent versions (unless you want to do it the same on all versions for consistency, but then again that limits the possibilities otherwise)

 In "yum check" and package-cleanup --problems etc. I doubt we can create
install transaction sets when we want to check requires. And I'm pretty sure the error message data we get out of that wouldn't be usable anyway.
 So I guess we need to find a way to get the bit flag data out, and have package-cleanup just call into the "yum check" core code.


 I'm also confused by your example, why does the .i386 version of /usr/bin/hello not satisfy the filedep?

Created attachment 520321 [details]
Proof-of-concept patch to use rpmts for finding rpmdb problems

(In reply to comment #10)
>  In "yum check" and package-cleanup --problems etc. I doubt we can create
> install transaction sets when we want to check requires. And I'm pretty sure
> the error message data we get out of that wouldn't be usable anyway.

Attached is a proof-of-concept hack to do just that, seems to work here (requires rpm >= 4.9.0 though). Various details are ignored here (it would report obsoletes and conflicts problems besides just requires etc) but the former problem formatting is preserved.

>  So I guess we need to find a way to get the bit flag data out, and have
> package-cleanup just call into the "yum check" core code.

Getting the dependency bits out is just a matter of grabbing 'requireflags' out of the headers, the interpretation of those is the least of your worries if you want to duplicate and track the rpm logic of finding existing problems.

But agreed, package-cleanup needs to be fixed to use the existing yum core code for doing its work, regardless of what approach is used.

>  I'm also confused by your example, why does the .i386 version of
> /usr/bin/hello not satisfy the filedep?

I thought I explained it there, but lets try again: On multilib systems if colored files are "shared" between primary and secondary arch, colored file(s) from the preferred (typicically the primary) arch replace the file(s) from the non-preferred arch. If you remove the preferred arch package, any colored files are now gone:

[root@localhost tmp]# rpm -Uvh hello-2.0-1.i686.rpm hello-2.0-1.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:hello                  ########################################### [ 50%]
   2:hello                  ########################################### [100%]
[root@localhost tmp]# ls -l /usr/bin/hello 
-rwxr-x--x. 1 root root 4216 May 26 13:13 /usr/bin/hello
[root@localhost tmp]# rpm -e hello.x86_64
[root@localhost tmp]# ls -l /usr/bin/hello 
ls: cannot access /usr/bin/hello: No such file or directory

If anything depended on /usr/bin/hello, it would now be very much broken despite hello.i686 appearing to provide that path according to the naive "old rules". Rpm >= 4.9.0 takes this into account and complains if a transaction would *physically* remove a file that something depends on, ie the case in comment #5 example.

That it still allows the above case where /usr/bin/hello disappears when there are no dependencies, breaking the other package in the process, is basically a bug/not-implemented-yet piece that's likely to change one of these days. As a first step towards correcting the situation, rpm >= 4.9.0 will at least notice these issues on verify (see bug 528383), so after the above example rpm would previously report no issues on verify, you'll now get:
[root@localhost tmp]# rpm -V hello.i686
missing     /usr/bin/hello

While it should be entirely possible to duplicate this logic in yum, maintaining identical behavior with whatever rpm version happens to be in use will take a fair bit of work and likely still be dragging behind - consider the existence of this bug :) and also its currently missing conflicts/obsoletes problems and rpmlib() dependency problems.

> If you remove the preferred arch package, any colored files are now gone

 Yeh, this is the bit I always forget ... my mind refuses to believe that rpm doesn't have the .i386 version of /usr/bin/hello there at the end of the removal.

Back to yum-utils, package-cleanup is the thing needing fixing here. Perhaps it can take advantage of the "strong / weak dependency" thing introduced in yum for another case related to the %pre/%post dependencies.

This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.