Hide Forgot
Description of problem: John Lockhardt pointed us to an interesting idea of running rpmdeplint against system configured repositories. The motivation is to reduce tickets for the current TPS system, get more users try out rpmdeplint and give us feedback. Idea is to add a new option to rpmdeplint: --repos-from-system which picks up any configured (yum?) system repositories. With the given rpms, it'll run the checks against the system repositories. Open Questions: * How do users get rpmdeplint installed? (At the time of writing it can only be installed with dnf)
I forgot to document one other caveat: hawkey and librepo need to be present on the system, possibly as a minimum version as well in order to run rpmdeplint.
isn't it available with pip and easy_install as well? I thought that was one of the advantages of putting it up on PyPi?
Good idea, but you still need to consider hawkey and librepo which are not Python packages. They need to be compatible and installed. That's one point we need to figure out if the RHEL7 boxes and their versions of these system dependencies are compatible.
Seems like librepo and hawkey are shipped in RHEL7 (librepo since 7.2 and hawkey in extras since 7.1... not sure if that's exactly right). So far they seem to be new enough versions. Surprisingly, the first issue I have hit running on RHEL7 is actually a missing API in rpm-python (4.11, vs. 4.13 we have been developing against in Fedora).
So the rpm.files type and its method .matches() (rpmfiles and rpmfilesCompare in C land) is what we are using in the conflict detection code to decide whether rpm will allow two files to coexist. This API was introduced in rpm 4.12, and the equivalent in previous versions (rpmfi, rpmfiCompare) was never exposed in the Python bindings at all. Also, from what I can tell, the rpm.fi type that *did* exist in Python doesn't even expose enough information about each file entry to perform the comparison ourselves, even if we wanted to reimplement the logic from rpmfiCompare (which I wouldn't want to do). Shipping an updated parallel-installable rpm package, or bundling the bindings somehow, is totally off the table I think. So running rpmdeplint on RHEL7 seems to be a non-starter. :-(
I guess one possibility might be to use ctypes to call the rpmfiCompare APIs on rpm 4.11...
The ctypes stuff is quite hairy: http://gerrit.beaker-project.org/5259 *but* with that sorted out, the tests are fully passing on RHEL7 (even though the hawkey and librepo versions are a bit older than what we have been using in Fedora).
Turns out rpmfluff needs a bit of work to be usable on RHEL7 as well...
(In reply to Dan Callaghan from comment #8) > Turns out rpmfluff needs a bit of work to be usable on RHEL7 as well... https://pagure.io/rpmfluff/pull-request/13
The other sucky thing about system-wide repo configuration is they will be using yumvars ($releasever and $basearch) which rpmdeplint will need to substitute. Probably by importing either yum or dnf, depending which is installed, and poking inside their API...
So dnf has had a public-looking (but undocumented) API for loading the system-wide yum repo configuration, dnf.conf.read.RepoReader, since 0.6.4. And if we're going to use that, we might as well use dnf code for fetching the repodata instead of interacting with librepo directly ourselves... But that doesn't help us on systems with yum.
Most likely libhif will/would at some point have something we could use here, but that is in so much flux still that I am not even going to bother... (What is libdnf? Seems to be a new direction again for the libhif+hawkey merger as far as I can see...)
Assuming we want to just use standard ConfigParser to load the yum repo config and then do the substitutions in rpmdeplint itself, we will need to get the yumvars to be substituted. In yum land, yumvars are pieced together across a bunch of different startup code and there is (of course) no sensible API we can call from rpmdeplint to figure out what they are. The important ones (and the only ones likely to need substituting in the repo config) are 'basearch' which comes from yum.rpmUtils.getBaseArch(), 'arch' which comes from yum.rpmUtils.getCanonArch(), and 'releasever' which comes from yum.config._getsysver("/",startupconf.distroverpkg), where startupconf.distroverpkg is most likely the default ['system-release(releasever)', 'redhat-release']. As for dnf, its implementation of yumvars is called dnf.conf.substitutions.Substitutions, containing: 'arch' from hawkey.detect_arch() 'basearch' from dnf.rpm.basearch(arch) and anything from the environment and /etc/dnf/vars (seems to be empty by default on Fedora). 'releasever' is magically injected during cli configuration as well, from dnf.rpm.detect_releasever(conf.installroot).
So this patch gets us most of the way. It parses yum config and does yumvars substitution: http://gerrit.beaker-project.org/5278 so the new --repos-from-system option does work on Fedora. However it breaks on my RHEL7 box because librepo doesn't know how to do the SSL client certificate magic in order to fetch from cdn.redhat.com: 2016-09-28 16:07:14,906 rpmdeplint.repodata DEBUG Loading repodata for rhel-7-server-extras-rpms from https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os [...] librepo.LibrepoException: (19, u'Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried', u'Usable URL not found') I'm not sure how important it is to have rpmdeplint work with live CDN repos which need SSL certificate magic. I don't even know how we could do that, since I think subscription-manager installs a yum plugin for doing the magic...
Setting this to MODIFIED based on the patches above, which are merged now. As for the issues with CDN repositories needing client certs in comment 14... that can be a separate RFE if it turns out we need it. (I suspect the resolution will be CANTFIX for that one too.)
rpmdeplint 1.2 has been released.