Bug 469363

Summary: list-tagged with --latest and --inherit returns the latest in the listed tag not out of the set of inherited tags
Product: [Fedora] Fedora Reporter: Bruno Wolff III <bruno>
Component: kojiAssignee: David Cantrell <dcantrell>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: medium    
Version: rawhideCC: dcantrell, dennis, mikem
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-10-31 17:02:38 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Bruno Wolff III 2008-10-31 14:37:03 UTC
Description of problem:
I am not sure if this is a bug or a feature request for an option to do what I want, a suggested change in tagging work practice, or a way to use list-tagged with more than one tag.
If you use the list-tagged command to get a list of all packages in a tag using inheritence you don't get later versions from the from inherited packages. So if near a release some things start getting tagged dist-f10-updates-candidates and then later versions get tagged dist-f10 and/or dist-f10 final, you can't pick up the latest set of all packages included in dist-f10 and f10-updates-candidates (in one command). I can see there could be issues if latest is by date (rather than version) and dist-f9 were to get an update that would take precedence over an f10 update that is actually a later version.
For now I can do a query for several tags and pipe them through sort -u and get what I want, but it would be nice if there was a way to do that in koji, so it could be done server side and save some overall work.

Version-Release number of selected component (if applicable):
koji-1.2.6-1.fc10.noarch

How reproducible:
100%

Steps to Reproduce:
1. Shown below
2.
3.
  
Actual results:
[bruno@cerberus ~]$ koji list-tagged --inherit dist-f10-updates-candidate xorg-x11-drv-ati
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
xorg-x11-drv-ati-6.6.2-4.fc6              dist-fc6              jkeating
xorg-x11-drv-ati-6.6.3-1.fc6              dist-fc6-updates      jkeating
xorg-x11-drv-ati-6.6.3-2.fc7              dist-fc7              jkeating
xorg-x11-drv-ati-6.6.3-4.fc7              dist-fc7-updates      ajax
xorg-x11-drv-ati-6.7.194-1.fc8            dist-f8               airlied
xorg-x11-drv-ati-6.7.195-3.fc8            dist-f8               airlied
xorg-x11-drv-ati-6.7.196-1.fc8            dist-f8-updates       airlied
xorg-x11-drv-ati-6.7.196-2.fc8            dist-f8-updates       airlied
xorg-x11-drv-ati-6.7.197-1.fc8            dist-f8-updates       airlied
xorg-x11-drv-ati-6.8.0-1.fc8              dist-f8-updates       airlied
xorg-x11-drv-ati-6.8.0-10.fc9             dist-f9               airlied
xorg-x11-drv-ati-6.8.0-11.fc9             dist-f9               airlied
xorg-x11-drv-ati-6.8.0-12.fc9             dist-f9               airlied
xorg-x11-drv-ati-6.8.0-14.fc9             dist-f9-updates       airlied
xorg-x11-drv-ati-6.8.0-19.fc9             dist-f9-updates       airlied
xorg-x11-drv-ati-6.8.0-3.fc9              dist-f9               airlied
xorg-x11-drv-ati-6.8.0-4.fc8              dist-f8-updates       airlied
xorg-x11-drv-ati-6.8.0-9.fc9              dist-f9               airlied
xorg-x11-drv-ati-6.9.0-29.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-30.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-31.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-32.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-33.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-34.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-35.fc10            dist-f10-updates-candidate  airlied
xorg-x11-drv-ati-6.9.0-36.fc10            dist-f10-updates-candidate  airlied
xorg-x11-drv-ati-6.9.0-37.fc10            dist-f10              airlied
xorg-x11-drv-ati-6.9.0-38.fc10            dist-f10              airlied
You have new mail in /home/bruno/Maildir
[bruno@cerberus ~]$ koji list-tagged --inherit --latest dist-f10-updates-candidate xorg-x11-drv-ati
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
xorg-x11-drv-ati-6.9.0-36.fc10            dist-f10-updates-candidate  airlied
[bruno@cerberus ~]$ 


Expected results:
[bruno@cerberus ~]$ koji list-tagged --inherit --latest dist-f10-updates-candidate xorg-x11-drv-ati
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
xorg-x11-drv-ati-6.9.0-38.fc10            dist-f10              airlied

Additional info:

Comment 1 Jesse Keating 2008-10-31 17:02:38 UTC
"latest" is purely a function of "the last one tagged".  Also inheritance ends once a build has been tagged locally, that is, there is a build of xorg-x11-drv-ati specifically tagged in dist-f10-updates-candidate.  That means it will /not/ inherit /any/ build of xorg-x11-drv-ati from /any/ parent tag.  This is critical so that we can keep exactly in a tag what we want in a tag.

Inheritance is only used for picking up builds from parents that we haven't already done in the child.

I don't see a bug here, just a slight misunderstanding of the nuances.

Comment 2 Bruno Wolff III 2008-10-31 17:24:09 UTC
Thanks for the explanation.

I'll just get a list for each tag that I might really want the latest from and merge them together. I'll end up with a couple different versions of a few packages in the reop, but I think that will work OK.

In this particular case getting a list from dist-f10-updates-candidates with inheritence and dist-f10 without inheritence should work for what I want. I might also need f10-final, but so far I haven't seen anything tagged with just f10-final.

Comment 3 Jesse Keating 2008-10-31 17:47:58 UTC
That's correct, everything we put in f10-final we put in dist-f10.  dist-f10 is an inheritance point up the stack, f10-final isn't.

As an aside, you /can/ use http://kojipkgs.fedoraproject.org/static-repos/ to get up to the hour new repos of various tags.  It's not mirrored though so use it wisely.

Comment 4 Mike McLean 2008-10-31 23:52:26 UTC
Just a little more explanation...

The latest concept is Koji is designed with process in mind, not so much nvr ordering. The idea is that the build considered latest can be easily controlled by tagging. Topmost level tag wins. Within a tag, most recently tagged build wins.

If you want latest by nvr ordering, you can construct this yourself from data available in the xmlrpc calls. Just call listTagged(sometag, inherit=True, latest=False). This will give you all the builds all the way down the inheritance line. Index them by package name and sort by nvr ordering (using rpm.labelCompare).

I suppose adding an option to the cli to handle this would be a reasonable rfe.

Comment 5 Bruno Wolff III 2008-11-01 06:00:28 UTC
Thanks for the additional explanation.

I am really looking for highest nvr in a set of tags and what they inherit from. Right now I am trying to get the latest from what will be in the final release, pending updates and updates-candidates. This allows for faster turn around for testing fixes. Also I cart home updates from work to home because of a slow network connection and the timing of rawhide updates hitting mirrors doesn't always work well for that.

My latest script grabs the latest for a couple of the tags with one set to look at inheritence to grab old stuff. So I end up with a couple of versions of a few things in the repo but that isn't a big deal. Grabbing the whole list and then sorting would work better, but I thought the extra data being sent would be more costly than having it pruned server side. But maybe that's wrong.

I am not doing a lot of testing of the iso images (though I did need to do that recently after an update caused some problems with encrypted file systems), so I can just do a createrepo to make a repo out of whatever I have at the end of the work day. Once the general release hits I expect to take a break from this until around the next alpha.

It looks like normally within a tag the highest nvr will be the latest, but I could see how rebuilding an older version for some reason might cause that not to be true. For the rest of the f10 prerelease time I'll just live with latest by time, but I'll look at doing things right when (and if) I want rapid updates for f11.

A command line option to do that would be nice. Currently I am grabbing stuff with a shell script that runs koji to get the list of rpms and then to get rpms I don't already have (from a local rawhide mirror or a previous pull). Having to get a list of all the the rpms would send a lot more data over the wire then I need. (Being able to limit the arches on the rpm list on the server side would cut this down even more.) Maybe for f11 I'll try to rewrite this part to use xmlrpc calls either using perl or learning and using python.