Red Hat Bugzilla – Bug 975627
use productmd to import distros instead of reading .composeinfo/.treeinfo directly
Last modified: 2018-02-05 19:41:31 EST
Created attachment 762683 [details]
prototype patch from Daniel Mach
Daniel Mach has written a Python library, productmd, to parse metadata for RHEL distro trees and composes. We could replace a lot of the logic in LabController/src/bkr/labcontroller/distro_import.py by making productmd do the hard work for us.
Eventually it might be possible use productmd for Fedora and RHEL-alikes too. In the near term we would need to support both the old parsers and productmd.
Prototype patch written by Daniel Mach is attached.
Thanks for the patch Daniel,
Can you point us to some .composeinfo files for testing RHEL4/5/6/7 ?
Also, I've noticed that in that patch you commented out all the TreeInfo* classes, although I can't see that the productmd how to productmd replaces these.
At least for the moment, beaker supports importing single trees by passing the tree URL. Although I guess we don't necessarily need to keep this (and in fact probably don't need to seeing as we can pass arch and variant options to distro_import.py now).
The thing that makes me nervous about Beaker relying on productmd is that it means we run a high risk of tree layouts becoming entirely implementation defined, rather than being at least nominally stable filesystem formats that can reasonably consumed by third parties.
(In reply to Nick Coghlan from comment #6)
> The thing that makes me nervous about Beaker relying on productmd is that it
> means we run a high risk of tree layouts becoming entirely implementation
> defined, rather than being at least nominally stable filesystem formats that
> can reasonably consumed by third parties.
I understand your concerns, but from my POV, providing product metadata is the only option that really makes any sense.
Look back at all Fedora / RHEL / other prouduct releases - they are different and will possibly be different in the future.
All we can do is to provide metadata to describe the release content.
What the tools do currently is guessing and we want to give them exact information they can rely on.
And one more thing about "implementation defined".
Metadata is defined this way currently, which is unfortunate.
I plan to write proper format docs, independent on implementation.
This will probably align with productmd 1.0 release.
Thanks Daniel, that's good news. I'm OK with using a common Python API, so long as the on-disk format is still well-defined.
That means the only blocker for this is productmd reaching open source status so the upstream Beaker project can depend on it. (Note that I created bug 1013414 to cover the compatibility fixes for the current import script)
So does that mean we are removing from beaker-import the tree level importing? Looking at the original patch that dmach supplied, productmd only supports compose level imports.
No, it just means we'll use productmd for the cases it supports.
Productmd has now been published as an open source project: https://github.com/release-engineering/productmd/
There are Sphinx docs in the repo - dmach, could you update this BZ with a link once those are published somewhere? (eg. ReadTheDocs)
Docs live here:
The composeinfo part is not fully finished yet, let me know if there's something unclear.
Beaker 20's looking pretty full already, so I suggest we take a closer look at this for Beaker 21.