Bug 975627 - use productmd to import distros instead of reading .composeinfo/.treeinfo directly
use productmd to import distros instead of reading .composeinfo/.treeinfo dir...
Status: NEW
Product: Beaker
Classification: Community
Component: lab controller (Show other bugs)
0.12
Unspecified Unspecified
medium Severity unspecified (vote)
: ---
: ---
Assigned To: beaker-dev-list
tools-bugs
DistroManagement
: FutureFeature, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-18 21:02 EDT by Dan Callaghan
Modified: 2016-05-26 09:20 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
: 1013414 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
prototype patch from Daniel Mach (6.09 KB, patch)
2013-06-18 21:02 EDT, Dan Callaghan
no flags Details | Diff

  None (edit)
Description Dan Callaghan 2013-06-18 21:02:11 EDT
Created attachment 762683 [details]
prototype patch from Daniel Mach

Daniel Mach has written a Python library, productmd, to parse metadata for RHEL distro trees and composes. We could replace a lot of the logic in LabController/src/bkr/labcontroller/distro_import.py by making productmd do the hard work for us.

Eventually it might be possible use productmd for Fedora and RHEL-alikes too. In the near term we would need to support both the old parsers and productmd.

Prototype patch written by Daniel Mach is attached.
Comment 2 Raymond Mancy 2013-06-19 08:00:19 EDT
Thanks for the patch Daniel,

Can you point us to some .composeinfo files for testing RHEL4/5/6/7 ?

Also, I've noticed that in that patch you commented out all the TreeInfo* classes, although I can't see that the productmd how to productmd replaces these.

At least for the moment, beaker supports importing single trees by passing the tree URL. Although I guess we don't necessarily need to keep this (and in fact probably don't need to seeing as we can pass arch and variant options to distro_import.py now).
Comment 6 Nick Coghlan 2013-09-30 00:23:58 EDT
The thing that makes me nervous about Beaker relying on productmd is that it means we run a high risk of tree layouts becoming entirely implementation defined, rather than being at least nominally stable filesystem formats that can reasonably consumed by third parties.
Comment 7 Daniel Mach 2013-09-30 08:50:00 EDT
(In reply to Nick Coghlan from comment #6)
> The thing that makes me nervous about Beaker relying on productmd is that it
> means we run a high risk of tree layouts becoming entirely implementation
> defined, rather than being at least nominally stable filesystem formats that
> can reasonably consumed by third parties.

I understand your concerns, but from my POV, providing product metadata is the only option that really makes any sense.

Look back at all Fedora / RHEL /  other prouduct releases - they are different and will possibly be different in the future.
All we can do is to provide metadata to describe the release content.
What the tools do currently is guessing and we want to give them exact information they can rely on.

And one more thing about "implementation defined".
Metadata is defined this way currently, which is unfortunate.
I plan to write proper format docs, independent on implementation.
This will probably align with productmd 1.0 release.
Comment 8 Nick Coghlan 2013-09-30 23:29:37 EDT
Thanks Daniel, that's good news. I'm OK with using a common Python API, so long as the on-disk format is still well-defined.

That means the only blocker for this is productmd reaching open source status so the upstream Beaker project can depend on it. (Note that I created bug 1013414 to cover the compatibility fixes for the current import script)
Comment 9 Raymond Mancy 2013-10-01 00:13:17 EDT
So does that mean we are removing from beaker-import the tree level importing? Looking at the original patch that dmach supplied, productmd only supports compose level imports.
Comment 10 Nick Coghlan 2013-10-01 02:31:48 EDT
No, it just means we'll use productmd for the cases it supports.
Comment 13 Nick Coghlan 2015-02-10 18:48:53 EST
Productmd has now been published as an open source project: https://github.com/release-engineering/productmd/

There are Sphinx docs in the repo - dmach, could you update this BZ with a link once those are published somewhere? (eg. ReadTheDocs)
Comment 14 Daniel Mach 2015-02-11 07:11:02 EST
Docs live here:
http://release-engineering.github.io/productmd/

The composeinfo part is not fully finished yet, let me know if there's something unclear.
Comment 15 Nick Coghlan 2015-02-11 23:17:06 EST
Beaker 20's looking pretty full already, so I suggest we take a closer look at this for Beaker 21.

Note You need to log in before you can comment on or make changes to this bug.