I am currently working on making https://github.com/cockpit-project/cockpit-ostree/ fit for OCI based deployments. I noticed that I run into a lot of "Invalid refspec" errors when trying to deploy a registry image with a tag name, such as quay.io/fedora/fedora-coreos:stable . This works fine for "untagged" (well, implied "latest") OCI names, like I use on my personal laptop with ghcr.io/martinpitt/workstation-ostree-config. This doesn't seem to happen with the rpm-ostree CLI, just through the D-Bus API. rpm-ostreed is rather hard to talk to via D-Bus, this is impossible to show with busctl/gdbus, due to the private D-Bus transaction. So I wrote a standalone Python reproducer using GDBus through GI (dbus-python cannot really do this either). Reproducible: Always Steps to Reproduce: 1. rpm-ostree rebase ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable 2. reboot 3. rpm-ostree install -A python3-gobject-base gobject-introspection 4. python3 rpm-ostree-check-update.py Actual Results: unix:path=/run/rpm-ostree-transaction.sock Start: True transaction signal org.projectatomic.rpmostree1.Transaction.Message ('Pulling manifest: ostree-unverified-image:docker://quay.io/fedora/fedora-coreos:stable',) transaction signal org.projectatomic.rpmostree1.Transaction.ProgressEnd () transaction signal org.projectatomic.rpmostree1.Transaction.Message ('No upgrade available.',) transaction signal org.projectatomic.rpmostree1.Transaction.Finished (false, 'Generating update variant: Invalid refspec ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable') Expected Results: On my personal laptop which uses a container image name without a tag (i.e. "latest"), this works fine: booted: /org/projectatomic/rpmostree1/fedora unix:path=/run/rpm-ostree-transaction.sock Start: True transaction signal org.projectatomic.rpmostree1.Transaction.Message ('Pulling manifest: ostree-unverified-image:docker://ghcr.io/martinpitt/workstation-ostree-config',) transaction signal org.projectatomic.rpmostree1.Transaction.ProgressEnd () transaction signal org.projectatomic.rpmostree1.Transaction.Message ('No upgrade available.',) transaction signal org.projectatomic.rpmostree1.Transaction.Finished (true, '')
Created attachment 1966814 [details] python GDBus GI reproducer
Hmm, apparently it's not the tag name. I changed our integration tests to use "latest", and I'm still getting Generating update variant: Invalid refspec ostree-unverified-registry:localhost:5000/ostree-oci Deployments: ● ostree-unverified-registry:localhost:5000/ostree-oci Digest: sha256:da2503574d72f4cd0982060416ef1d47f3ce8db23821e660029bbce03c0d5353 Version: cockpit-base.1 (2023-05-25T09:13:30Z) `rpm-ostree upgrade --download-only` works fine, though. Adjusting the title accordingly.
> Steps to Reproduce Sorry, I forgot an important step to enable GDBus through GI. Full reproducer: 1. rpm-ostree rebase ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable 2. reboot 3. rpm-ostree install -A python3-gobject-base gobject-introspection 4. python3 rpm-ostree-check-update.py
This may relate to https://github.com/coreos/rpm-ostree/issues/4176
I'd really appreciate a quick check if we are using the API wrong, or if this is an actual bug in rpm-ostree. I noticed https://github.com/coreos/rpm-ostree/issues/4176 as well, but it has a different symptom. May be the same root cause, of course. Thanks!
I'm pretty sure it's just a dup of 4176 with the same root cause. We have https://github.com/coreos/rpm-ostree/pull/4486 inbound to attempt to fix it.
Thanks Colin! I subscribed to that PR.
I tested this again on current Fedora CoreOS 39.20240225.3.0 (aka current ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable), and this indeed works now. Thanks!
@walters : I'm still struggling with the concepts, with integrating this into the UI, though. https://coreos.github.io/rpm-ostree/container/#url-format-for-ostree-native-containers sounds like these would be "proper" OSTree repository specifications similar to https://ostree.fedoraproject.org or http://127.0.0.1:12345, but they don't work: # ostree remote add oci ostree-unverified-registry:quay.io/fedora/fedora-coreos stable; ostree remote refs oci; ostree remote delete oci error: Invalid URI scheme in ostree-unverified-registry:quay.io/fedora/fedora-coreos # ostree remote add oci ostree-unverified-image:docker://quay.io/fedora/fedora-coreos stable; ostree remote refs oci; ostree remote delete oci error: Invalid URI scheme in ostree-unverified-image:docker://quay.io/fedora/fedora-coreos I tried a few other variants with the same result. So I suppose these aren't "real" remotes. But it also doesn't seem to work on the fly: # ostree remote refs ostree-unverified-registry:quay.io/fedora/fedora-coreos error: Remote "ostree-unverified-registry:quay.io/fedora/fedora-coreos" not found So are container registries even something that the ostree command can understand, or is this more or less just a "special hack" in `rpm-ostree rebase`? My real laptop uses an OCI repo as well, "rpm-ostree status" says: ● ostree-unverified-registry:ghcr.io/martinpitt/workstation-ostree-config but indeed there's no remote in /etc/ostree/remotes.d/ at all. If that's the case, should these not be treated the same as "classic" ostree remotes, but also be a special case in the cockpit GUI? Thanks in advance for any hint!
> If that's the case, should these not be treated the same as "classic" ostree remotes, but also be a special case in the cockpit GUI? Yes. Containers are not ostree. > So are container registries even something that the ostree command can understand, or is this more or less just a "special hack" in `rpm-ostree rebase`? At a technical level it's a higher level tooling in https://github.com/ostreedev/ostree-rs-ext which is processed by `ostree container`. But note that as of recently we've created github.com/containers/bootc which is going really far towards putting ostree in the background entirely. (It would probably make sense to have Cockpit support bootc directly too)