Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1808428 - Regression: The transport "ostree:" is not supported in this build
Summary: Regression: The transport "ostree:" is not supported in this build
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: skopeo
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Lokesh Mandvekar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-28 14:03 UTC by Lukas Slebodnik
Modified: 2020-05-16 04:20 UTC (History)
8 users (show)

Fixed In Version: skopeo-0.1.41-2.0.1.37.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-16 04:20:50 UTC
Type: Bug


Attachments (Terms of Use)

Description Lukas Slebodnik 2020-02-28 14:03:36 UTC
Description of problem:
The new version of skopeo (0.1.41-1.fc30) was pushed to stable 4 days ago 
https://bodhi.fedoraproject.org/updates/FEDORA-2020-2a0aac3502
And we cannot anymore pull images to ostree with that version

Version-Release number of selected component (if applicable):
sh$ rpm -q skopeo
skopeo-0.1.41-1.fc30.x86_64

How reproducible:
Deterministic

Steps to Reproduce:
1. dnf install -y skopeo
2. ostree init --repo=/var/tmp/os3/ --mode=bare-user
3. /usr/bin/skopeo copy docker://docker.io/fedora:30 ostree:docker.io/fedora:30@/var/tmp/os3/

Actual results:
[testuser@abe8b38f8fc5 ~]$ ostree init --repo=/var/tmp/os3/ --mode=bare-user
[testuser@abe8b38f8fc5 ~]$ 
[testuser@abe8b38f8fc5 ~]$ /usr/bin/skopeo copy docker://docker.io/fedora:30 ostree:docker.io/fedora:30@/var/tmp/os3/
FATA[0000] Invalid destination name ostree:docker.io/fedora:30@/var/tmp/os3/: The transport "ostree:" is not supported in this build

Expected results:
[testuser@abe8b38f8fc5 ~]$ ostree init --repo=/var/tmp/os3/ --mode=bare-user
[testuser@abe8b38f8fc5 ~]$ 
[testuser@abe8b38f8fc5 ~]$ /usr/bin/skopeo copy docker://docker.io/fedora:30 ostree:docker.io/fedora:30@/var/tmp/os3/
Getting image source signatures
Copying blob 401909e6e2aa done
Copying config 177d5adf0c done
Writing manifest to image destination
Storing signatures
[testuser@abe8b38f8fc5 ~]$ echo $?
0

Additional info:

Comment 1 Lukas Slebodnik 2020-03-18 15:45:16 UTC
Giuseppe,

It looks like to did some related changes in upstream.
fedora 30 still has atomic utility which support ostree but latest upgrade of skopeo broke it.
How do you plan to fix it in fedora 30?

Comment 2 Giuseppe Scrivano 2020-03-18 16:17:22 UTC
there should not be any atomic tool on Fedora 30.  The last build I can see on Bodhi is for Fedora 28.

Can you please check from where the atomic tool is coming from?

Comment 3 Lukas Slebodnik 2020-03-18 17:15:27 UTC
(In reply to Giuseppe Scrivano from comment #2)
> there should not be any atomic tool on Fedora 30.  The last build I can see
> on Bodhi is for Fedora 28.
> 
> Can you please check from where the atomic tool is coming from?

Yes I can.
sh# podman run -ti --rm docker.io/fedora:30 bash
[root@e8c909734d28 /]# dnf repolist
repo id                                   repo name
fedora                                    Fedora 30 - x86_64
fedora-modular                            Fedora Modular 30 - x86_64
updates                                   Fedora 30 - x86_64 - Updates
updates-modular                           Fedora Modular 30 - x86_64 - Updates
[root@e8c909734d28 /]# dnf -e0 -d0 info atomic
Available Packages
Name         : atomic
Version      : 1.22.1
Release      : 28.gitb507039.fc30
Architecture : x86_64
Size         : 1.0 M
Source       : atomic-1.22.1-28.gitb507039.fc30.src.rpm
Repository   : fedora
Summary      : Tool for managing ProjectAtomic systems and containers
URL          : https://github.com/projectatomic/atomic
License      : LGPLv2+
Description  : The goal of Atomic is to provide a high level, coherent entrypoint to the
             : system, and fill in gaps.
             :
             : atomic can make it easier to interact with container runtimes for different
             : kinds of containers, such as super-privileged and system containers.
             :
             : The atomic host subcommand wraps rpm-ostree providing unified management.

[root@e8c909734d28 /]# cat /etc/os-release 
NAME=Fedora
VERSION="30 (Container Image)"
ID=fedora
VERSION_ID=30
VERSION_CODENAME=""
PLATFORM_ID="platform:f30"
PRETTY_NAME="Fedora 30 (Container Image)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:30"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f30/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=30
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=30
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Container Image"
VARIANT_ID=container

Comment 4 Lukas Slebodnik 2020-03-18 17:16:37 UTC
I can see it also in koji
https://koji.fedoraproject.org/koji/buildinfo?buildID=1184437
and f30 branch is not empty either
https://src.fedoraproject.org/rpms/atomic/tree/f30

atomic utility was removed just from f31+

Comment 5 Giuseppe Scrivano 2020-03-19 07:52:56 UTC
Dan, should we revert back to the previous build for Skopeo?

Comment 6 Daniel Walsh 2020-03-19 10:43:15 UTC
Yes I think so.

Comment 7 Lukas Slebodnik 2020-03-19 18:57:52 UTC
Giuseppe,
may I know what was the reason for removing support for ostree from skopeo in upstream?
Colin[1] (can) use ostree format[2] to do simple labels/filesystem checks for containers in openshift.
Which drop same capabilities by default
     "CapDrop": ["KILL", "MKNOD", "SETGID", "SETUID"]

I know you prefer to do lot of things with rootless podman,
but rootless != capabilityless.

Correct me is I am wrong but SETGID", "SETUID" are required even for rootless podman
die to ambient capabilities. And thus we cannot use it in openshift with default scc.

The colin uses just a single feature from atomic utility "atomic mount" which I could easily
reimplement and remove dependency on atomic utility. But skopeo would need to support ostree.
So we will need to either stuck with fedora 30 (even after EOL or to move to centos7 + install everything
from pip due to requirement on python3.6).

How difficult would be to support ostree even in skopeo >= 0.1.41-1?

[1] https://github.com/user-cont/colin
[2] https://github.com/user-cont/colin/blob/master/colin/core/target.py#L437

Comment 8 Giuseppe Scrivano 2020-03-20 07:36:53 UTC
The main reason from dropping support for ostree is that atomic is the only known user of it, and we are not maintaining atomic anymore.

The ostree backend is tailored for a very specific use case and required a lot of logic into the atomic tool.

Without a user namespace, you can extract (which is really an extraction) the image but all the files are owned by the same user.

That can be achieved also in an environment without SETUID/SETGID, as long as you are able to create a user namespace.  I'd expect `podman mount` to work also without capabilities, and the vfs backend to work without having access to /dev/fuse.

You can also use umoci to extract an OCI image: https://github.com/openSUSE/umoci

Comment 9 Lukas Slebodnik 2020-03-20 11:54:36 UTC
(In reply to Giuseppe Scrivano from comment #8)
> The main reason from dropping support for ostree is that atomic is the only
> known user of it, and we are not maintaining atomic anymore.
> 
> The ostree backend is tailored for a very specific use case and required a
> lot of logic into the atomic tool.
> 
> Without a user namespace, you can extract (which is really an extraction)
> the image but all the files are owned by the same user.
>

I knot but that is not a problem for my use-case. That was also the reason for
for ostree --mode=bare-user (in the description of BZ)
 
> That can be achieved also in an environment without SETUID/SETGID, as long
> as you are able to create a user namespace.

The openshift cluster is not maintained by me :-)
How can I test whether I can use user namespaces there?
The cluster seems to be deployed on el7 based on umame
3.10.0-957.21.2.el7.x86_64

Cause I would be glad to use podman directly.

Anyway, ostree support should stay in f30 :-)

Comment 10 Lukas Slebodnik 2020-03-20 12:05:13 UTC
(In reply to Giuseppe Scrivano from comment #8)
> You can also use umoci to extract an OCI image:
> https://github.com/openSUSE/umoci

That is quite interesting just have a question about difference in  skopeo inspect

bash-5.0$ export IMAGE_NAME=docker.io/fedora/apache:latest
bash-5.0$ /usr/bin/skopeo copy docker://$IMAGE_NAME oci:/var/tmp/oci:$IMAGE_NAME
Getting image source signatures
Copying blob 0fc456f626d7 done
Copying blob 8eea4f8b1da3 done
Copying blob 40da690b3498 done
Copying blob 4e81794d88f1 done
Copying blob ebb42f0b0e1a done
Copying config aeee6d164d done
Writing manifest to image destination
Storing signatures

bash-5.0$ ostree init --repo=/var/tmp/os3/ --mode=bare-user
bash-5.0$ /usr/bin/skopeo copy docker://$IMAGE_NAME ostree:$IMAGE_NAME@/var/tmp/os3
Getting image source signatures
Copying blob 0fc456f626d7 done
Copying blob 8eea4f8b1da3 done
Copying blob 40da690b3498 done
Copying blob 4e81794d88f1 done
Copying blob ebb42f0b0e1a done
Copying config c786010769 done
Writing manifest to image destination
Storing signatures

bash-5.0$ diff -u <(skopeo inspect oci:/var/tmp/oci:$IMAGE_NAME) <(skopeo inspect ostree:$IMAGE_NAME@/var/tmp/os3)
--- /dev/fd/63  2020-03-20 12:01:14.463930483 +0000
+++ /dev/fd/62  2020-03-20 12:01:14.463930483 +0000
@@ -1,8 +1,8 @@
 {
-    "Digest": "sha256:811764eb3b54db5650e0c1e55dd54f499bcc051b0a92efef58b448124c586b72",
+    "Digest": "sha256:8531786520bb57b155bbb39d3c670dceab554b9c4ccdb556ccfbe89b23df414c",
     "RepoTags": [],
     "Created": "2017-02-08T23:42:37.72165736Z",
-    "DockerVersion": "",
+    "DockerVersion": "1.12.6-cs6",
     "Labels": {
         "RUN": "docker run -d -p 80:80 $IMAGE"
     },

Is the change in digest expected or bug in skopeo?
(sorry for misusing BZ but I do not want to create unnecessary bugs)

Comment 11 Giuseppe Scrivano 2020-03-20 13:08:44 UTC
probably a bug in skopeo

You can check whether user namespaces are supported by running "podman unshare" as unprivileged user, or alternatively run directly "unshare -r COMMAND"

Comment 12 Lukas Slebodnik 2020-04-07 15:51:35 UTC
(In reply to Giuseppe Scrivano from comment #5)
> Dan, should we revert back to the previous build for Skopeo?

(In reply to Daniel Walsh from comment #6)
> Yes I think so.

Giusseppe,
when do you plan to fix it?

Comment 14 Lukas Slebodnik 2020-04-16 12:17:12 UTC
Could we revert the version of skopeo on Fedora 30?

Comment 16 Ben Cotton 2020-04-30 20:11:51 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 17 Fedora Update System 2020-05-07 18:04:41 UTC
FEDORA-2020-948a6a30da has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-948a6a30da

Comment 18 Fedora Update System 2020-05-08 04:52:30 UTC
FEDORA-2020-948a6a30da has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-948a6a30da`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-948a6a30da

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2020-05-16 04:20:50 UTC
FEDORA-2020-948a6a30da has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.