Bug 1506953

Summary: Images fedora:26 and registry.fedoraproject.org/fedora:26 are very different.
Product: [Fedora] Fedora Container Images Reporter: Jan Pazdziora (Red Hat) <jpazdziora>
Component: dockerAssignee: Adam Miller <admiller>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: adimania, admiller, amurdaca, dustymabe, dwalsh, fkluknav, gscrivan, ichavero, jcajka, jpazdziora, jzehrarnyg, lsm5, marianne, nalin, santiago, smilner, vbatts
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-29 11:46:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora (Red Hat) 2017-10-27 09:22:22 UTC
Description of problem:

I've recently used some layered image on top of fedora:26 and then I did docker pull fedora:26 and new image was pulled. Which was surprising but then I realized that the layered image actually uses registry.fedoraproject.org/fedora:26 as its base image. So I wondered if the images are the same or not.

It turns out, that already the metadata are different.

Version-Release number of selected component (if applicable):

REPOSITORY                         TAG IMAGE ID      CREATED      SIZE
registry.fedoraproject.org/fedora  26  ba857ff07853  12 days ago  231.2 MB
docker.io/fedora                   26  b0b140824a48  5 weeks ago  231.1 MB

How reproducible:

Deterministic.

Steps to Reproduce:
1. docker pull fedora:26
2. docker pull registry.fedoraproject.org/fedora:26
3. docker inspect fedora:26 > fedora-26.inspect
4. docker inspect registry.fedoraproject.org/fedora:26 > fedora-26-registry.inspect
5. diff -u fedora-26.inspect fedora-26-registry.inspect

Actual results:

--- fedora-26.inspect	2017-10-27 05:13:16.213333613 -0400
+++ fedora-26-registry.inspect	2017-10-27 05:13:30.995945085 -0400
@@ -1,18 +1,18 @@
 [
     {
-        "Id": "sha256:b0b140824a486ccc0f7968f3c6ceb6982b4b77e82ef8b4faaf2806049fc266df",
+        "Id": "sha256:ba857ff078531df1e486f6127d51d0b80ea0da690d2580ae1386929c8e66b1d3",
         "RepoTags": [
-            "docker.io/fedora:26"
+            "registry.fedoraproject.org/fedora:26"
         ],
         "RepoDigests": [
-            "docker.io/fedora@sha256:613830d906f8897c05fd737aea8c7c00e4ecb5a518eedf71958c2c110532957c"
+            "registry.fedoraproject.org/fedora@sha256:64907a529db7e1b25d76c84928e2e1a82889c92a23f73b8f16385d05705ecea6"
         ],
         "Parent": "",
-        "Comment": "",
-        "Created": "2017-09-15T23:28:22.816153183Z",
-        "Container": "325c38d9922c257e0ccfec8503e61a156fa6ddbba9633f33b3e8acdf19d1307a",
+        "Comment": "Created by Image Factory",
+        "Created": "2017-10-15T06:48:22Z",
+        "Container": "",
         "ContainerConfig": {
-            "Hostname": "325c38d9922c",
+            "Hostname": "",
             "Domainname": "",
             "User": "",
             "AttachStdin": false,
@@ -21,26 +21,17 @@
             "Tty": false,
             "OpenStdin": false,
             "StdinOnce": false,
-            "Env": [
-                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "DISTTAG=f26container",
-                "FGC=f26",
-                "FBR=f26"
-            ],
-            "Cmd": [
-                "/bin/sh",
-                "-c",
-                "#(nop) ADD file:6fef8af91ffd240db5c0e394621757fc453e5f7312e33f04d98d04fef009b078 in / "
-            ],
-            "Image": "sha256:b01a4b5daeddf89b88f00abe8271e0e10ff1d520d227f91ce9af5d10dc94aaf1",
+            "Env": null,
+            "Cmd": null,
+            "Image": "",
             "Volumes": null,
             "WorkingDir": "",
             "Entrypoint": null,
             "OnBuild": null,
-            "Labels": {}
+            "Labels": null
         },
-        "DockerVersion": "17.06.2-ce",
-        "Author": "[Adam Miller \u003cmaxamillion\u003e] [Patrick Uiterwijk \u003cpatrick\u003e]",
+        "DockerVersion": "1.10.1",
+        "Author": "",
         "Config": {
             "Hostname": "",
             "Domainname": "",
@@ -52,35 +43,40 @@
             "OpenStdin": false,
             "StdinOnce": false,
             "Env": [
-                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                 "DISTTAG=f26container",
-                "FGC=f26",
-                "FBR=f26"
+                "FGC=f26"
             ],
-            "Cmd": null,
-            "Image": "sha256:b01a4b5daeddf89b88f00abe8271e0e10ff1d520d227f91ce9af5d10dc94aaf1",
+            "Cmd": [
+                "/bin/bash"
+            ],
+            "Image": "",
             "Volumes": null,
             "WorkingDir": "",
             "Entrypoint": null,
             "OnBuild": null,
-            "Labels": null
+            "Labels": {
+                "license": "MIT",
+                "name": "fedora",
+                "vendor": "Fedora Project",
+                "version": "26"
+            }
         },
         "Architecture": "amd64",
         "Os": "linux",
-        "Size": 231069345,
-        "VirtualSize": 231069345,
+        "Size": 231161819,
+        "VirtualSize": 231161819,
         "GraphDriver": {
             "Name": "devicemapper",
             "Data": {
-                "DeviceId": "63",
-                "DeviceName": "docker-253:0-33555815-20000a9a9cf804cb30baa3124e7f998176329e6bd29ca8524e931fd1c1228fcc",
+                "DeviceId": "64",
+                "DeviceName": "docker-253:0-33555815-93683bd4f76692dfd47871a992d6bd8461a59b4d3e25c30b6d64db511e4af6f4",
                 "DeviceSize": "10737418240"
             }
         },
         "RootFS": {
             "Type": "layers",
             "Layers": [
-                "sha256:72e455bdda27d1fad7d0b6e4e2b7b10893e90576a5008e12af9bdaea0f926b1d"
+                "sha256:fab2802aeea4acad60a0d1f618c8a4101bfa1cd1f4dde65f35880d2681ad0303"
             ]
         }
     }

Expected results:

No difference because the images are the same. Or much smaller differences, for example the Cmd and Labels should be the same. And of course ideally the Layers as well.

Additional info:

Please reconcile the Fedora images found in various places.

Also, it's be good to include information about how the image was built (from which source) and how issues with those images (like this one) should be reported. The registry.fedoraproject.org/fedora:26 says

"Comment": "Created by Image Factory",

but with no indication what it is, where it lives, and what was the exact source for this base image.

Comment 1 Yu Qi Zhang 2017-10-27 18:10:37 UTC
I'm not sure who updates the fedora image in the Docker hub. CC'ing dusty since he knows the base images much better than me.

Comment 2 Adam Miller 2017-10-27 19:22:14 UTC
I'm the main point of contact for both the registry.fedoraproject.org one and the one on Docker Hub.

The reason these are different is that the way you submit Docker Hub official images is through the docker-library/official-images[0] repo on GitHub. In there you provide a Dockerfile and a rootfs. The images themselves are actually built on Docker's systems. We don't have a choice in this, that's just how its done and to update a container image you file a pull request. We have this process documented here[1], but it basically boils down to: pull the image tarball from koji, strip the metadata and extract the rootfs tarball, add to the Dockerfile, force commit to github.

On the contrary, the registry.fedoraproject.org images comes directly from Fedora systems and we simply pull the container image tarball from koji, import it to docker, tag it, and push it. (There's a script so we don't do it by hand, but that's what it does)

Ultimately the source of both originates from the image builds in koji which use image-factory[2]'s docker image plugin to produce the resulting artifact. However, we have to unpack it and push only the rootfs up to Docker Hub.

[0] - https://github.com/docker-library/official-images
[1] - https://github.com/fedora-cloud/docker-brew-fedora
[2] - http://imgfac.org/

Comment 3 Jan Pazdziora (Red Hat) 2017-10-30 07:19:43 UTC
Couldn't the docker.io/fedora be just redirect to registry.fedoraproject.org?

Comment 4 Adam Miller 2017-10-31 13:46:45 UTC
Not to the best of my knowledge, no. That's not how any of the official base images on Docker Hub are handled except for certain software vendors that Docker has contracts in place to allow such an agreement.

Comment 5 Fedora End Of Life 2018-05-03 08:21:40 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 6 Jan Pazdziora (Red Hat) 2018-05-09 09:50:22 UTC
I guess, this one is likely WONTFIX, right?

Comment 7 Fedora End Of Life 2018-05-29 11:46:50 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.