Bug 1601216

Summary: fedora:rawhide is significantly older than registry.fedoraproject.org/fedora:rawhide
Product: [Fedora] Fedora Container Images Reporter: Jan Pazdziora <jpazdziora>
Component: fedora-container-imageAssignee: Clement Verna <clems.verna>
Status: ASSIGNED --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 39CC: bcotton, clems.verna, dwalsh, fedoraproject, gscrivan, jpazdziora, jzehrarnyg, smilner
Target Milestone: ---Flags: bcotton: fedora_prioritized_bug-
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2018-07-15 08:34:19 UTC
Description of problem:

Fedora seems to maintain images both in docker.io and in registry.fedoraproject.org. For reasons explained in bug 1506953, the images cannot be exactly the same.

However, it seems the images in docker.io haven't been kept updated.

Version-Release number of selected component (if applicable):

fedora:rawhide
fedora:28

How reproducible:

Deterministic.

Steps to Reproduce:
1. To make sure we work with the latest iamges:
   for i in docker.io/fedora:rawhide registry.fedoraproject.org/fedora:rawhide ; do docker pull $i ; done
2. Check the age:
   docker images | grep rawhide
3. Check for example systemd versions:
   docker run fedora:rawhide rpm -q systemd
   docker run registry.fedoraproject.org/fedora:rawhide rpm -q systemd

Actual results:

registry.fedoraproject.org/fedora   rawhide             3ccf10bd1325        6 weeks ago         247 MB
docker.io/fedora                    rawhide             5292e27c6dac        4 months ago        242 MB

systemd-237-6.git84c8da5.fc29.x86_64

systemd-238-8.git0e0aa59.fc29.x86_64

Expected results:

If the fedora images in docker.io exist, they should be kept up-to-date with content in registry.fedoraproject.org, so their ages should differ by a couple of days max.

Additional info:

The same problem is with :28 and :27 as well:

registry.fedoraproject.org/fedora   28                  e555121ced0f        10 days ago         253 MB
docker.io/fedora                    28                  cc510acfcd70        2 months ago        253 MB

registry.fedoraproject.org/fedora   27                  cdbb563c65db        13 days ago         235 MB
docker.io/fedora                    27                  9110ae7f579f        4 months ago        235 MB

Comment 1 Ben Cotton 2020-02-11 21:36:54 UTC
Due to a gap in process, bugs filed against Fedora Container Images rawhide did not get branched with other Fedora bugs. Based on the creation date, this bug appears to have been filed during or before the Fedora 29 development cycle. If this is not correct, please update the version appropriately or set needinfo for bcotton.

Since Fedora 29 is EOL, I will close this bug in two weeks if it has not been updated to a supported (30, 31) or development (32, rawhide) version.

Comment 2 Jan Pazdziora 2020-02-12 08:53:01 UTC
The problem still exists today:

$ for i in docker.io/fedora:rawhide registry.fedoraproject.org/fedora:rawhide ; do docker pull $i ; done
Trying to pull repository docker.io/library/fedora ... 
sha256:239ba4c30c322bd601595f137a351dc42defdb3acf11ae3a07996b879df798fd: Pulling from docker.io/library/fedora
Digest: sha256:239ba4c30c322bd601595f137a351dc42defdb3acf11ae3a07996b879df798fd
Status: Image is up to date for docker.io/fedora:rawhide
Trying to pull repository registry.fedoraproject.org/fedora ... 
sha256:9e4d801869b84aba2ee1e124366152db994ec0028f028f72a8572f2f7a8c473b: Pulling from registry.fedoraproject.org/fedora
Digest: sha256:9e4d801869b84aba2ee1e124366152db994ec0028f028f72a8572f2f7a8c473b
Status: Image is up to date for registry.fedoraproject.org/fedora:rawhide

$ docker images | grep rawhide
registry.fedoraproject.org/fedora  rawhide a3689ffc0440 23 hours ago        197 MB
docker.io/fedora                   rawhide e13031c001a8 4 months ago        201 MB

$ docker run fedora:rawhide rpm -q systemd
systemd-243-2.gitfab6f01.fc32.x86_64
$ docker run registry.fedoraproject.org/fedora:rawhide rpm -q systemd
systemd-245~rc1-2.fc32.x86_64

Comment 3 Jan Pazdziora 2020-02-12 08:54:07 UTC
Ben, I'm not sure how to update the Fedora version from rawhide to rawhide to avoid the autoclosure. The problem still seems to be present on rawhide.

Comment 4 Ben Cotton 2020-02-12 14:04:52 UTC
Okay, I'll make sure I exclude it from the list (which is thankfully short enough that I can do that). I'm going to nominate it as a prioritized bug[1] as well. (I have apparently not enabled this flag for the Fedora Container Images product, so I'll do that first)

[1] https://docs.fedoraproject.org/en-US/program_management/prioritized_bugs/

Comment 5 Clement Verna 2020-02-26 18:07:49 UTC
I have open an issue in the DockerHub official images repo to open the conversation about push rawhide everyday there. You can follow the discussion here https://github.com/docker-library/official-images/issues/7529

Comment 6 Ben Cotton 2020-02-26 18:27:53 UTC
In today's Prioritized Bugs meeting, we agreed to reject this as a prioritized bug, but we will work to improve the process so that third-party registries have more up-to-date rawhide images.

Comment 7 Ben Cotton 2020-08-11 13:05:06 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 8 Daniel Walsh 2021-01-28 11:29:49 UTC
This is not a Docker bug, Docker is not supported in Fedora,  This is a container image problem, if it still exists.

Comment 9 Ben Cotton 2021-11-04 16:12:21 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Jan Pazdziora 2021-11-14 13:22:27 UTC
The issue is still present today. After doing

$ podman pull registry.fedoraproject.org/fedora:rawhide
Trying to pull registry.fedoraproject.org/fedora:rawhide...
Getting image source signatures
Copying blob de6dc67518ad done  
Copying config f180c29b3e done  
Writing manifest to image destination
Storing signatures
f180c29b3e23761c76fd659f55d3b04b96c189d70cc0e1f23e3b029bdac56e11

$ podman pull docker.io/fedora:rawhide
Trying to pull docker.io/library/fedora:rawhide...
Getting image source signatures
Copying blob a029976c022b done  
Copying config 1cf0d0c364 done  
Writing manifest to image destination
Storing signatures
1cf0d0c364a631bc7dd12c8a1f59d01e4ccca6ffece5bd504e1567d057de92e6

I get

$ podman images rawhide
REPOSITORY                            TAG             IMAGE ID      CREATED      SIZE
registry.fedoraproject.org/fedora     rawhide         f180c29b3e23  5 hours ago  166 MB
docker.io/library/fedora              rawhide         1cf0d0c364a6  11 days ago  166 MB

Comment 11 Ben Cotton 2022-02-08 21:28:05 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 12 Ben Cotton 2023-04-25 18:21:52 UTC
This message is a reminder that Fedora Linux 36 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '36'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 36 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 13 Jan Pazdziora 2023-04-25 19:25:17 UTC
The significant delay is still there today:

$ podman pull registry.fedoraproject.org/fedora:rawhide
Trying to pull registry.fedoraproject.org/fedora:rawhide...
Getting image source signatures
Copying blob 1a542e04e1a9 done  
Copying config 5301679403 done  
Writing manifest to image destination
Storing signatures
5301679403a9674800c41d65103ee293ba6dcf1ac8e754c528e515b65fe6c5d8
$ podman pull docker.io/fedora:rawhide
Trying to pull docker.io/library/fedora:rawhide...
Getting image source signatures
Copying blob 629916c30b0c done  
Copying config 8797889d87 done  
Writing manifest to image destination
Storing signatures
8797889d87bbd29f56983e5670c58d3a7c7b3aeb58e5b6314c6528e2a3460229

$ podman images | grep rawhide
registry.fedoraproject.org/fedora  rawhide     5301679403a9  10 hours ago  197 MB
docker.io/library/fedora           rawhide     8797889d87bb  5 days ago    197 MB

Comment 14 Fedora Release Engineering 2023-08-16 07:04:16 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.