Bug 1754170

Summary:	The crio command fails with: 'Invalid character(s) found in prerelease "module_f30"'
Product:	[Fedora] Fedora	Reporter:	Tristan Cacqueray <tdecacqu>
Component:	cri-o	Assignee:	Lokesh Mandvekar <lsm5>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	30	CC:	akaris, amurdaca, dwalsh, fkluknav, gabibbo97, jpazdziora, kas, lsm5, santiago, spaced.wombat, vrutkovs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	cri-o-1.14-3020200226174119.a5b0195c cri-o-1.14-3120200226174119.f636be4b	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-06 02:25:32 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1685454, 1713395

Description Tristan Cacqueray 2019-09-21 13:09:14 UTC

Description of problem:

The crio command fails early with this error message:

FATA[2019-09-21 02:24:05.993597312Z] Invalid character(s) found in prerelease "module_f30"

Version-Release number of selected component (if applicable):
cri-o-2:1.14.9-1.module_f30+5132+c35a5687.x86_64 from the cri-o module 1.14

How reproducible:
Always

Steps to Reproduce:
1. dnf module install cri-o:1.14
2. dnf install cri-o
3. systemctl start crio

Actual results:
Sep 21 02:23:56 fedora systemd[1]: Starting Open Container Initiative Daemon...
Sep 21 02:23:56 fedora crio[4246]: time="2019-09-21 02:23:56.336893978Z" level=fatal msg="Invalid character(s) found in prerelease "module_f30""

Expected results:
The service to start

Additional info:
The 1.13 stream doesn't work either, the service start but then crictl can't connect:
FATA[2019-09-21T02:32:53.476989555Z] failed to connect: failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded

Comment 1 Tristan Cacqueray 2019-09-21 13:12:44 UTC

Debug output:

# /bin/crio --log-level debug
DEBU[2019-09-21 13:11:39.306458782Z] found valid runtime 'runc' for runtime_path '/usr/bin/runc'
 
DEBU[2019-09-21 13:11:39.307647331Z] [graphdriver] trying provided driver "overlay" 
DEBU[2019-09-21 13:11:39.390859262Z] overlay test mount with multiple lowers succeeded 
DEBU[2019-09-21 13:11:39.539846944Z] overlay test mount indicated that metacopy is being used 
WARN[2019-09-21 13:11:39.547991992Z] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[2019-09-21 13:11:39.548039364Z] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
INFO[2019-09-21 13:11:39.548993395Z] Found CNI network crio-bridge (type=bridge) at /etc/cni/net.d/100-crio-bridge.conf 
INFO[2019-09-21 13:11:39.549144796Z] Found CNI network 200-loopback.conf (type=loopback) at /etc/cni/net.d/200-loopback.conf 
INFO[2019-09-21 13:11:39.549248736Z] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist 
DEBU[2019-09-21 13:11:39.555921299Z] seccomp status: true                         
DEBU[2019-09-21 13:11:39.556649215Z] Golang's threads limit set to 115200         
DEBU[2019-09-21 13:11:39.558651864Z] sandboxes: []                                
FATA[2019-09-21 13:11:39.558695681Z] Invalid character(s) found in prerelease "module_f30" 

# /bin/crio --version
crio version 1.14.9-1.module_f30+5132+c35a5687

Comment 2 Jan Pazdziora 2019-10-17 10:24:59 UTC

What is the plan for CRI-O on Fedora 30? The cri-o-1.14.9-1.module_f30+5132+c35a5687 is the latest 1.14 build in Koji for Fedora 30. I can see cri-o-1.15.2-1.module_f30+6830+9be88745 but I cannot enabled that module:

# dnf module install -y cri-o:1.15
Fedora Modular 30 - x86_64                      2.9 MB/s | 2.7 MB     00:00    
Fedora Modular 30 - x86_64 - Updates            5.6 MB/s | 4.0 MB     00:00    
Fedora 30 - x86_64 - Updates                     18 MB/s |  21 MB     00:01    
Fedora 30 - x86_64                               23 MB/s |  70 MB     00:03    
Error: Problems in request:
missing groups or modules: cri-o:1.15

Comment 3 Lokesh Mandvekar 2019-10-17 12:53:01 UTC

(In reply to Jan Pazdziora from comment #2)
> What is the plan for CRI-O on Fedora 30? The
> cri-o-1.14.9-1.module_f30+5132+c35a5687 is the latest 1.14 build in Koji for
> Fedora 30. I can see cri-o-1.15.2-1.module_f30+6830+9be88745 but I cannot
> enabled that module:
> 
> # dnf module install -y cri-o:1.15
> Fedora Modular 30 - x86_64                      2.9 MB/s | 2.7 MB     00:00 
> 
> Fedora Modular 30 - x86_64 - Updates            5.6 MB/s | 4.0 MB     00:00 
> 
> Fedora 30 - x86_64 - Updates                     18 MB/s |  21 MB     00:01 
> 
> Fedora 30 - x86_64                               23 MB/s |  70 MB     00:03 
> 
> Error: Problems in request:
> missing groups or modules: cri-o:1.15

I've had trouble submitting crio module update via bodhi and via cli. I guess the plan for now is to just change the module name to something else because there's both a cri-o rpm and module, which causes strange behaviour.

Comment 4 Lokesh Mandvekar 2019-10-17 13:02:41 UTC

(In reply to Jan Pazdziora from comment #2)
> What is the plan for CRI-O on Fedora 30? The
> cri-o-1.14.9-1.module_f30+5132+c35a5687 is the latest 1.14 build in Koji for
> Fedora 30. I can see cri-o-1.15.2-1.module_f30+6830+9be88745 but I cannot
> enabled that module:
> 
> # dnf module install -y cri-o:1.15
> Fedora Modular 30 - x86_64                      2.9 MB/s | 2.7 MB     00:00 
> 
> Fedora Modular 30 - x86_64 - Updates            5.6 MB/s | 4.0 MB     00:00 
> 
> Fedora 30 - x86_64 - Updates                     18 MB/s |  21 MB     00:01 
> 
> Fedora 30 - x86_64                               23 MB/s |  70 MB     00:03 
> 
> Error: Problems in request:
> missing groups or modules: cri-o:1.15

Btw, would you be interested in co-maintaining this, so that I'm not a bottleneck? :)

Comment 5 Jan Pazdziora 2019-10-17 13:34:19 UTC

(In reply to Lokesh Mandvekar from comment #4)
> 
> Btw, would you be interested in co-maintaining this, so that I'm not a
> bottleneck? :)

No, sorry. I'm merely a user here, currently too away from the actual CRI-O development.

Comment 6 Jan Pazdziora 2019-10-18 10:14:31 UTC

Also reported in bug 1743017.

Comment 7 Jan Pazdziora 2019-11-01 13:35:12 UTC

Also reported in bug 1740730.

Comment 8 Andreas Karis 2019-11-05 00:01:17 UTC

It's the same with 1.16 and fedora 31:

[root@vm2 ~]# /usr/bin/crio
FATA[2019-11-04 23:55:01.343159428Z] Invalid character(s) found in prerelease "module_f31" 
[root@vm2 ~]# cat /etc/redhat-release 
Fedora release 31 (Thirty One)
[root@vm2 ~]# 
[root@vm2 ~]# 
[root@vm2 ~]# /usr/bin/crio --log-level debug
DEBU[2019-11-04 23:57:35.936357766Z] using runtime executable from $PATH "/usr/bin/runc"  file="config/config.go:797"
DEBU[2019-11-04 23:57:35.936436229Z] found valid runtime "runc" for runtime_path "/usr/bin/runc"  file="config/config.go:803"
INFO[2019-11-04 23:57:35.936763216Z] using conmon executable "/usr/libexec/crio/conmon"  file="config/config.go:734"
DEBU[2019-11-04 23:57:35.936988247Z] [graphdriver] trying provided driver "overlay"  file="drivers/driver.go:244"
DEBU[2019-11-04 23:57:35.937053567Z] cached value indicated that overlay is supported  file="overlay/overlay.go:176"
DEBU[2019-11-04 23:57:35.937123706Z] cached value indicated that metacopy is being used  file="overlay/overlay.go:208"
DEBU[2019-11-04 23:57:35.937571168Z] cached value indicated that native-diff is not being used  file="overlay/overlay.go:456"
WARN[2019-11-04 23:57:35.937606774Z] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled  file="overlay/overlay.go:457"
DEBU[2019-11-04 23:57:35.937628666Z] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true  file="overlay/overlay.go:270"
INFO[2019-11-04 23:57:35.937982495Z] Found CNI network crio-bridge (type=bridge) at /etc/cni/net.d/100-crio-bridge.conf  file="ocicni/ocicni.go:291"
INFO[2019-11-04 23:57:35.938053801Z] Found CNI network 200-loopback.conf (type=loopback) at /etc/cni/net.d/200-loopback.conf  file="ocicni/ocicni.go:291"
INFO[2019-11-04 23:57:35.938189190Z] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist  file="ocicni/ocicni.go:291"
INFO[2019-11-04 23:57:35.940200964Z] no seccomp profile specified, using the internal default  file="server/server.go:412"
INFO[2019-11-04 23:57:35.940265200Z] assuming user-provided apparmor profile: crio-default-1.16.0-0.1.rc1.git6a4b481.module_f31+6837+7e34758d-rc1  file="server/server.go:436"
DEBU[2019-11-04 23:57:35.940322302Z] Golang's threads limit set to 14040           file="server/server.go:311"
DEBU[2019-11-04 23:57:35.940806227Z] sandboxes: []                                 file="server/server.go:494"
DEBU[2019-11-04 23:57:35.940941928Z] registered SIGHUP watcher for file "/etc/crio/crio.conf"  file="server/server.go:736"
DEBU[2019-11-04 23:57:35.941042273Z] registered SIGHUP watcher for file "/etc/containers/registries.conf"  file="server/server.go:736"
FATA[2019-11-04 23:57:35.941153303Z] Invalid character(s) found in prerelease "module_f31"  file="crio/main.go:221"
[root@vm2 ~]# dnf module info cri-o
Last metadata expiration check: 0:25:53 ago on Mon 04 Nov 2019 11:34:56 PM UTC.
Name         : cri-o
Stream       : 1.11
Version      : 3120190813194440
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : fedora-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.11.15-1.module_f31+5343+99937626.src
             : cri-o-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-o-debuginfo-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-o-debugsource-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-tools-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.src
             : cri-tools-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64
             : cri-tools-debuginfo-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64
             : cri-tools-debugsource-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64

Name         : cri-o
Stream       : 1.11
Version      : 3120190813194440
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : updates-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.11.15-1.module_f31+5343+99937626.src
             : cri-o-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-o-debuginfo-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-o-debugsource-2:1.11.15-1.module_f31+5343+99937626.x86_64
             : cri-tools-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.src
             : cri-tools-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64
             : cri-tools-debuginfo-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64
             : cri-tools-debugsource-0:1.11.1-2.gitedabfb5.module_f31+5343+99937626.x86_64

Name         : cri-o
Stream       : 1.12
Version      : 3120190720195307
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : fedora-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.12.10-1.module_f31+5127+76f0ca40.src
             : cri-o-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-o-debuginfo-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-o-debugsource-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-tools-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.src
             : cri-tools-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64
             : cri-tools-debuginfo-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64
             : cri-tools-debugsource-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64

Name         : cri-o
Stream       : 1.12
Version      : 3120190720195307
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : updates-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.12.10-1.module_f31+5127+76f0ca40.src
             : cri-o-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-o-debuginfo-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-o-debugsource-2:1.12.10-1.module_f31+5127+76f0ca40.x86_64
             : cri-tools-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.src
             : cri-tools-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64
             : cri-tools-debuginfo-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64
             : cri-tools-debugsource-0:1.12.0-1.gite4dff19.module_f31+5127+76f0ca40.x86_64

Name         : cri-o
Stream       : 1.14
Version      : 3120190720192718
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : fedora-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.14.9-1.module_f31+5133+e8343ec4.src
             : cri-o-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-o-debuginfo-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-o-debugsource-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-0:1.14.0-1.module_f31+5133+e8343ec4.src
             : cri-tools-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-debuginfo-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-debugsource-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64

Name         : cri-o
Stream       : 1.14
Version      : 3120190720192718
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : updates-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.14.9-1.module_f31+5133+e8343ec4.src
             : cri-o-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-o-debuginfo-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-o-debugsource-2:1.14.9-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-0:1.14.0-1.module_f31+5133+e8343ec4.src
             : cri-tools-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-debuginfo-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64
             : cri-tools-debugsource-0:1.14.0-1.module_f31+5133+e8343ec4.x86_64

Name         : cri-o
Stream       : 1.16 [e] [a]
Version      : 3120191022102649
Context      : f636be4b
Architecture : x86_64
Profiles     : default
Repo         : updates-modular
Summary      : Kubernetes Container Runtime Interface for OCI-based containers
Description  : CRI-O is the Kubernetes Container Runtime Interface for OCI-based containers.
Artifacts    : cri-o-2:1.16.0-0.1.rc1.git6a4b481.module_f31+6837+7e34758d.src
             : cri-o-2:1.16.0-0.1.rc1.git6a4b481.module_f31+6837+7e34758d.x86_64
             : cri-tools-0:1.16.1-1.module_f31+6837+7e34758d.src
             : cri-tools-0:1.16.1-1.module_f31+6837+7e34758d.x86_64
             : cri-tools-debuginfo-0:1.16.1-1.module_f31+6837+7e34758d.x86_64
             : cri-tools-debugsource-0:1.16.1-1.module_f31+6837+7e34758d.x86_64

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled, [a]ctive
[root@vm2 ~]#

Comment 9 Fedora Update System 2019-11-19 15:42:20 UTC

FEDORA-MODULAR-2019-749fd1db1f has been submitted as an update to Fedora 30 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-749fd1db1f

Comment 10 Fedora Update System 2019-11-19 15:42:35 UTC

FEDORA-MODULAR-2019-e3c905831f has been submitted as an update to Fedora 31 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-e3c905831f

Comment 11 Fedora Update System 2019-11-20 04:36:27 UTC

cri-o-1.16-3020191115203826.a5b0195c has been pushed to the Fedora 30 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-749fd1db1f

Comment 12 Fedora Update System 2019-11-20 04:39:37 UTC

cri-o-1.16-3120191115203826.f636be4b has been pushed to the Fedora 31 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-e3c905831f

Comment 13 Jan Pazdziora 2019-11-25 08:44:12 UTC

Upgrading from cri-o-1.16.0-0.1.rc1.git6a4b481.module_f31+6837+7e34758d.x86_64 to ccri-o-1.16.0-0.4.rc2.module_f31+7054+4022d60f.x86_64 does not fix the problem:

Nov 25 03:39:32 machine.example.test systemd[1]: Starting Container Runtime Interface for OCI (CRI-O)...
Nov 25 03:39:32 machine.example.test crio[10053]: time="2019-11-25 03:39:32.135457557-05:00" level=fatal msg="Invalid character(s) found in build meta data \"7054+4022d60f-rc2\""
Nov 25 03:39:32 machine.example.test systemd[1]: crio.service: Main process exited, code=exited, status=1/FAILURE
Nov 25 03:39:32 machine.example.test systemd[1]: crio.service: Failed with result 'exit-code'.
Nov 25 03:39:32 machine.example.test systemd[1]: Failed to start Container Runtime Interface for OCI (CRI-O).

Comment 14 Fedora Update System 2019-12-04 16:08:30 UTC

FEDORA-MODULAR-2019-a955aebdcf has been submitted as an update to Fedora 30 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-a955aebdcf

Comment 15 Fedora Update System 2019-12-04 16:08:53 UTC

FEDORA-MODULAR-2019-24636b4ff9 has been submitted as an update to Fedora 31 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-24636b4ff9

Comment 16 Fedora Update System 2019-12-05 15:03:54 UTC

cri-o-1.16-3120191204144621.f636be4b has been pushed to the Fedora 31 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-24636b4ff9

Comment 17 Adi Stadelmann 2020-01-21 21:19:14 UTC

may i ask you kindly to push 1.16.1 to stable for fedora 31, so a fix for Bug 1772280 is at least available?

Comment 18 Fedora Update System 2020-01-22 02:10:59 UTC

cri-o-1.16-3120191204144621.f636be4b has been pushed to the Fedora 31 Modular stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2020-02-26 18:19:54 UTC

FEDORA-MODULAR-2020-c230523214 has been submitted as an update to Fedora 30 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2020-c230523214

Comment 20 Fedora Update System 2020-02-26 19:51:25 UTC

FEDORA-MODULAR-2020-36f52d4e8f has been submitted as an update to Fedora 31 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2020-36f52d4e8f

Comment 21 Fedora Update System 2020-02-27 18:45:31 UTC

cri-o-1.14-3020200226174119.a5b0195c has been pushed to the Fedora 30 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2020-c230523214

Comment 22 Fedora Update System 2020-02-27 18:49:05 UTC

cri-o-1.14-3120200226174119.f636be4b has been pushed to the Fedora 31 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2020-36f52d4e8f

Comment 23 Fedora Update System 2020-03-06 02:25:32 UTC

cri-o-1.14-3020200226174119.a5b0195c has been pushed to the Fedora 30 Modular stable repository. If problems still persist, please make note of it in this bug report.

Comment 24 Fedora Update System 2020-03-06 02:41:21 UTC

cri-o-1.14-3120200226174119.f636be4b has been pushed to the Fedora 31 Modular stable repository. If problems still persist, please make note of it in this bug report.