Bug 2151817 - [CNV-4.11.2] Failing to import DataVolumes
Summary: [CNV-4.11.2] Failing to import DataVolumes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Storage
Version: 4.11.2
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 4.11.2
Assignee: Maya Rashish
QA Contact: Natalie Gavrielov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-12-08 08:12 UTC by Mor Cohen
Modified: 2023-01-12 14:08 UTC (History)
3 users (show)

Fixed In Version: 4.11.2-6
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-01-12 14:08:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker CNV-23270 0 None None None 2022-12-08 08:27:40 UTC
Red Hat Product Errata RHEA-2023:0155 0 None None None 2023-01-12 14:08:59 UTC

Description Mor Cohen 2022-12-08 08:12:56 UTC
Description of problem:
After deploying CNV-4.11.2 DataVolumes are failing to get imported, stuck on the ImportInProgress phase. 



Version-Release number of selected component (if applicable):
CNV-4.11.2-4

How reproducible:
100%

Steps to Reproduce:
1. Deploy CNV from the 388386 IIB.
2.
3.

Actual results:


Expected results:


Additional info:
[cnv-qe-jenkins@c01-vrf-t1-411-xd655-executor ~]$ oc get dv -A
NAMESPACE                            NAME                          PHASE              PROGRESS   RESTARTS   AGE
openshift-virtualization-os-images   centos-stream8-2f16c067b974   ImportInProgress   N/A        159        14h
openshift-virtualization-os-images   centos-stream9-b603efbe0295   ImportInProgress   N/A        160        14h
openshift-virtualization-os-images   centos7-680e9b4e0fba          ImportInProgress   N/A        159        14h
openshift-virtualization-os-images   fedora-56ccabc01cbe           ImportInProgress   N/A        161        14h
openshift-virtualization-os-images   rhel8-13b024fed1a3            ImportInProgress   N/A        161        14h
openshift-virtualization-os-images   rhel9-35d9b2336799            ImportInProgress   N/A        159        14h

[cnv-qe-jenkins@c01-vrf-t1-411-xd655-executor ~]$ oc get pods -A | grep -v Running | grep -v Completed
NAMESPACE                                          NAME                                                              READY   STATUS             RESTARTS          AGE
openshift-virtualization-os-images                 importer-centos-stream8-2f16c067b974                              1/2     CrashLoopBackOff   160 (2m1s ago)    14h
openshift-virtualization-os-images                 importer-centos-stream9-b603efbe0295                              1/2     CrashLoopBackOff   161 (117s ago)    14h
openshift-virtualization-os-images                 importer-fedora-56ccabc01cbe                                      1/2     CrashLoopBackOff   162 (2m52s ago)   14h
openshift-virtualization-os-images                 importer-rhel8-13b024fed1a3                                       1/2     CrashLoopBackOff   162 (3m31s ago)   14h
openshift-virtualization-os-images                 importer-rhel9-35d9b2336799                                       1/2     CrashLoopBackOff   160 (2m45s ago)   14h


$ oc logs -n openshift-virtualization-os-images importer-centos-stream8-2f16c067b974
Defaulted container "importer" out of: importer, server, init (init)
I1208 07:58:43.215458       1 importer.go:83] Starting importer
I1208 07:58:43.215506       1 importer.go:150] begin import process
I1208 07:58:43.219728       1 data-processor.go:379] Calculating available size
I1208 07:58:43.219747       1 data-processor.go:391] Checking out file system volume size.
I1208 07:58:43.219763       1 data-processor.go:399] Request image size not empty.
I1208 07:58:43.219770       1 data-processor.go:404] Target size 34087042032.
I1208 07:58:43.220339       1 nbdkit.go:296] Waiting for nbdkit PID.
I1208 07:58:43.400808       1 nbdkit.go:283] Log line from nbdkit: /var/run/nbdkit.sock: Permission denied
I1208 07:58:43.401560       1 nbdkit.go:289] Stopped watching nbdkit log.
E1208 07:58:58.221381       1 nbdkit.go:264] Failed waiting for nbdkit to start up: timed out waiting for nbdkit to be ready
E1208 07:58:58.221516       1 data-processor.go:278] timed out waiting for nbdkit to be ready
kubevirt.io/containerized-data-importer/pkg/image.waitForNbd
	/remote-source/app/pkg/image/nbdkit.go:321
kubevirt.io/containerized-data-importer/pkg/image.(*Nbdkit).StartNbdkit
	/remote-source/app/pkg/image/nbdkit.go:262
kubevirt.io/containerized-data-importer/pkg/importer.(*HTTPDataSource).Info
	/remote-source/app/pkg/importer/http-datasource.go:147
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).initDefaultPhases.func1
	/remote-source/app/pkg/importer/data-processor.go:201
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).ProcessDataWithPause
	/remote-source/app/pkg/importer/data-processor.go:275
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).ProcessData
	/remote-source/app/pkg/importer/data-processor.go:184
main.handleImport
	/remote-source/app/cmd/cdi-importer/importer.go:156
main.main
	/remote-source/app/cmd/cdi-importer/importer.go:122
runtime.main
	/usr/lib/golang/src/runtime/proc.go:250
runtime.goexit
	/usr/lib/golang/src/runtime/asm_amd64.s:1594
Unable to obtain information about data source

Comment 1 Maya Rashish 2022-12-08 09:07:59 UTC
Sorry, this is likely due to a partial backport (of PSA fixes) that we also decided we don't want in 4.11.
I didn't realize it is also broken, I am reverting it.

Comment 2 Maya Rashish 2022-12-08 11:46:29 UTC
Let me know if it was this change that broke things, and the revert is sufficient.

Comment 3 Mor Cohen 2022-12-08 11:49:18 UTC
Hey Maya, the revert is sufficient, thanks!

[cnv-qe-jenkins@c01-vrf-t1-411-xd655-executor ~]$ oc get dv -A
NAMESPACE                            NAME                          PHASE       PROGRESS   RESTARTS   AGE
openshift-virtualization-os-images   centos-stream8-2f16c067b974   Succeeded   100.0%                6m12s
openshift-virtualization-os-images   centos-stream9-250b87464d0d   Succeeded   100.0%                6m13s
openshift-virtualization-os-images   centos7-680e9b4e0fba          Succeeded   100.0%                6m12s
openshift-virtualization-os-images   fedora-0e5ac38b20ab           Succeeded   100.0%                3m24s
openshift-virtualization-os-images   fedora-56ccabc01cbe           Succeeded   100.0%                6m13s


Thanks!

Comment 11 errata-xmlrpc 2023-01-12 14:08:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.11.2 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:0155


Note You need to log in before you can comment on or make changes to this bug.