Bug 1879381
| Summary: | [cnv deployment] cdi-apiserver cannot start due to missing libdevmapper | ||
|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Tareq Alayan <talayan> |
| Component: | Storage | Assignee: | Adam Litke <alitke> |
| Status: | CLOSED ERRATA | QA Contact: | Ying Cui <ycui> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 2.5.0 | CC: | alitke, awels, cnv-qe-bugs, fdeutsch, lbednar, mrashish, ocohen, stirabos, yturgema |
| Target Milestone: | --- | Keywords: | Automation, AutomationBlocker, Regression, TestBlocker |
| Target Release: | 2.5.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | virt-cdi-apiserver-container-v2.5.0-13 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-17 13:24:24 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Tareq Alayan
2020-09-16 07:05:51 UTC
The root cause is: [cloud-user@ocp-psi-executor ~]$ oc logs -n openshift-cnv cdi-apiserver-6dbc9f557c-hkjk8 /usr/bin/virt-cdi-apiserver: error while loading shared libraries: libdevmapper.so.1.02: cannot open shared object file: No such file or directory Please notice that CDI operator in upstream release 1.23.1 is not able to start due to:
{"level":"error","ts":1600241122.6585944,"logger":"cmd","msg":"","error":"cdis.cdi.kubevirt.io \"cdi\" not found","stacktrace":"kubevirt.io/containerized-data-importer/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\tvendor/github.com/go-logr/zapr/zapr.go:128\nmain.main\n\tcmd/cdi-operator/operator.go:109\nruntime.main\n\tGOROOT/src/runtime/proc.go:203"}
https://github.com/kubevirt/containerized-data-importer/pull/1374
should address it.
We fear that the issue is still not visible downstream just because it's shadowed by this one.
Multiple people are working on this (waiting on a pull request from mhenriks+awels and alitke to update downstream.) Still reproducible with HCO-v2.5.0-286 and IIB-12395 HCO-v2.5.0-286 & IIB-12395 pulled following images: registry.redhat.io/container-native-virtualization/virt-cdi-apiserver@sha256:01dfc2265a8ef701d1cb7d2070b4d9e15d5f6581f578c945af6f91421ae946cf registry.redhat.io/container-native-virtualization/virt-cdi-controller@sha256:e29711fd8733e6641b7ea0b1afff3fc9078833271d0cbdaff6d27fd160e28cfe registry.redhat.io/container-native-virtualization/virt-cdi-operator@sha256:3668f59d1ff7ed7c1f5f88cc72138d7766d7b4784b671c30da7cfc8644f83f92 registry.redhat.io/container-native-virtualization/virt-cdi-uploadproxy@sha256:e3f04555f20029c6cbde357d495d11e27430126dd3f47a743532fcaf94f59613 Note that these addresses are redirected by imageContentSourcePolicy to: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-* We determined what was pulling in libdevmapper, and figured out a way to not have it included at all. In the comments on this bugzilla, I see two different issues, for which we have PRs to fix them. 1. dev mapper being needed. 2. Comment #3 where the operator fails to start when deployed through OLM. The problem is use looking for the CR when it is not created yet. These are two separate issues unrelated to each other. We have fixed both, just going through the motions to create a release with them included. CDI upstream v1.23.2 contains the fix. Still need downstream builds. working in HCO-v2.5.0-193 cdi-apiserver-f5fb5fc84-jh6qq 1/1 Running 0 23m cdi-deployment-845766b7f5-5kqtv 1/1 Running 0 23m cdi-operator-f96b47b57-qx9bg 1/1 Running 0 23m cdi-uploadproxy-687c79ff74-4qkpm 1/1 Running 0 23m oc logs cdi-apiserver-7c8b77b8c5-6v5nz -nopenshift-cnv
/usr/bin/virt-cdi-apiserver: error while loading shared libraries: libdevmapper.so.1.02: cannot open shared object file: No such file or directory
We still see this issue deploying from hco-bundle-registry-container-v2.6.0-5
which points to:
- image: registry.redhat.io/container-native-virtualization/virt-cdi-apiserver@sha256:19ade5c8e0954870fb456b31738096ae6567f94c37b08589b5583a3a1bdd2a22
name: registry.redhat.io/container-native-virtualization/virt-cdi-apiserver:v2.6.0-2
...
- image: registry.redhat.io/container-native-virtualization/virt-cdi-operator@sha256:0e4dea18e66838ab4603382415047a9e293080ddf40ebc06164fcaea0913c5eb
name: registry.redhat.io/container-native-virtualization/virt-cdi-operator:v2.6.0-2
Adam, can you please trigger a rebuild for 2.6?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 2.5.0 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:5127 |