Bug 1824596

Summary:	PreStop hook fails in csi-hostpath driver pod used in tests
Product:	OpenShift Container Platform	Reporter:	Jan Safranek <jsafrane>
Component:	Storage	Assignee:	Jan Safranek <jsafrane>
Status:	CLOSED ERRATA	QA Contact:	Qin Ping <piqin>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.5	CC:	aos-bugs
Target Milestone:	---
Target Release:	4.5.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-07-13 17:27:59 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jan Safranek 2020-04-16 11:03:38 UTC

Description of problem:

See https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_cluster-etcd-operator/312/pull-ci-openshift-cluster-etcd-operator-master-e2e-gcp/1175

Apr 16 02:47:54.240 W ns/e2e-volumemode-7208 pod/csi-hostpathplugin-0 node/ci-op-8lmbm-w-b-g294t.c.openshift-gce-devel-ci.internal reason/FailedPreStopHook Exec lifecycle hook ([/bin/sh -c rm -rf /registration/csi-hostpath /registration/csi-hostpath-reg.sock]) for Container "node-driver-registrar" in Pod "csi-hostpathplugin-0_e2e-volumemode-7208(c3a0be58-f762-4513-ad18-e4e8d183d547)" failed - error: rpc error: code = Unknown desc = command error: time="2020-04-16T02:47:54Z" level=error msg="exec failed: container_linux.go:349: starting container process caused \"exec: \\\"/bin/sh\\\": stat /bin/sh: no such file or directory\""\nexec failed: container_linux.go:349: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory"\n, stdout: , stderr: , exit code -1, message: ""

A CSI tests is cleaning up and deletes csi-hostpath driver pod. The driver pod has PreStop hook to delete driver's socket, however, there is no /bin/sh in the node driver registrar image (we're using distroless) and the hook fails.

The test itself succeeds, however, logs are full of hard to understand errors, which complicates debugging of other failures.

Version-Release number of selected component (if applicable):
4.5

Comment 1 Jan Safranek 2020-04-16 13:11:12 UTC

Note: this is e2e tests we got from upstream, i.e. it uses upstream node registrar image that is distroless.

Our OCP images are based on RHEL and they have a shell.

Comment 2 Jan Safranek 2020-04-16 13:19:38 UTC

Upstream PR: https://github.com/kubernetes-csi/csi-driver-host-path/pull/176

Comment 3 Jan Safranek 2020-04-17 07:57:53 UTC

"backport" to kubernetes/kubernetes: https://github.com/kubernetes/kubernetes/pull/90234

Comment 6 Qin Ping 2020-05-09 12:16:19 UTC

Checked in this e2e log: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_cluster-etcd-operator/338/pull-ci-openshift-cluster-etcd-operator-master-e2e-gcp/1312/artifacts/e2e-gcp/e2e.log

Did not find this issue, so marked this bug as verified.

Comment 7 errata-xmlrpc 2020-07-13 17:27:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409