Bug 2038985
| Summary: | No feedback when HPP path is sharing host filesystem | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Alex Kalenyuk <akalenyu> | ||||||
| Component: | Storage | Assignee: | Alex Kalenyuk <akalenyu> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Jenia Peimer <jpeimer> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 4.10.0 | CC: | alitke, cnv-qe-bugs, yadu | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 4.10.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | release note | ||||||||
| Fixed In Version: | hostpath-csi-driver-rhel8 v4.10.0-31, CNV v4.10.0-651 | Doc Type: | Release Note | ||||||
| Doc Text: |
When using the hostpath provisioner (HPP) to provide storage for virtual machine disks, configure it with dedicated storage that is separate from the node's root filesystem. Otherwise, the node could run out of storage and become unschedulable. A new HPPSharingPoolPathWithOS alert now warns administrators if this misconfiguration is detected.
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2022-03-16 16:06:09 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
|
Description
Alex Kalenyuk
2022-01-10 17:04:59 UTC
Created attachment 1860101 [details]
HPP Alert screenshot
Created attachment 1860103 [details]
HPP Alert screenshot 2
Verified on CNV v4.10.0-651, hostpath-provisioner-operator v4.10.0-61
Screenshots of the UI are attached.
To see the alerting logs using CLI:
Install HPP CR that shares a path with OS:
$ cat hpp-cr-basic.yaml
apiVersion: hostpathprovisioner.kubevirt.io/v1beta1
kind: HostPathProvisioner
metadata:
name: hostpath-provisioner
spec:
imagePullPolicy: IfNotPresent
storagePools:
- name: hpp-csi-local
path: "/var/hpvolumes"
workload:
nodeSelector:
kubernetes.io/os: linux
Connect to the node:
$ oc debug node/c01-jp410-fr-pfmr4-worker-0-k6mzs
sh-4.4# chroot /host
sh-4.4#
See where the OS is located:
sh-4.4# findmnt -T /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/vda4[/ostree/deploy/rhcos/deploy/11aa5111b525d060f5b3d4badc43c7acbe9bc8d6ff2949c90f9a35287c018821.0] xfs rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsiz
See that hpp path /var/hpvolumes shares the same path as OS:
sh-4.4# findmnt -T /var/hpvolumes
TARGET SOURCE FSTYPE OPTIONS
/var /dev/vda4[/ostree/deploy/rhcos/var] xfs rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,prjquota
$ oc get pods -A | grep hostpath
openshift-cnv hostpath-provisioner-csi-7m8kb 4/4 Running 0 9m35s
openshift-cnv hostpath-provisioner-csi-hqp4b 4/4 Running 0 9m35s
openshift-cnv hostpath-provisioner-csi-q7q2j 4/4 Running 0 9m35s
openshift-cnv hostpath-provisioner-operator-5cc9cf795d-szctm 1/1 Running 0 46h
$ oc logs -n openshift-cnv hostpath-provisioner-csi-7m8kb -c hostpath-provisioner
I0209 12:06:55.675874 1 plugin.go:44] Starting Prometheus metrics endpoint server
I0209 12:06:55.676247 1 plugin.go:47] Starting new HostPathDriver, config: {kubevirt.io.hostpath-provisioner unix:///csi/csi.sock c01-jp410-fr-pfmr4-worker-0-s4xcc map[] v4.10.0 <nil>}
I0209 12:06:55.728163 1 hostpath.go:88] name: hpp-csi-local, dataDir: hpp-csi-local-data-dir/csi
I0209 12:06:55.728218 1 hostpath.go:107] Driver: kubevirt.io.hostpath-provisioner, version: v4.10.0
I0209 12:06:55.728311 1 server.go:194] Starting domain socket: unix///csi/csi.sock
I0209 12:06:55.728416 1 server.go:89] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"}
I0209 12:06:55.737740 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:07:55.746494 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:08:55.764060 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:09:55.777407 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:10:55.785761 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:11:55.802204 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:12:55.814555 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:13:55.821748 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
I0209 12:14:55.831517 1 utils.go:221] pool (hpp-csi-local, hpp-csi-local-data-dir/csi), shares path with OS which can lead to node disk pressure
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 4.10.0 Images security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0947 |