Bug 2011903
Summary: | vsphere-problem-detector: session leak | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Robert Bost <rbost> |
Component: | Storage | Assignee: | Robert Bost <rbost> |
Storage sub component: | Operators | QA Contact: | Wei Duan <wduan> |
Status: | CLOSED ERRATA | Docs Contact: | |
Severity: | high | ||
Priority: | high | CC: | aos-bugs, gcharot, jdobson, jsafrane |
Version: | 4.10 | ||
Target Milestone: | --- | ||
Target Release: | 4.10.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-03-10 16:17:59 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 2033733 |
Description
Robert Bost
2021-10-07 16:50:58 UTC
I did some test in vsphere6.7 which we have admin access to check the session with the nightly 4.10.0-0.nightly-2021-10-16-173656. 1. In the running cluster, I did not see the vsphere-problem-detector session alive: $ govc session.ls | wc -l 25 $ govc session.ls | grep vsphere-prob | wc -l 0 2. Using the command "while true; do govc session.ls | grep vsphere-prob;done" during restarting vsphere-problem-detector pod, I still could not see the alive vsphere-problem-detector session. From my side, it seems no session leak any more. @Robert What do you think? BTW, where do you find this issue? I understand it is not in VMC as you could check the session in the bug description. " $ while true; do date; govc session.ls | grep vsphere-prob; sleep 5m ; done Thu Oct 7 10:20:26 AM MDT 2021 523e179e-4ead-fd7b-b51a-4f51ad4789a0 VSPHERE.LOCAL\rbost 2021-10-07 16:20 12s x.x.x.x vsphere-problem-detector/v0.0.0-unknown " I'm wondering if you could help verify on your env (no need on VMC) to double confirm. Thanks. > $ govc session.ls | grep vsphere-prob | wc -l The `grep vsphere-prob` would only be correct if https://github.com/openshift/vsphere-problem-detector/pull/58 was merged or you had a build containing that change. Otherwise, the user agents will not be set as you would expect. > BTW, where do you find this issue? I understand it is not in VMC as you could check the session in the bug description. I was testing in a vSphere installation in IBM Cloud where full vSphere admin privileges are given and happy to test there again, however.. This issue may be blocked until https://github.com/openshift/vsphere-problem-detector/pull/58 can be merged otherwise we cannot really isolate what is vsphere-problem-detector and what is not. @Robert Thanks for the info. With regards to https://issues.redhat.com/browse/RFE-2123, I suggest we backport it everywhere we have vsphere-problem-detector installed by default. This is a report from 4.7: https://access.redhat.com/support/cases/#/case/02988444 - they fixed their credentials and it's closed, but it looks related. Verified pass on 4.10.0-0.nightly-2021-12-06-201335 1. Check on the cluster after running for a while, no session from vsphere-problem-detector $ govc session.ls | grep vsphere-prob | wc -l 0 2. Using following command to monitor the session and restart the vsphere-problem-detector-operator pod, we see new session launched and releases immediately $ while true; do date; govc session.ls | grep vsphere-prob; done $ oc -n openshift-cluster-storage-operator delete pod vsphere-problem-detector-operator-8794689bc-kqm7l (restart vsphere-problem-detector-operator pod) Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC 5227b896-edbb-5049-6544-327c122e5689 VSPHERE.LOCAL\openshift-qe-machineset 2021-12-14 02:29 0s 10.8.30.2 vsphere-problem-detector/4.10.0-202111222329.p0.gbda2d97.assembly.stream-bda2d97 Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Tue 14 Dec 2021 02:25:32 AM UTC Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056 |