Bug 1438678
| Summary: | vdsm-client returns - Exception in thread Client localhost:54321 (most likely raised during interpreter shutdown) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Michael Burman <mburman> | ||||||||
| Component: | vdsm | Assignee: | Marcin Sobczyk <msobczyk> | ||||||||
| Status: | CLOSED DEFERRED | QA Contact: | Lucie Leistnerova <lleistne> | ||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 4.2.0 | CC: | bugs, igoihman, lleistne, lsurette, lsvaty, mperina, nsednev, pkliczew, schandle, srevivo, ycui | ||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2022-01-19 12:44:59 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
Michael Burman
2017-04-04 07:56:50 UTC
It's a timing issue which I'm finding it hard to reproduce. Please set vdsm log level to DEBUG and attach to the bug. Created attachment 1270170 [details]
vdsm log
Managed to reproduce it, this is the full error: :Exception in thread Client localhost:54321 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner File "/usr/lib64/python2.7/threading.py", line 764, in run File "/usr/lib/python2.7/site-packages/vdsm/concurrent.py", line 185, in run File "/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py", line 222, in process_requests File "/usr/lib64/python2.7/asyncore.py", line 220, in loop File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2 <type 'exceptions.TypeError'>: 'NoneType' object is not callable Piotr, have you encountered this behavior before? No, I haven't seen it before. What are the steps to reproduce? It looks like a race condition is asyncore module and we do get a response from vdsm after this error message. I've encountered this issue only on el7 systems. Run `vdsm-client Host getCapabilities | less` until you'll manage to reproduce it. I was getting the same error when I've deployed SHE. On the host of which hosted the engine run the following commands: 1. "hosted-engine --set-maintenance --mode=global" 2. "hosted-engine --vm-shutdown" 3. wait few minutes for vm machine to shutdown 4. "hosted-engine --vm-start" I'm getting as follows: alma03 ~]# hosted-engine --vm-start VM exists and is down, cleaning up and restarting Exception in thread Client localhost:54321 (most likely raised during interpreter shutdown):[root@alma03 ~]# Attaching screencast and logs from my environment here, just in case they might be of use. Created attachment 1296182 [details]
sosreport from the engine
Created attachment 1296183 [details]
sosreport from host
Components on host: ovirt-imageio-common-1.0.0-0.el7ev.noarch mom-0.5.9-1.el7ev.noarch ovirt-imageio-daemon-1.0.0-0.el7ev.noarch sanlock-3.5.0-1.el7.x86_64 ovirt-setup-lib-1.1.3-1.el7ev.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch qemu-kvm-rhev-2.9.0-16.el7.x86_64 ovirt-vmconsole-1.0.4-1.el7ev.noarch vdsm-4.19.21-1.el7ev.x86_64 ovirt-hosted-engine-ha-2.1.4-1.el7ev.noarch libvirt-client-3.2.0-14.el7.x86_64 ovirt-vmconsole-host-1.0.4-1.el7ev.noarch ovirt-hosted-engine-setup-2.1.3.4-1.el7ev.noarch ovirt-host-deploy-1.6.6-1.el7ev.noarch Linux version 3.10.0-691.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jun 29 10:30:04 EDT 2017 Linux 3.10.0-691.el7.x86_64 #1 SMP Thu Jun 29 10:30:04 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.4 (Maipo) On engine: rhev-guest-tools-iso-4.1-5.el7ev.noarch rhevm-doc-4.1.4-1.el7ev.noarch rhevm-dependencies-4.1.1-1.el7ev.noarch rhevm-4.1.4-0.2.el7.noarch rhevm-branding-rhev-4.1.0-2.el7ev.noarch rhevm-setup-plugins-4.1.2-1.el7ev.noarch Linux version 3.10.0-693.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jul 6 19:56:57 EDT 2017 Linux3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.4 (Maipo) Closing due to resource limitation |