Bug 1142647
| Summary: | supervdsm leaks memory when using glusterfs | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] oVirt | Reporter: | zhang guoqing <zhangguoqingas> | ||||
| Component: | vdsm | Assignee: | Darshan <dnarayan> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Gil Klein <gklein> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.5 | CC: | amureini, bazulay, bugs, danken, dnarayan, ecohen, gabicr, gklein, iheim, mgoldboi, mmorgan, rbalakri, sabose, s.kieske, tjeyasin, yeylon, zhangguoqingas | ||||
| Target Milestone: | --- | Keywords: | Regression | ||||
| Target Release: | 3.5.1 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | gluster | ||||||
| Fixed In Version: | ovirt-3.5.1_rc1 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-01-21 16:02:54 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Gluster | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1093594 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
|
Description
zhang guoqing
2014-09-17 07:15:36 UTC
Could you attach supervdsm.log anyway? Do you spot anything different in the log, relative to the cluster that has no glusterFS? Created attachment 938700 [details]
vdsm-logs and some pngs
Thanks，first of all！
I really cann't find helpful log, so I patch all logs of nodes' vdsm here, and some pngs that may productive to analyze the bug-1142647.
I see that supervdsm is asked to call
/usr/sbin/gluster --mode=script volume info --xml
every 5 seconds. Is this expected?
Also (and unrelated to the leak),
MainProcess|Thread-51::DEBUG::2014-09-17 10:16:59,274::supervdsmServer::101::SuperVdsm.ServerCallback::(wrapper) call wrapper with (None,) {}
does not show the called function name, only "wrapper".
supervdsm calling "/usr/sbin/gluster --mode=script volume info --xml" every 5 sec is expected behaviour. Please note, the nodes are VM that be created from my OpenStack environment, in which hypervisor is KVM too. The node(VM) "cat /proc/cpuinfo | grep vmx" is not None, so I take it as ovirt-node. Therefore, I'm not sure that, whether above situation makes a difference to this Bug-1142647 or not. Thanks all! Darshan, can post this to the ovirt-3.5 branch? It's a nasty regression that I'd like to avoid. (In reply to Dan Kenigsberg from comment #6) > Darshan, can post this to the ovirt-3.5 branch? It's a nasty regression that > I'd like to avoid. Done. I have also glusterfs
After upgrade from 3.4.4. to 3.5.0 I can see n all my 3 nodes
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
nod1 SPM running 0 VM
753 root 15 -5 17.208g 7.737g 10832 S 0.0 49.4 1:28.46 supervdsmServer
nod2 running 3 VM
641 root 15 -5 17.573g 7.888g 10768 S 0.0 33.5 1:17.09 supervdsmServer
nod3 running 2 VM
6391 root 15 -5 19.072g 8.646g 10844 S 9.3 44.1 38:17.05 supervdsmServer
So Supervdsm server ocupy around 33-49% of memory alone!
Also I've got
systemctl status supervdsmd
supervdsmd.service - "Auxiliary vdsm service for running helper functions as root"
Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static)
Active: active (running) since Tue 2014-10-21 11:32:40 EEST; 23h ago
Main PID: 753 (supervdsmServer)
CGroup: name=systemd:/system/supervdsmd.service
ââ753 /usr/bin/python /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
Oct 21 11:39:51 nod1 daemonAdapter[753]: Process Process-4:
Oct 21 11:39:51 nod1 daemonAdapter[753]: Traceback (most recent call last):
Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Oct 21 11:39:51 nod1 daemonAdapter[753]: self.run()
Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
Oct 21 11:39:51 nod1 daemonAdapter[753]: self._target(*self._args, **self._kwargs)
Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/share/vdsm/supervdsmServer", line 242, in child
Oct 21 11:39:51 nod1 daemonAdapter[753]: pipe.recv()
Oct 21 11:39:51 nod1 daemonAdapter[753]: IOError: [Errno 4] Interrupted system call
Thanks for your report. This bug is destined to be hacked-away in ovirt-3.5.1 release. This is an automated message: This bug should be fixed in oVirt 3.5.1 RC1, moving to QA oVirt 3.5.1 has been released. If problems still persist, please make note of it in this bug report. |