Bug 1958825
| Summary: | Memory leak in qemu-ga for Windows | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Juan Orti <jortialc> | ||||
| Component: | virtio-win | Assignee: | Yvugenfi <yvugenfi> | ||||
| virtio-win sub component: | qemu-ga-win | QA Contact: | dehanmeng <demeng> | ||||
| Status: | CLOSED ERRATA | Docs Contact: | |||||
| Severity: | urgent | ||||||
| Priority: | urgent | CC: | ailan, coli, demeng, jinzhao, juzhang, lijin, marcandre.lureau, mjankula, rhodain, xiagao, ymankad, yvugenfi, zhguo | ||||
| Version: | 8.4 | Keywords: | Reopened, Triaged, ZStream | ||||
| Target Milestone: | beta | Flags: | pm-rhel:
mirror+
|
||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Windows | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | mingw-qemu-ga-win-102.7.0-0.el8 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1974383 (view as bug list) | Environment: | |||||
| Last Closed: | 2021-11-09 18:52:47 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1974383, 1985906 | ||||||
| Attachments: |
|
||||||
could be dup of bz1929144 - fix qemu-ga-win resource leaks. It's fixed in rhel8.4.0 virtio-win package(virtio-win-1.9.16-2.el8) already. Dehan, Could you reproduce and have a confirm? Reproduce this issue on virtio-win-1.9.15-0.el8 and the output is like comment0. As commet2 said, this is a duplicated bz of bz_1929144 that has been fixed in rhel8.4.0 virtio-win package(virtio-win-1.9.16-2.el8) already. here are the actual result for now: PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName ------- ------ ----- ----- ------ -- -- ----------- 152 44 2244 9312 0.11 4192 0 qemu-ga 152 44 2316 9376 0.14 4192 0 qemu-ga 152 44 2364 9417 0.16 4192 0 qemu-ga 152 44 2416 9487 0.17 4192 0 qemu-ga 152 44 2434 9536 0.19 4192 0 qemu-ga 152 44 2462 9472 0.22 4192 0 qemu-ga 152 44 2492 9588 0.23 4192 0 qemu-ga 152 44 2531 9733 0.25 4192 0 qemu-ga 152 44 2476 9238 0.27 4192 0 qemu-ga 152 44 2589 9383 0.30 4192 0 qemu-ga 152 44 2602 9489 0.33 4192 0 qemu-ga 152 44 2598 9874 0.34 4192 0 qemu-ga This is a dup bz as above comments said. so close it as DUPLICATE. *** This bug has been marked as a duplicate of bug 1929144 *** I am reopening the BZ as this does not seem to be a dup of bz1929144. If you compare the test results in comment 0 and comment 3. You can see that the original issue and memory consumption is caused by the increased number of handles, but the reproducer has only increases in the memory, not in the handles. (In reply to Roman Hodain from comment #6) > I am reopening the BZ as this does not seem to be a dup of bz1929144. If you > compare the test results in comment 0 and comment 3. You can see that the > original issue and memory consumption is caused by the increased number of > handles, but the reproducer has only increases in the memory, not in the > handles. Hi Romain It might be my fault, I should paste more explanation on it. The result of comment3 described the latest rpm that has been fixed handles leak issue, so The value of the handle remains unchanged. the result of reproducing the problem was the same as comment0 actually. Thanks Dehan Meng (In reply to dehanmeng from comment #9) > (In reply to Roman Hodain from comment #6) > > I am reopening the BZ as this does not seem to be a dup of bz1929144. If you > > compare the test results in comment 0 and comment 3. You can see that the > > original issue and memory consumption is caused by the increased number of > > handles, but the reproducer has only increases in the memory, not in the > > handles. > > Hi Romain > It might be my fault, I should paste more explanation on it. The result of > comment3 described the latest rpm that has been fixed handles leak issue, so > The value of the handle remains unchanged. the result of reproducing the > problem was the same as comment0 actually. > > Thanks > Dehan Meng I double-checked, memory has indeed kept growing. this is another issue of resource leak. thanks for pointing it out. PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName ------- ------ ----- ----- ------ -- -- ----------- 143 9 1820 8404 0.03 4300 0 qemu-ga 145 13 1840 8508 0.03 4300 0 qemu-ga 152 14 2112 9272 0.06 4300 0 qemu-ga 152 14 2132 9312 0.11 4300 0 qemu-ga 152 14 2156 9336 0.13 4300 0 qemu-ga 152 14 2184 9352 0.14 4300 0 qemu-ga 152 14 2236 9404 0.19 4300 0 qemu-ga 152 14 2232 9412 0.20 4300 0 qemu-ga 152 14 2288 9456 0.22 4300 0 qemu-ga 152 13 2180 9404 0.25 4300 0 qemu-ga 152 13 2252 9476 0.30 4300 0 qemu-ga 152 13 2272 9496 0.33 4300 0 qemu-ga 152 13 2264 9488 0.34 4300 0 qemu-ga 152 14 2664 9840 0.59 4300 0 qemu-ga 152 14 2668 9844 0.63 4300 0 qemu-ga 152 14 2668 9852 0.67 4300 0 qemu-ga 152 14 2804 9980 0.69 4300 0 qemu-ga 152 14 2804 9980 0.70 4300 0 qemu-ga 152 14 2804 9984 0.72 4300 0 qemu-ga 152 14 2804 9984 0.77 4300 0 qemu-ga 152 14 2804 9984 0.78 4300 0 qemu-ga 152 14 2804 9992 0.80 4300 0 qemu-ga 152 14 2880 10064 0.81 4300 0 qemu-ga 152 14 2948 10132 0.86 4300 0 qemu-ga 152 14 2948 10132 0.88 4300 0 qemu-ga 152 14 2948 10132 0.89 4300 0 qemu-ga 152 14 2948 10132 0.91 4300 0 qemu-ga 152 14 3000 10196 0.94 4300 0 qemu-ga 150 13 3032 10240 0.95 4300 0 qemu-ga 150 13 3032 10240 0.97 4300 0 qemu-ga 150 13 3032 10240 1.00 4300 0 qemu-ga 150 13 3140 10376 1.03 4300 0 qemu-ga 150 13 3164 10384 1.05 4300 0 qemu-ga 150 13 3140 10376 1.06 4300 0 qemu-ga 150 13 3164 10384 1.08 4300 0 qemu-ga 150 13 3232 10452 1.11 4300 0 qemu-ga 150 13 3208 10444 1.13 4300 0 qemu-ga 150 13 3232 10452 1.14 4300 0 qemu-ga 150 13 3208 10444 1.17 4300 0 qemu-ga 150 13 3232 10452 1.22 4300 0 qemu-ga 150 13 3232 10452 1.23 4300 0 qemu-ga 150 13 3512 10720 1.25 4300 0 qemu-ga 150 13 3512 10720 1.28 4300 0 qemu-ga 150 13 3512 10724 1.33 4300 0 qemu-ga 150 13 3512 10724 1.34 4300 0 qemu-ga 150 13 3512 10724 1.36 4300 0 qemu-ga 150 13 3512 10724 1.39 4300 0 qemu-ga 150 13 3512 10724 1.44 4300 0 qemu-ga 150 13 3512 10724 1.45 4300 0 qemu-ga 150 13 3512 10724 1.48 4300 0 qemu-ga 150 13 3512 10724 1.52 4300 0 qemu-ga 150 13 3652 10860 1.55 4300 0 qemu-ga 150 13 3652 10860 1.56 4300 0 qemu-ga 150 13 3792 10996 1.58 4300 0 qemu-ga 150 13 3792 10996 1.61 4300 0 qemu-ga 150 13 3932 11132 1.88 4300 0 qemu-ga 150 13 4072 11264 1.94 4300 0 qemu-ga 150 13 4072 11264 1.97 4300 0 qemu-ga 150 13 4072 11264 2.00 4300 0 qemu-ga 150 13 4072 11264 2.02 4300 0 qemu-ga 150 13 4072 11268 2.05 4300 0 qemu-ga 150 13 4072 11268 2.08 4300 0 qemu-ga 150 13 4212 11404 2.11 4300 0 qemu-ga 150 13 4212 11404 2.13 4300 0 qemu-ga 150 13 4212 11404 2.16 4300 0 qemu-ga 150 13 4212 11404 2.17 4300 0 qemu-ga 150 13 4212 11404 2.22 4300 0 qemu-ga 150 13 4212 11404 2.23 4300 0 qemu-ga 150 13 4356 11548 2.27 4300 0 qemu-ga 150 13 4332 11536 2.28 4300 0 qemu-ga 150 13 4332 11536 2.31 4300 0 qemu-ga 150 13 4336 11540 2.33 4300 0 qemu-ga 150 13 4468 11668 2.36 4300 0 qemu-ga 150 13 4464 11664 2.38 4300 0 qemu-ga 150 13 4472 11672 2.39 4300 0 qemu-ga 150 13 4496 11684 2.41 4300 0 qemu-ga 150 13 4484 11684 2.44 4300 0 qemu-ga 150 13 4484 11684 2.45 4300 0 qemu-ga 150 13 4484 11680 2.47 4300 0 qemu-ga 150 13 4484 11680 2.50 4300 0 qemu-ga 150 13 4480 11680 2.53 4300 0 qemu-ga 150 13 4636 11804 2.55 4300 0 qemu-ga 150 13 4684 11860 2.58 4300 0 qemu-ga 150 14 4832 12004 2.61 4300 0 qemu-ga 150 14 4836 12008 2.66 4300 0 qemu-ga 150 14 4836 12008 2.67 4300 0 qemu-ga 150 14 4836 12008 2.69 4300 0 qemu-ga 150 14 4832 12004 2.72 4300 0 qemu-ga 150 14 4828 12088 2.77 4300 0 qemu-ga 150 14 4824 12084 2.78 4300 0 qemu-ga 150 14 4832 12092 2.80 4300 0 qemu-ga 150 14 4824 12084 2.81 4300 0 qemu-ga 150 14 4976 12228 2.84 4300 0 qemu-ga 150 14 4964 12220 2.86 4300 0 qemu-ga 150 14 5104 12352 2.88 4300 0 qemu-ga 150 14 5112 12360 2.89 4300 0 qemu-ga 150 14 5104 12352 2.94 4300 0 qemu-ga 150 14 5104 12352 2.95 4300 0 qemu-ga 150 14 5112 12360 2.97 4300 0 qemu-ga 150 14 5104 11996 3.00 4300 0 qemu-ga (In reply to dehanmeng from comment #10) > (In reply to dehanmeng from comment #9) > > (In reply to Roman Hodain from comment #6) > > > I am reopening the BZ as this does not seem to be a dup of bz1929144. If you > > > compare the test results in comment 0 and comment 3. You can see that the > > > original issue and memory consumption is caused by the increased number of > > > handles, but the reproducer has only increases in the memory, not in the > > > handles. > > > > Hi Romain > > It might be my fault, I should paste more explanation on it. The result of > > comment3 described the latest rpm that has been fixed handles leak issue, so > > The value of the handle remains unchanged. the result of reproducing the > > problem was the same as comment0 actually. > > > > Thanks > > Dehan Meng > > I double-checked, memory has indeed kept growing. this is another issue of > resource leak. thanks for pointing it out. > PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; > sleep 5 } > > Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName > ------- ------ ----- ----- ------ -- -- ----------- > 143 9 1820 8404 0.03 4300 0 qemu-ga > 145 13 1840 8508 0.03 4300 0 qemu-ga > 152 14 2112 9272 0.06 4300 0 qemu-ga > 152 14 2132 9312 0.11 4300 0 qemu-ga > 152 14 2156 9336 0.13 4300 0 qemu-ga > 152 14 2184 9352 0.14 4300 0 qemu-ga > 152 14 2236 9404 0.19 4300 0 qemu-ga > 152 14 2232 9412 0.20 4300 0 qemu-ga > 152 14 2288 9456 0.22 4300 0 qemu-ga > 152 13 2180 9404 0.25 4300 0 qemu-ga > 152 13 2252 9476 0.30 4300 0 qemu-ga > 152 13 2272 9496 0.33 4300 0 qemu-ga > 152 13 2264 9488 0.34 4300 0 qemu-ga > 152 14 2664 9840 0.59 4300 0 qemu-ga > 152 14 2668 9844 0.63 4300 0 qemu-ga > 152 14 2668 9852 0.67 4300 0 qemu-ga > 152 14 2804 9980 0.69 4300 0 qemu-ga > 152 14 2804 9980 0.70 4300 0 qemu-ga > 152 14 2804 9984 0.72 4300 0 qemu-ga > 152 14 2804 9984 0.77 4300 0 qemu-ga > 152 14 2804 9984 0.78 4300 0 qemu-ga > 152 14 2804 9992 0.80 4300 0 qemu-ga > 152 14 2880 10064 0.81 4300 0 qemu-ga > 152 14 2948 10132 0.86 4300 0 qemu-ga > 152 14 2948 10132 0.88 4300 0 qemu-ga > 152 14 2948 10132 0.89 4300 0 qemu-ga > 152 14 2948 10132 0.91 4300 0 qemu-ga > 152 14 3000 10196 0.94 4300 0 qemu-ga > 150 13 3032 10240 0.95 4300 0 qemu-ga > 150 13 3032 10240 0.97 4300 0 qemu-ga > 150 13 3032 10240 1.00 4300 0 qemu-ga > 150 13 3140 10376 1.03 4300 0 qemu-ga > 150 13 3164 10384 1.05 4300 0 qemu-ga > 150 13 3140 10376 1.06 4300 0 qemu-ga > 150 13 3164 10384 1.08 4300 0 qemu-ga > 150 13 3232 10452 1.11 4300 0 qemu-ga > 150 13 3208 10444 1.13 4300 0 qemu-ga > 150 13 3232 10452 1.14 4300 0 qemu-ga > 150 13 3208 10444 1.17 4300 0 qemu-ga > 150 13 3232 10452 1.22 4300 0 qemu-ga > 150 13 3232 10452 1.23 4300 0 qemu-ga > 150 13 3512 10720 1.25 4300 0 qemu-ga > 150 13 3512 10720 1.28 4300 0 qemu-ga > 150 13 3512 10724 1.33 4300 0 qemu-ga > 150 13 3512 10724 1.34 4300 0 qemu-ga > 150 13 3512 10724 1.36 4300 0 qemu-ga > 150 13 3512 10724 1.39 4300 0 qemu-ga > 150 13 3512 10724 1.44 4300 0 qemu-ga > 150 13 3512 10724 1.45 4300 0 qemu-ga > 150 13 3512 10724 1.48 4300 0 qemu-ga > 150 13 3512 10724 1.52 4300 0 qemu-ga > 150 13 3652 10860 1.55 4300 0 qemu-ga > 150 13 3652 10860 1.56 4300 0 qemu-ga > 150 13 3792 10996 1.58 4300 0 qemu-ga > 150 13 3792 10996 1.61 4300 0 qemu-ga > 150 13 3932 11132 1.88 4300 0 qemu-ga > 150 13 4072 11264 1.94 4300 0 qemu-ga > 150 13 4072 11264 1.97 4300 0 qemu-ga > 150 13 4072 11264 2.00 4300 0 qemu-ga > 150 13 4072 11264 2.02 4300 0 qemu-ga > 150 13 4072 11268 2.05 4300 0 qemu-ga > 150 13 4072 11268 2.08 4300 0 qemu-ga > 150 13 4212 11404 2.11 4300 0 qemu-ga > 150 13 4212 11404 2.13 4300 0 qemu-ga > 150 13 4212 11404 2.16 4300 0 qemu-ga > 150 13 4212 11404 2.17 4300 0 qemu-ga > 150 13 4212 11404 2.22 4300 0 qemu-ga > 150 13 4212 11404 2.23 4300 0 qemu-ga > 150 13 4356 11548 2.27 4300 0 qemu-ga > 150 13 4332 11536 2.28 4300 0 qemu-ga > 150 13 4332 11536 2.31 4300 0 qemu-ga > 150 13 4336 11540 2.33 4300 0 qemu-ga > 150 13 4468 11668 2.36 4300 0 qemu-ga > 150 13 4464 11664 2.38 4300 0 qemu-ga > 150 13 4472 11672 2.39 4300 0 qemu-ga > 150 13 4496 11684 2.41 4300 0 qemu-ga > 150 13 4484 11684 2.44 4300 0 qemu-ga > 150 13 4484 11684 2.45 4300 0 qemu-ga > 150 13 4484 11680 2.47 4300 0 qemu-ga > 150 13 4484 11680 2.50 4300 0 qemu-ga > 150 13 4480 11680 2.53 4300 0 qemu-ga > 150 13 4636 11804 2.55 4300 0 qemu-ga > 150 13 4684 11860 2.58 4300 0 qemu-ga > 150 14 4832 12004 2.61 4300 0 qemu-ga > 150 14 4836 12008 2.66 4300 0 qemu-ga > 150 14 4836 12008 2.67 4300 0 qemu-ga > 150 14 4836 12008 2.69 4300 0 qemu-ga > 150 14 4832 12004 2.72 4300 0 qemu-ga > 150 14 4828 12088 2.77 4300 0 qemu-ga > 150 14 4824 12084 2.78 4300 0 qemu-ga > 150 14 4832 12092 2.80 4300 0 qemu-ga > 150 14 4824 12084 2.81 4300 0 qemu-ga > 150 14 4976 12228 2.84 4300 0 qemu-ga > 150 14 4964 12220 2.86 4300 0 qemu-ga > 150 14 5104 12352 2.88 4300 0 qemu-ga > 150 14 5112 12360 2.89 4300 0 qemu-ga > 150 14 5104 12352 2.94 4300 0 qemu-ga > 150 14 5104 12352 2.95 4300 0 qemu-ga > 150 14 5112 12360 2.97 4300 0 qemu-ga > 150 14 5104 11996 3.00 4300 0 qemu-ga Dehan, what commands are issuing from the host during the reproduction? (In reply to Juan Orti from comment #0) > Created attachment 1781597 [details] > qemu-ga.log > > Description of problem: > qemu-ga for Windows leaks memory and handles and this makes the server crash > when all memory is exhausted. > > Version-Release number of selected component (if applicable): > virtio-win-1.9.15-0.el8.noarch > qemu-ga for Windows version 101.2.0.0 > > How reproducible: > Reported by a customer in their Windows 2019 servers. I reproduced it easily > by installing latest qemu-ga in a Windows 2016 VM. > > Steps to Reproduce: > 1. Install Windows server in RHV 4.4 environment. > 2. Install qemu-ga using ISO from virtio-win-1.9.15-0.el8.noarch > 3. Run in PowerShell: > > while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } > > Actual results: > Handles and PM (The amount of pageable memory that the process is using, in > kilobytes) keep increasing. > > PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; > sleep 5 } > > Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName > ------- ------ ----- ----- ------ -- -- ----------- > 170 10 2456 9796 0,16 2868 0 qemu-ga > 171 10 2456 9800 0,17 2868 0 qemu-ga > 171 10 2456 9800 0,17 2868 0 qemu-ga > 172 10 2456 9796 0,17 2868 0 qemu-ga > 172 10 2456 9796 0,17 2868 0 qemu-ga > 173 10 2456 9800 0,19 2868 0 qemu-ga > 174 10 2404 9848 0,20 2868 0 qemu-ga > 175 10 2420 9868 0,20 2868 0 qemu-ga > 175 10 2420 9868 0,20 2868 0 qemu-ga > 176 10 2412 9860 0,20 2868 0 qemu-ga > 176 10 2412 9860 0,20 2868 0 qemu-ga > 177 10 2444 9876 0,22 2868 0 qemu-ga > 177 10 2444 9876 0,22 2868 0 qemu-ga > 178 10 2444 9876 0,22 2868 0 qemu-ga > 178 10 2444 9876 0,22 2868 0 qemu-ga > 179 10 2444 9876 0,22 2868 0 qemu-ga > 179 10 2444 9876 0,23 2868 0 qemu-ga > 180 10 2444 9876 0,25 2868 0 qemu-ga > 180 10 2568 9892 0,25 2868 0 qemu-ga > 181 10 2444 9876 0,25 2868 0 qemu-ga > 181 10 2444 9876 0,25 2868 0 qemu-ga > 182 10 2444 9876 0,25 2868 0 qemu-ga > 182 10 2444 9876 0,25 2868 0 qemu-ga > 183 10 2444 9876 0,25 2868 0 qemu-ga > 183 10 2444 9876 0,25 2868 0 qemu-ga > 184 10 2444 9876 0,25 2868 0 qemu-ga > 184 10 2444 9876 0,25 2868 0 qemu-ga > 185 10 2568 9892 0,25 2868 0 qemu-ga > 185 10 2568 9900 0,25 2868 0 qemu-ga > 186 10 2568 9904 0,25 2868 0 qemu-ga > 189 10 2612 9968 0,25 2868 0 qemu-ga > 190 10 2524 9948 0,25 2868 0 qemu-ga > 190 10 2524 9948 0,25 2868 0 qemu-ga > 191 10 2524 9948 0,25 2868 0 qemu-ga > 191 10 2524 9948 0,25 2868 0 qemu-ga > 192 10 2524 9948 0,25 2868 0 qemu-ga > 192 10 2524 9948 0,25 2868 0 qemu-ga > 193 10 2524 9948 0,25 2868 0 qemu-ga > 191 10 2492 9932 0,27 2868 0 qemu-ga > 192 10 2492 9932 0,28 2868 0 qemu-ga > 192 10 2492 9932 0,28 2868 0 qemu-ga > 193 10 2508 9932 0,28 2868 0 qemu-ga > 193 10 2508 9932 0,28 2868 0 qemu-ga > 194 10 2508 9928 0,28 2868 0 qemu-ga > 194 10 2508 9928 0,28 2868 0 qemu-ga > 195 10 2508 9928 0,28 2868 0 qemu-ga > 195 10 2508 9928 0,28 2868 0 qemu-ga > 196 10 2540 9944 0,28 2868 0 qemu-ga > 196 10 2540 9944 0,28 2868 0 qemu-ga > 197 10 2504 9936 0,28 2868 0 qemu-ga > 197 10 2504 9936 0,28 2868 0 qemu-ga > 198 10 2504 9936 0,28 2868 0 qemu-ga > 198 10 2504 9936 0,28 2868 0 qemu-ga > 199 10 2612 9944 0,28 2868 0 qemu-ga > 200 10 2676 10048 0,30 2868 0 qemu-ga > 201 10 2680 10032 0,30 2868 0 qemu-ga > 201 10 2680 10032 0,30 2868 0 qemu-ga > 203 10 2704 10032 0,31 2868 0 qemu-ga > 203 10 2704 10032 0,31 2868 0 qemu-ga > 204 10 2704 10032 0,33 2868 0 qemu-ga > 204 10 2704 10032 0,33 2868 0 qemu-ga > 205 10 2704 10032 0,33 2868 0 qemu-ga > 205 10 2704 10032 0,33 2868 0 qemu-ga > 204 10 2568 10000 0,33 2868 0 qemu-ga > 206 10 2632 10032 0,33 2868 0 qemu-ga > 207 10 2664 10020 0,33 2868 0 qemu-ga > 207 10 2664 10020 0,33 2868 0 qemu-ga > 208 10 2664 10024 0,33 2868 0 qemu-ga > 208 10 2664 10024 0,33 2868 0 qemu-ga > 209 10 2664 10024 0,33 2868 0 qemu-ga > 209 10 2664 10024 0,33 2868 0 qemu-ga > 210 10 2664 10024 0,33 2868 0 qemu-ga > 210 10 2664 10024 0,33 2868 0 qemu-ga > 211 10 2664 10024 0,33 2868 0 qemu-ga > 211 10 2664 10024 0,33 2868 0 qemu-ga > 212 10 2664 10024 0,33 2868 0 qemu-ga > 213 10 2760 10108 0,36 2868 0 qemu-ga > 214 10 2760 10108 0,36 2868 0 qemu-ga > > Expected results: > No memory leaks > > Additional info: Hello Juan, Can you please provide the crash dump? Also, is there a log of the qemu-ga commands send from the host side? Thanks, Yan. (In reply to Yvugenfi from comment #13) > > I double-checked, memory has indeed kept growing. this is another issue of > > resource leak. thanks for pointing it out. > > PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; > > sleep 5 } > > > > Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName > > ------- ------ ----- ----- ------ -- -- ----------- > > 143 9 1820 8404 0.03 4300 0 qemu-ga > > 145 13 1840 8508 0.03 4300 0 qemu-ga > > 152 14 2112 9272 0.06 4300 0 qemu-ga > > 152 14 2132 9312 0.11 4300 0 qemu-ga > > 152 14 2156 9336 0.13 4300 0 qemu-ga > > 152 14 2184 9352 0.14 4300 0 qemu-ga > > 152 14 2236 9404 0.19 4300 0 qemu-ga > > 152 14 2232 9412 0.20 4300 0 qemu-ga > > 152 14 2288 9456 0.22 4300 0 qemu-ga > > 152 13 2180 9404 0.25 4300 0 qemu-ga > > 152 13 2252 9476 0.30 4300 0 qemu-ga > > 152 13 2272 9496 0.33 4300 0 qemu-ga > > 152 13 2264 9488 0.34 4300 0 qemu-ga > > 152 14 2664 9840 0.59 4300 0 qemu-ga > > 152 14 2668 9844 0.63 4300 0 qemu-ga > > 152 14 2668 9852 0.67 4300 0 qemu-ga > > 152 14 2804 9980 0.69 4300 0 qemu-ga > > 152 14 2804 9980 0.70 4300 0 qemu-ga > > 152 14 2804 9984 0.72 4300 0 qemu-ga > > 152 14 2804 9984 0.77 4300 0 qemu-ga > > 152 14 2804 9984 0.78 4300 0 qemu-ga > > 152 14 2804 9992 0.80 4300 0 qemu-ga > > 152 14 2880 10064 0.81 4300 0 qemu-ga > > 152 14 2948 10132 0.86 4300 0 qemu-ga > > 152 14 2948 10132 0.88 4300 0 qemu-ga > > 152 14 2948 10132 0.89 4300 0 qemu-ga > > 152 14 2948 10132 0.91 4300 0 qemu-ga > > 152 14 3000 10196 0.94 4300 0 qemu-ga > > 150 13 3032 10240 0.95 4300 0 qemu-ga > > 150 13 3032 10240 0.97 4300 0 qemu-ga > > 150 13 3032 10240 1.00 4300 0 qemu-ga > > 150 14 5104 11996 3.00 4300 0 qemu-ga > > Dehan, what commands are issuing from the host during the reproduction? Hi Yvugenfi, I operate this command "cat <(for i in {1..1000}; do echo '{"execute":"guest-get-osinfo"}'; sleep 2; echo '{"execute":"guest-get-devices"}'; sleep 2; done) | nc -U /tmp/qga.sock > log" The fix in mingw-qemu-ga-win-102.4.0-0.el8 Tried to verify this issue on win2016-64 guest with the fixed build mingw-qemu-ga-win-102.2.0-2.el8, but the issue seems still be there. memory still keep raising and didn't have no sign of stopping.
Actutal result:
PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; sleep 5 }
Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName
------- ------ ----- ----- ------ -- -- -----------
150 13 2108 9316 0.17 1612 0 qemu-ga
150 13 2136 9344 0.20 1612 0 qemu-ga
150 13 2224 9432 0.28 1612 0 qemu-ga
150 13 4604 11420 2.16 1612 0 qemu-ga
150 13 4604 11420 2.17 1612 0 qemu-ga
150 13 4604 11420 2.19 1612 0 qemu-ga
150 13 4744 11564 2.27 1612 0 qemu-ga
150 13 4744 11564 2.28 1612 0 qemu-ga
150 13 4744 11564 2.30 1612 0 qemu-ga
150 13 4744 11564 2.31 1612 0 qemu-ga
150 14 4892 11704 2.34 1612 0 qemu-ga
150 14 4892 11704 2.36 1612 0 qemu-ga
150 14 4892 11704 2.38 1612 0 qemu-ga
150 14 4892 11704 2.45 1612 0 qemu-ga
150 14 4892 11704 2.47 1612 0 qemu-ga
150 14 5032 11840 2.50 1612 0 qemu-ga
150 14 5032 11840 2.52 1612 0 qemu-ga
150 14 5172 11976 2.59 1612 0 qemu-ga
150 14 5172 11976 2.63 1612 0 qemu-ga
150 14 5452 12248 2.80 1612 0 qemu-ga
150 14 5452 12248 2.81 1612 0 qemu-ga
150 14 5452 12248 2.81 1612 0 qemu-ga
150 14 5800 12572 2.98 1612 0 qemu-ga
150 14 5800 12572 3.02 1612 0 qemu-ga
150 14 5800 12572 3.03 1612 0 qemu-ga
150 14 5800 12572 3.05 1612 0 qemu-ga
150 14 5800 12572 3.06 1612 0 qemu-ga
150 14 5800 12572 3.09 1612 0 qemu-ga
150 14 6780 13504 3.75 1612 0 qemu-ga
150 14 6780 13504 3.77 1612 0 qemu-ga
150 14 6780 13504 3.78 1612 0 qemu-ga
150 14 6920 13636 3.81 1612 0 qemu-ga
150 14 7060 13768 3.83 1612 0 qemu-ga
150 14 7060 13768 3.84 1612 0 qemu-ga
150 14 7060 13776 3.94 1612 0 qemu-ga
150 14 7060 13776 3.95 1612 0 qemu-ga
150 14 7200 13908 4.00 1612 0 qemu-ga
150 14 7200 13908 4.02 1612 0 qemu-ga
150 14 7340 14044 4.03 1612 0 qemu-ga
150 14 7340 14044 4.05 1612 0 qemu-ga
150 14 7340 14044 4.08 1612 0 qemu-ga
150 14 7340 14044 4.09 1612 0 qemu-ga
150 14 7340 14044 4.09 1612 0 qemu-ga
150 14 7340 14044 4.13 1612 0 qemu-ga
150 14 7340 14044 4.16 1612 0 qemu-ga
150 14 7480 14176 4.17 1612 0 qemu-ga
150 14 7480 14176 4.19 1612 0 qemu-ga
150 14 7480 14176 4.20 1612 0 qemu-ga
150 14 7480 14176 4.23 1612 0 qemu-ga
150 14 7480 14176 4.25 1612 0 qemu-ga
150 14 7620 14308 4.27 1612 0 qemu-ga
150 14 7620 14308 4.28 1612 0 qemu-ga
Sorry about the last outdated comment, I retried the latest version of qemu-ga build. version: kernel-4.18.0-312.el8.x86_64 qemu-ga-win-102.4.0-0.el8.noarch.rpm qemu-kvm-docs-6.0.0-19.module+el8.5.0+11385+6e7d542e.x86_64 RHEL-8.5.0-20210611.n.0 Verify steps as comment10; Actual result: PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName ------- ------ ----- ----- ------ -- -- ----------- 145 13 2052 8380 0.02 3612 0 qemu-ga 152 14 2264 9060 0.05 3612 0 qemu-ga 152 14 2292 9100 0.11 3612 0 qemu-ga 152 14 2312 9112 0.14 3612 0 qemu-ga 152 14 2296 9108 0.16 3612 0 qemu-ga 152 14 2296 9108 0.19 3612 0 qemu-ga 152 14 2296 9108 0.25 3612 0 qemu-ga 152 14 2300 9108 0.27 3612 0 qemu-ga 152 14 2300 9108 0.28 3612 0 qemu-ga 152 14 2300 9112 0.30 3612 0 qemu-ga 152 14 2260 9068 0.36 3612 0 qemu-ga 152 14 2232 9076 0.39 3612 0 qemu-ga 152 14 2220 9064 0.42 3612 0 qemu-ga 152 14 2244 9088 0.44 3612 0 qemu-ga 152 14 2280 9092 0.48 3612 0 qemu-ga 152 14 2280 9096 0.52 3612 0 qemu-ga 152 14 2280 9096 0.53 3612 0 qemu-ga 152 14 2248 9092 0.55 3612 0 qemu-ga 152 14 2240 9088 0.58 3612 0 qemu-ga 152 14 2232 9072 0.61 3612 0 qemu-ga 152 14 2240 9084 0.63 3612 0 qemu-ga 152 14 2248 9096 0.64 3612 0 qemu-ga 152 14 2248 9096 0.69 3612 0 qemu-ga 152 14 2240 9084 0.72 3612 0 qemu-ga 152 14 2248 9096 0.73 3612 0 qemu-ga 152 14 2240 9084 0.75 3612 0 qemu-ga 152 14 2240 9084 0.80 3612 0 qemu-ga 152 14 2248 9096 0.92 3612 0 qemu-ga 152 14 2252 9084 1.22 3612 0 qemu-ga 152 14 2244 9096 1.23 3612 0 qemu-ga 152 14 2256 9092 1.25 3612 0 qemu-ga 152 14 2240 9092 1.27 3612 0 qemu-ga 152 14 2244 9096 1.30 3612 0 qemu-ga 152 14 2256 9092 1.31 3612 0 qemu-ga 152 14 2240 9092 1.33 3612 0 qemu-ga 152 14 2252 9088 1.34 3612 0 qemu-ga 152 14 2256 9092 1.38 3612 0 qemu-ga 152 14 2240 9092 1.39 3612 0 qemu-ga 152 14 2252 9088 1.41 3612 0 qemu-ga 152 14 2244 9096 1.42 3612 0 qemu-ga 152 14 2240 9092 1.45 3612 0 qemu-ga 152 14 2252 9088 1.47 3612 0 qemu-ga 152 14 2244 9096 1.48 3612 0 qemu-ga 152 14 2256 9092 1.50 3612 0 qemu-ga 152 14 2252 9088 1.53 3612 0 qemu-ga 152 14 2244 9096 1.53 3612 0 qemu-ga 152 14 2256 9092 1.55 3612 0 qemu-ga 152 14 2240 9092 1.56 3612 0 qemu-ga 152 14 2244 9096 1.59 3612 0 qemu-ga 152 14 2256 9092 1.61 3612 0 qemu-ga 152 14 2240 9092 1.63 3612 0 qemu-ga 152 14 2252 9088 1.64 3612 0 qemu-ga 152 14 2256 9092 1.67 3612 0 qemu-ga 152 14 2240 9092 1.69 3612 0 qemu-ga Expected result: memory shouldn't leak, In a stable floating interval Thanks for your effort Yan, this issue is gone. Cheers Dehan Meng Hi Yan,
After testing for several qga commands that query guest infos, located command {"execute": "guest-exec", "arguments": {"path": "PING", "arg": [ "www.redhat.com" ],"capture-output": true}} caused handles/memory leak raise swiftly. and command {"execute":"guest-get-disks"} cause handles changed float are relatively large, but memory maintain a safe range.
Other info we can wait for Roman to provide us dump file and check it more clearly. It is certain that there is a problem with these two command lines
Thanks
Dehan
(In reply to dehanmeng from comment #26) > Hi Yan, > > After testing for several qga commands that query guest infos, located > command {"execute": "guest-exec", "arguments": {"path": "PING", "arg": [ > "www.redhat.com" ],"capture-output": true}} caused handles/memory leak raise > swiftly. This is not a leak. Just a regular usage because another child process is executed. and command {"execute":"guest-get-disks"} cause handles changed > float are relatively large, but memory maintain a safe range. > Other info we can wait for Roman to provide us dump file and check it more > clearly. It is certain that there is a problem with these two command lines > > Thanks > Dehan Hi Roman, Could you please ask the customer about how to reproduce this issue? definitely, it's a really important thing for now, we'll also hit this issue in the further if we can't reproduce it this time. BRs Dehan Meng Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virtio-win bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:4341 |
Created attachment 1781597 [details] qemu-ga.log Description of problem: qemu-ga for Windows leaks memory and handles and this makes the server crash when all memory is exhausted. Version-Release number of selected component (if applicable): virtio-win-1.9.15-0.el8.noarch qemu-ga for Windows version 101.2.0.0 How reproducible: Reported by a customer in their Windows 2019 servers. I reproduced it easily by installing latest qemu-ga in a Windows 2016 VM. Steps to Reproduce: 1. Install Windows server in RHV 4.4 environment. 2. Install qemu-ga using ISO from virtio-win-1.9.15-0.el8.noarch 3. Run in PowerShell: while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } Actual results: Handles and PM (The amount of pageable memory that the process is using, in kilobytes) keep increasing. PS C:\Users\Administrator> while(1) { Get-Process -ProcessName qemu-ga; sleep 5 } Handles NPM(K) PM(K) WS(K) CPU(s) Id SI ProcessName ------- ------ ----- ----- ------ -- -- ----------- 170 10 2456 9796 0,16 2868 0 qemu-ga 171 10 2456 9800 0,17 2868 0 qemu-ga 171 10 2456 9800 0,17 2868 0 qemu-ga 172 10 2456 9796 0,17 2868 0 qemu-ga 172 10 2456 9796 0,17 2868 0 qemu-ga 173 10 2456 9800 0,19 2868 0 qemu-ga 174 10 2404 9848 0,20 2868 0 qemu-ga 175 10 2420 9868 0,20 2868 0 qemu-ga 175 10 2420 9868 0,20 2868 0 qemu-ga 176 10 2412 9860 0,20 2868 0 qemu-ga 176 10 2412 9860 0,20 2868 0 qemu-ga 177 10 2444 9876 0,22 2868 0 qemu-ga 177 10 2444 9876 0,22 2868 0 qemu-ga 178 10 2444 9876 0,22 2868 0 qemu-ga 178 10 2444 9876 0,22 2868 0 qemu-ga 179 10 2444 9876 0,22 2868 0 qemu-ga 179 10 2444 9876 0,23 2868 0 qemu-ga 180 10 2444 9876 0,25 2868 0 qemu-ga 180 10 2568 9892 0,25 2868 0 qemu-ga 181 10 2444 9876 0,25 2868 0 qemu-ga 181 10 2444 9876 0,25 2868 0 qemu-ga 182 10 2444 9876 0,25 2868 0 qemu-ga 182 10 2444 9876 0,25 2868 0 qemu-ga 183 10 2444 9876 0,25 2868 0 qemu-ga 183 10 2444 9876 0,25 2868 0 qemu-ga 184 10 2444 9876 0,25 2868 0 qemu-ga 184 10 2444 9876 0,25 2868 0 qemu-ga 185 10 2568 9892 0,25 2868 0 qemu-ga 185 10 2568 9900 0,25 2868 0 qemu-ga 186 10 2568 9904 0,25 2868 0 qemu-ga 189 10 2612 9968 0,25 2868 0 qemu-ga 190 10 2524 9948 0,25 2868 0 qemu-ga 190 10 2524 9948 0,25 2868 0 qemu-ga 191 10 2524 9948 0,25 2868 0 qemu-ga 191 10 2524 9948 0,25 2868 0 qemu-ga 192 10 2524 9948 0,25 2868 0 qemu-ga 192 10 2524 9948 0,25 2868 0 qemu-ga 193 10 2524 9948 0,25 2868 0 qemu-ga 191 10 2492 9932 0,27 2868 0 qemu-ga 192 10 2492 9932 0,28 2868 0 qemu-ga 192 10 2492 9932 0,28 2868 0 qemu-ga 193 10 2508 9932 0,28 2868 0 qemu-ga 193 10 2508 9932 0,28 2868 0 qemu-ga 194 10 2508 9928 0,28 2868 0 qemu-ga 194 10 2508 9928 0,28 2868 0 qemu-ga 195 10 2508 9928 0,28 2868 0 qemu-ga 195 10 2508 9928 0,28 2868 0 qemu-ga 196 10 2540 9944 0,28 2868 0 qemu-ga 196 10 2540 9944 0,28 2868 0 qemu-ga 197 10 2504 9936 0,28 2868 0 qemu-ga 197 10 2504 9936 0,28 2868 0 qemu-ga 198 10 2504 9936 0,28 2868 0 qemu-ga 198 10 2504 9936 0,28 2868 0 qemu-ga 199 10 2612 9944 0,28 2868 0 qemu-ga 200 10 2676 10048 0,30 2868 0 qemu-ga 201 10 2680 10032 0,30 2868 0 qemu-ga 201 10 2680 10032 0,30 2868 0 qemu-ga 203 10 2704 10032 0,31 2868 0 qemu-ga 203 10 2704 10032 0,31 2868 0 qemu-ga 204 10 2704 10032 0,33 2868 0 qemu-ga 204 10 2704 10032 0,33 2868 0 qemu-ga 205 10 2704 10032 0,33 2868 0 qemu-ga 205 10 2704 10032 0,33 2868 0 qemu-ga 204 10 2568 10000 0,33 2868 0 qemu-ga 206 10 2632 10032 0,33 2868 0 qemu-ga 207 10 2664 10020 0,33 2868 0 qemu-ga 207 10 2664 10020 0,33 2868 0 qemu-ga 208 10 2664 10024 0,33 2868 0 qemu-ga 208 10 2664 10024 0,33 2868 0 qemu-ga 209 10 2664 10024 0,33 2868 0 qemu-ga 209 10 2664 10024 0,33 2868 0 qemu-ga 210 10 2664 10024 0,33 2868 0 qemu-ga 210 10 2664 10024 0,33 2868 0 qemu-ga 211 10 2664 10024 0,33 2868 0 qemu-ga 211 10 2664 10024 0,33 2868 0 qemu-ga 212 10 2664 10024 0,33 2868 0 qemu-ga 213 10 2760 10108 0,36 2868 0 qemu-ga 214 10 2760 10108 0,36 2868 0 qemu-ga Expected results: No memory leaks Additional info: