Bug 1358090

Summary: shared memory leakage in vncviewer
Product: Red Hat Enterprise Linux 7 Reporter: Tsukahara Ken <tsukahara.ken>
Component: tigervncAssignee: Jan Grulich <jgrulich>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.2CC: tpelka
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 20:50:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1393395    
Attachments:
Description Flags
workaround none

Description Tsukahara Ken 2016-07-20 02:42:27 UTC
Created attachment 1181888 [details]
workaround

Description of problem:
vncviewer creates a shared memory segment for MIT-SHM.
It is not removed when vncviewer is killed by signal.

Version-Release number of selected component (if applicable):
tigervnc-1.3.1-4.el7_2.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Check initial shared memory information.
   $ ipcs -m
   ------ Shared Memory Segments --------
   key        shmid      owner      perms      bytes      nattch     status
   0x011456ac 0          root       600        1000       6
   0x00000000 229377     user1      600        524288     2          dest
   0x00000000 262146     user1      600        4194304    2          dest
   0x00000000 360451     user1      600        4194304    2          dest
   0x00000000 458758     user1      600        4194304    2          dest
2. Run vncviewer. -display to local Xorg(using MIT-SHM).
   $ vncviewer -display :0.0 192.168.0.100:5900
3. Check shared memory information. shmid 753668 is created and attached.
   $ ipcs -m
   ------ Shared Memory Segments --------
   key        shmid      owner      perms      bytes      nattch     status
   0x011456ac 0          root       600        1000       6
   0x00000000 229377     user1      600        524288     2          dest
   0x00000000 262146     user1      600        4194304    2          dest
   0x00000000 360451     user1      600        4194304    2          dest
   0x00000000 753668     user1      777        5242880    2
   0x00000000 458758     user1      600        4194304    2          dest
4. Kill the vncviewer by SIGTERM (or SIGINT).
   $ killall vncviewer

Actual results:
   shmid 753668 still exists.
   $ ipcs -m
   ------ Shared Memory Segments --------
   key        shmid      owner      perms      bytes      nattch     status
   0x011456ac 0          root       600        1000       6
   0x00000000 229377     user1      600        524288     2          dest
   0x00000000 262146     user1      600        4194304    2          dest
   0x00000000 360451     user1      600        4194304    2          dest
   0x00000000 753668     user1      777        5242880    0
   0x00000000 458758     user1      600        4194304    2          dest

Expected results:
   shmid 753668 should be removed.
   $ ipcs -m
   ------ Shared Memory Segments --------
   key        shmid      owner      perms      bytes      nattch     status
   0x011456ac 0          root       600        1000       6
   0x00000000 229377     user1      600        524288     2          dest
   0x00000000 262146     user1      600        4194304    2          dest
   0x00000000 360451     user1      600        4194304    2          dest
   0x00000000 458758     user1      600        4194304    2          dest

Additional info:
   PlatformPixelBuffer destructor is never called after CleanupSignalHandler().

Workaround:
   Mark the shared memory as "destroyed" as soon as possible.
   It will be invisible soon after the vncviewer is killed.
   See attachment.

Comment 2 Jan Grulich 2017-01-19 09:15:37 UTC
Should be already fixed in upstream, granting dev_ack.

Comment 3 Tsukahara Ken 2017-01-19 10:50:13 UTC
OK. I just have sent a pull-request in the github.
https://github.com/TigerVNC/tigervnc/pull/402

Comment 4 Jan Grulich 2017-01-19 10:59:30 UTC
Perfect, I actually meant that it's already fixed in upstream, but apparently I was looking into something else. Thanks.

Comment 5 Tsukahara Ken 2017-01-19 16:30:24 UTC
It just has been merged into upstream.
https://github.com/TigerVNC/tigervnc/pull/402

Comment 9 errata-xmlrpc 2017-08-01 20:50:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2000