Bug 1418536 - Portmap allocates way too much memory (256KB) on stack
Summary: Portmap allocates way too much memory (256KB) on stack
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: 3.10
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jeff Darcy
QA Contact:
URL:
Whiteboard:
Depends On: 1418095
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-02 03:24 UTC by Jeff Darcy
Modified: 2017-03-06 17:44 UTC (History)
2 users (show)

Fixed In Version: glusterfs-3.10.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1418095
Environment:
Last Closed: 2017-03-06 17:44:58 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Jeff Darcy 2017-02-02 03:24:28 UTC
+++ This bug was initially created as a clone of Bug #1418095 +++

Some functions were allocating 64K booleans, which are (crazily) mapped to 4-byte ints, for a total of 256KB per call.  Besides being generally wasteful, this means any code that creates worker threads - e.g. syncops, io-threads translator - must allocate much bigger stacks for each thread *just in case* it calls into portmap.  With multiplexing, this limits the number of threads we can have, and therefore the number of bricks we can support in one process.

--- Additional comment from Worker Ant on 2017-01-31 15:18:23 EST ---

REVIEW: https://review.gluster.org/15745 (libglusterfs+transport+io-threads: fix 256KB stack abuse) posted (#5) for review on master by Jeff Darcy (jdarcy)

--- Additional comment from Worker Ant on 2017-02-01 19:59:29 EST ---

COMMIT: https://review.gluster.org/15745 committed in master by Shyamsundar Ranganathan (srangana) 
------
commit c8a23cc6cd289dd28deb136bf2550f28e2761ef3
Author: Jeff Darcy <jdarcy>
Date:   Thu Oct 27 11:51:47 2016 -0400

    libglusterfs+transport+io-threads: fix 256KB stack abuse
    
    Some functions were allocating 64K booleans, which are (crazily) mapped to
    4-byte ints, for a total of 256KB per call.  Changed to use bitfields instead,
    so usage is now only 8KB per call.  This was the impediment to changing the
    io-threads stack size, so that has been adjusted too.
    
    Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
    BUG: 1418095
    Signed-off-by: Jeff Darcy <jdarcy>
    Reviewed-on: https://review.gluster.org/15745
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: N Balachandran <nbalacha>
    Reviewed-by: Shyamsundar Ranganathan <srangana>

Comment 1 Worker Ant 2017-02-02 03:26:05 UTC
REVIEW: https://review.gluster.org/16511 (libglusterfs+transport+io-threads: fix 256KB stack abuse) posted (#1) for review on release-3.10 by Jeff Darcy (jdarcy)

Comment 2 Worker Ant 2017-02-02 17:32:53 UTC
COMMIT: https://review.gluster.org/16511 committed in release-3.10 by Shyamsundar Ranganathan (srangana) 
------
commit c10507ce75547a7a7899fbf36be650ddc89ba467
Author: Jeff Darcy <jdarcy>
Date:   Thu Oct 27 11:51:47 2016 -0400

    libglusterfs+transport+io-threads: fix 256KB stack abuse
    
    Some functions were allocating 64K booleans, which are (crazily)
    mapped to 4-byte ints, for a total of 256KB per call.  Changed to use
    bitfields instead, so usage is now only 8KB per call.  This was the
    impediment to changing the io-threads stack size, so that has been
    adjusted too.
    
    Backport of:
    > Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
    > BUG: 1418095
    > Reviewed-on: https://review.gluster.org/15745
    
    Change-Id: Ia5dada61703e6bea95f2511da71feb573fc9a429
    BUG: 1418536
    Signed-off-by: Jeff Darcy <jdarcy>
    Reviewed-on: https://review.gluster.org/16511
    Reviewed-by: N Balachandran <nbalacha>
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Vijay Bellur <vbellur>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 3 Shyamsundar 2017-03-06 17:44:58 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.