Bug 1619416
Summary: | memory grows until swap is 100% utilized and some brick daemons crashes during creating of large number of small files | ||
---|---|---|---|
Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Martin Bukatovic <mbukatov> |
Component: | protocol | Assignee: | Amar Tumballi <atumball> |
Status: | CLOSED ERRATA | QA Contact: | Karan Sandha <ksandha> |
Severity: | medium | Docs Contact: | |
Priority: | high | ||
Version: | rhgs-3.4 | CC: | amukherj, apaladug, nchilaka, rhs-bugs, rkavunga, sankarshan, sheggodu, smali, storage-qa-internal, vdas |
Target Milestone: | --- | ||
Target Release: | RHGS 3.4.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | glusterfs-3.12.2-17 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-09-04 06:52:17 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1503137 | ||
Attachments: |
Description
Martin Bukatovic
2018-08-20 19:19:26 UTC
Created attachment 1477364 [details]
screenshot usm1 (affected cluster): host dashboard with crash and growing memory utilization visible
See 2 events in Host dashboard of affected storage machine:
1st the memory runs out (Swap Utilization chart peaks at 100%), brick crashes
(see Brick Down counter) and copying stops (Total Brick Utilization
chart stops gworing, Brick IOPS chart shows goes down to zero).
Then the workload is stopped (no IOPS reported), but the memory keeps growing
until it reaches 100% of swap utilization again and then 2nd brick daemon on
the machine crashes.
Created attachment 1477367 [details]
screenshot of host dashboard from cluster using older builds, which doesn't crash during same workload
Screenshot from another cluster (usm1), which uses older gluster and wa builds,
where gluster doesn't crash during the same workload.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607 |