Bug 910104
| Summary: | OOM issues when running many raid -> thin pool conversions w/ lvmetad running | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Corey Marthaler <cmarthal> |
| Component: | lvm2 | Assignee: | Petr Rockai <prockai> |
| Status: | CLOSED ERRATA | QA Contact: | Cluster QE <mspqa-list> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.4 | CC: | agk, cmarthal, dwysocha, heinzm, jbrassow, msnitzer, prajnoha, prockai, thornber, tlavigne, zkabelac |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | lvm2-2.02.100-1.el6 | Doc Type: | Bug Fix |
| Doc Text: |
Cause: Cached metadata in lvmetad could be leaked under some circumstances during metadata updates.
Consequence: Memory use of lvmetad could continually grow during long periods of time, possibly resulting in out of memory conditions.
Fix: The leak has been fixed.
Result: The memory used by lvmetad is proportional to the amount of metadata it holds at any given time, and can no longer grow without bound over time.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-11-21 23:20:31 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Corey Marthaler
2013-02-11 20:18:42 UTC
dmeventd memory size looks like its leaking memory. Also be aware: Bug 956769 - MD RAID1/10 are leaking memory when they are stopped Although, it would take quite a few iterations for this to become a problem... Ignoring out-of-sequence reply from dmeventd. Expected 5312:0 but received 5241:11 HELLO HELLO 1 This report seems to fall inbetween:
commit 15fdd5c90dda7f00f691668c13d5401206d22021
Date: Wed Jan 16 11:09:37 2013 +0100
and
commit 95372a852bbcacc9f194324832b94fcf1493f7c5
Date: Wed Apr 3 13:46:12 2013 +0200
lvmetad: Fix a memory leak introduced in 15fdd5c90dd.
so I believe this was fixed in 95372a852bbcacc9f194324832b94fcf1493f7c5. Either way, I can't reproduce the problem, and if there was a leak in lvmetad, we'd be still seeing many failures like this one.
So please re-test this one. If it's already been fixed, we'll attach it to the errata for documentation reasons. I'm not seeing this issue any more. Marking verified in the latest rpms. 2.6.32-422.el6.x86_64 lvm2-2.02.100-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 lvm2-libs-2.02.100-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 lvm2-cluster-2.02.100-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 udev-147-2.50.el6 BUILT: Fri Oct 11 12:58:10 CEST 2013 device-mapper-1.02.79-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 device-mapper-libs-1.02.79-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 device-mapper-event-1.02.79-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 device-mapper-event-libs-1.02.79-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 cmirror-2.02.100-6.el6 BUILT: Wed Oct 16 14:26:00 CEST 2013 ============================================================ Iteration 1412 of 10000 started at Fri Oct 18 20:42:30 CEST 2013 ============================================================ SCENARIO - [raid1_to_pool_conversion] Create raid1 volumes and convert them to pool and pool meta volumes lvcreate --type raid1 -m 1 -L 100M -n to_pool_convert snapper_thinp lvcreate --type raid1 -m 1 -L 100M -n to_pool_meta_convert snapper_thinp lvconvert --thinpool snapper_thinp/to_pool_convert --poolmetadata to_pool_meta_convert device-mapper: remove ioctl on failed: Device or resource busy device-mapper: remove ioctl on failed: Device or resource busy lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n origin lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n other1 lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n other2 lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n other3 lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n other4 lvcreate --virtualsize 500M --thinpool snapper_thinp/to_pool_convert -n other5 Making snapshot of origin volume lvcreate -K -s /dev/snapper_thinp/origin -n snap_of_pool_convert Removing volume snapper_thinp/snap_of_pool_convert Removing thin origin and other virtual thin volumes Removing thinpool snapper_thinp/to_pool_convert SCENARIO - [raid10_to_pool_conversion] Create raid10 volumes and convert them to pool and pool meta volumes lvcreate --type raid10 -m 1 -L 100M -n to_pool_convert snapper_thinp lvcreate --type raid10 -m 1 -L 100M -n to_pool_meta_convert snapper_thinp lvconvert --thinpool snapper_thinp/to_pool_convert --poolmetadata to_pool_meta_convert device-mapper: remove ioctl on failed: Device or resource busy device-mapper: remove ioctl on failed: Device or resource busy lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n origin lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n other1 lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n other2 lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n other3 lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n other4 lvcreate --virtualsize 500M -T snapper_thinp/to_pool_convert -n other5 Making snapshot of origin volume lvcreate -K -s /dev/snapper_thinp/origin -n snap_of_pool_convert Removing volume snapper_thinp/snap_of_pool_convert Removing thin origin and other virtual thin volumes Removing thinpool snapper_thinp/to_pool_convert Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1704.html |