Bug 1398315
Summary: | [compound FOPs]: Memory leak while doing FOPs with brick down | ||||||
---|---|---|---|---|---|---|---|
Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Nag Pavan Chilakam <nchilaka> | ||||
Component: | replicate | Assignee: | Krutika Dhananjay <kdhananj> | ||||
Status: | CLOSED ERRATA | QA Contact: | Nag Pavan Chilakam <nchilaka> | ||||
Severity: | urgent | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | rhgs-3.2 | CC: | amukherj, kdhananj, mchangir, rhs-bugs, sasundar, sbhaloth, storage-qa-internal | ||||
Target Milestone: | --- | ||||||
Target Release: | RHGS 3.2.0 | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | glusterfs-3.8.4-10 | Doc Type: | If docs needed, set a value | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | |||||||
: | 1399578 (view as bug list) | Environment: | |||||
Last Closed: | 2017-03-23 05:51:04 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1351528, 1399578, 1399891 | ||||||
Attachments: |
|
Description
Nag Pavan Chilakam
2016-11-24 12:37:15 UTC
and here comes the OOM Kill :) [Thu Nov 24 18:13:50 2016] glusterfs invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0 [Thu Nov 24 18:13:50 2016] glusterfs cpuset=/ mems_allowed=0-1 [Thu Nov 24 18:13:50 2016] CPU: 3 PID: 2953 Comm: glusterfs Not tainted 3.10.0-510.el7.x86_64 #1 [Thu Nov 24 18:13:50 2016] Hardware name: Supermicro X9DRW-3LN4F+/X9DRW-3TF+/X9DRW-3LN4F+/X9DRW-3TF+, BIOS 1.0b 05/29/2012 [Thu Nov 24 18:13:50 2016] ffff880475a93ec0 000000002e7f78c3 ffff88046bb23990 ffffffff81685ccc [Thu Nov 24 18:13:50 2016] ffff88046bb23a20 ffffffff81680c77 ffffffff812ae65b ffff880476e27d00 [Thu Nov 24 18:13:50 2016] ffff880476e27d18 ffffffff00000202 fffeefff00000000 0000000000000001 [Thu Nov 24 18:13:50 2016] Call Trace: [Thu Nov 24 18:13:50 2016] [<ffffffff81685ccc>] dump_stack+0x19/0x1b [Thu Nov 24 18:13:50 2016] [<ffffffff81680c77>] dump_header+0x8e/0x225 [Thu Nov 24 18:13:50 2016] [<ffffffff812ae65b>] ? cred_has_capability+0x6b/0x120 [Thu Nov 24 18:13:50 2016] [<ffffffff8113cb03>] ? delayacct_end+0x33/0xb0 [Thu Nov 24 18:13:50 2016] [<ffffffff8118460e>] oom_kill_process+0x24e/0x3c0 [Thu Nov 24 18:13:50 2016] [<ffffffff81184e46>] out_of_memory+0x4b6/0x4f0 [Thu Nov 24 18:13:50 2016] [<ffffffff81681780>] __alloc_pages_slowpath+0x5d7/0x725 [Thu Nov 24 18:13:50 2016] [<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420 [Thu Nov 24 18:13:50 2016] [<ffffffff811d209a>] alloc_pages_vma+0x9a/0x150 [Thu Nov 24 18:13:50 2016] [<ffffffff811c2e8b>] read_swap_cache_async+0xeb/0x160 [Thu Nov 24 18:13:50 2016] [<ffffffff811c2fa8>] swapin_readahead+0xa8/0x110 [Thu Nov 24 18:13:50 2016] [<ffffffff811b120c>] handle_mm_fault+0xb1c/0xfe0 [Thu Nov 24 18:13:50 2016] [<ffffffff81691794>] __do_page_fault+0x154/0x450 [Thu Nov 24 18:13:50 2016] [<ffffffff81691ac5>] do_page_fault+0x35/0x90 [Thu Nov 24 18:13:50 2016] [<ffffffff8168dfc0>] ? bstep_iret+0xf/0xf [Thu Nov 24 18:13:50 2016] [<ffffffff8168dd88>] page_fault+0x28/0x30 [Thu Nov 24 18:13:50 2016] Mem-Info: [Thu Nov 24 18:13:50 2016] active_anon:3322839 inactive_anon:510929 isolated_anon:0 active_file:174 inactive_file:754 isolated_file:0 unevictable:0 dirty:0 writeback:136 unstable:0 slab_reclaimable:11575 slab_unreclaimable:22836 mapped:291 shmem:742 pagetables:45178 bounce:0 free:32274 free_pcp:30 free_cma:0 [Thu Nov 24 18:13:50 2016] Node 0 DMA free:15848kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15932kB managed:15848kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [Thu Nov 24 18:13:50 2016] lowmem_reserve[]: 0 1763 7777 7777 [Thu Nov 24 18:13:50 2016] Node 0 DMA32 free:33960kB min:10020kB low:12524kB high:15028kB active_anon:1239140kB inactive_anon:445404kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:2052108kB managed:1807368kB mlocked:0kB dirty:0kB writeback:0kB mapped:612kB shmem:608kB slab_reclaimable:1464kB slab_unreclaimable:8624kB kernel_stack:336kB pagetables:3624kB unstable:0kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [Thu Nov 24 18:13:50 2016] lowmem_reserve[]: 0 0 6014 6014 [Thu Nov 24 18:13:50 2016] Node 0 Normal free:34060kB min:34180kB low:42724kB high:51268kB active_anon:4993472kB inactive_anon:713816kB active_file:632kB inactive_file:3244kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:6291456kB managed:6158340kB mlocked:0kB dirty:0kB writeback:344kB mapped:456kB shmem:2316kB slab_reclaimable:13080kB slab_unreclaimable:44936kB kernel_stack:3136kB pagetables:50900kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:20840 all_unreclaimable? yes [Thu Nov 24 18:13:50 2016] lowmem_reserve[]: 0 0 0 0 [Thu Nov 24 18:13:50 2016] Node 1 Normal free:45228kB min:45820kB low:57272kB high:68728kB active_anon:7058744kB inactive_anon:884496kB active_file:64kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8388608kB managed:8255248kB mlocked:0kB dirty:0kB writeback:200kB mapped:96kB shmem:44kB slab_reclaimable:31756kB slab_unreclaimable:37784kB kernel_stack:2400kB pagetables:126188kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:8515 all_unreclaimable? yes [Thu Nov 24 18:13:50 2016] lowmem_reserve[]: 0 0 0 0 [Thu Nov 24 18:13:50 2016] Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15848kB [Thu Nov 24 18:13:50 2016] Node 0 DMA32: 1013*4kB (UM) 688*8kB (UEM) 266*16kB (UEM) 54*32kB (UEM) 16*64kB (UEM) 6*128kB (EM) 7*256kB (EM) 5*512kB (M) 8*1024kB (UEM) 2*2048kB (M) 0*4096kB = 33972kB [Thu Nov 24 18:13:50 2016] Node 0 Normal: 206*4kB (UEM) 158*8kB (UEM) 87*16kB (UEM) 59*32kB (UEM) 87*64kB (UEM) 38*128kB (UM) 24*256kB (UE) 6*512kB (UEM) 10*1024kB (M) 0*2048kB 0*4096kB = 35256kB [Thu Nov 24 18:13:50 2016] Node 1 Normal: 164*4kB (UEM) 114*8kB (UEM) 68*16kB (UEM) 47*32kB (UEM) 17*64kB (UEM) 6*128kB (UM) 11*256kB (UEM) 35*512kB (UM) 19*1024kB (UM) 0*2048kB 0*4096kB = 46208kB [Thu Nov 24 18:13:50 2016] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Nov 24 18:13:50 2016] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Nov 24 18:13:50 2016] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Nov 24 18:13:50 2016] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Nov 24 18:13:50 2016] 28778 total pagecache pages [Thu Nov 24 18:13:50 2016] 27044 pages in swap cache [Thu Nov 24 18:13:50 2016] Swap cache stats: add 2112210, delete 2085166, find 18057/22414 [Thu Nov 24 18:13:50 2016] Free swap = 0kB [Thu Nov 24 18:13:50 2016] Total swap = 8257532kB [Thu Nov 24 18:13:50 2016] 4187026 pages RAM [Thu Nov 24 18:13:50 2016] 0 pages HighMem/MovableOnly [Thu Nov 24 18:13:50 2016] 127825 pages reserved [Thu Nov 24 18:13:50 2016] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [Thu Nov 24 18:13:50 2016] [ 731] 0 731 9204 172 21 49 0 systemd-journal [Thu Nov 24 18:13:50 2016] [ 752] 0 752 67411 0 34 608 0 lvmetad [Thu Nov 24 18:13:50 2016] [ 768] 0 768 11319 1 23 245 -1000 systemd-udevd [Thu Nov 24 18:13:50 2016] [ 1091] 0 1091 13854 23 28 87 -1000 auditd [Thu Nov 24 18:13:50 2016] [ 1113] 0 1113 4860 81 14 38 0 irqbalance [Thu Nov 24 18:13:50 2016] [ 1116] 81 1116 8207 95 17 52 -900 dbus-daemon [Thu Nov 24 18:13:50 2016] [ 1119] 997 1119 28962 47 26 50 0 chronyd [Thu Nov 24 18:13:50 2016] [ 1127] 998 1127 132067 81 55 1658 0 polkitd [Thu Nov 24 18:13:50 2016] [ 1128] 0 1128 6048 43 16 30 0 systemd-logind [Thu Nov 24 18:13:50 2016] [ 1131] 0 1131 31556 26 19 130 0 crond [Thu Nov 24 18:13:50 2016] [ 1141] 0 1141 81800 261 82 4781 0 firewalld [Thu Nov 24 18:13:50 2016] [ 1148] 0 1148 27509 1 10 31 0 agetty [Thu Nov 24 18:13:50 2016] [ 1150] 0 1150 109534 294 68 345 0 NetworkManager [Thu Nov 24 18:13:50 2016] [ 1250] 0 1250 28206 1 55 3122 0 dhclient [Thu Nov 24 18:13:50 2016] [ 1508] 0 1508 54944 164 38 135 0 rsyslogd [Thu Nov 24 18:13:50 2016] [ 1511] 0 1511 138288 91 89 2576 0 tuned [Thu Nov 24 18:13:50 2016] [ 1516] 0 1516 28335 1 11 38 0 rhsmcertd [Thu Nov 24 18:13:50 2016] [ 1538] 0 1538 20617 25 42 189 -1000 sshd [Thu Nov 24 18:13:50 2016] [ 1552] 0 1552 26971 0 9 24 0 rhnsd [Thu Nov 24 18:13:50 2016] [ 2331] 0 2331 22244 16 41 239 0 master [Thu Nov 24 18:13:50 2016] [ 2363] 89 2363 22270 15 44 235 0 pickup [Thu Nov 24 18:13:50 2016] [ 2365] 89 2365 22287 14 44 236 0 qmgr [Thu Nov 24 18:13:50 2016] [ 2869] 0 2869 35726 28 71 291 0 sshd [Thu Nov 24 18:13:50 2016] [ 2873] 0 2873 29316 81 15 492 0 bash [Thu Nov 24 18:13:50 2016] [ 2951] 0 2951 22585439 3780372 43885 2041242 0 glusterfs [Thu Nov 24 18:13:50 2016] [ 2969] 0 2969 35726 26 68 291 0 sshd [Thu Nov 24 18:13:50 2016] [ 2973] 0 2973 28846 72 14 39 0 bash [Thu Nov 24 18:13:50 2016] [ 2998] 0 2998 31927 68 17 70 0 screen [Thu Nov 24 18:13:50 2016] [ 2999] 0 2999 38218 4753 32 4734 0 bash [Thu Nov 24 18:13:50 2016] [ 3674] 0 3674 35726 316 72 0 0 sshd [Thu Nov 24 18:13:50 2016] [ 3678] 0 3678 28846 109 12 0 0 bash [Thu Nov 24 18:13:50 2016] [ 3815] 0 3815 130941 18547 177 0 0 yum [Thu Nov 24 18:13:50 2016] Out of memory: Kill process 2951 (glusterfs) score 929 or sacrifice child [Thu Nov 24 18:13:50 2016] Killed process 2951 (glusterfs) total-vm:90341756kB, anon-rss:15121488kB, file-rss:0kB, shmem-rss:0kB hit this while validating RFE 1360978 - [RFE]Reducing number of network round trips As discussed in Bug triage meeting providing qa_ack http://review.gluster.org/#/c/15965/ <--- patch posted in upstream master. Moving this bug to POST state. -Krutika I am failing this bug as the memory is not getting released even after fops are completed Refer below bz for more details wrt validation 1398311 - [compound FOPs]:in replica pair one brick is down the other Brick process and fuse client process consume high memory at a increasing pace upstream mainline patch http://review.gluster.org/#/c/16210/ posted for review. downstream patch : https://code.engineering.redhat.com/gerrit/#/c/93488/ BZ added to erratum https://errata.devel.redhat.com/advisory/24866 Moving to ON_QA on_qa validation: ================= there was minimal mem consumption when compared to before fix Hence moving the testcase to pass, ie the bz to verified had a 1x2 vol with cfops enabled mounted vol one two client cli45(el7.3) and cli24(el6.7) From cli45, started to create a 10gb file using dd after about 3gb was created brought down b1 kept tracking client side and brick process memeory consumption below are the details, in a loop for every 30sec cli45: before file create started: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 41632 3196 S 0.0 0.3 0:00.05 glusterfs after create was issued: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 46764 3804 S 75.0 0.3 0:21.34 glusterfs ==>it shot by 5MB but from there on it was constant or minimal increase before brick b1 was killed: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 46848 3824 S 66.7 0.3 4:49.47 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 46848 3824 S 73.3 0.3 5:11.78 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 46848 3824 S 62.5 0.3 5:33.70 glusterfs after b1 was killed: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 643012 46848 3824 S 68.8 0.3 5:55.95 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47264 3936 S 68.8 0.3 6:17.59 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47268 3936 S 73.3 0.3 6:38.93 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47268 3936 S 68.8 0.3 7:00.40 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47268 3936 S 81.2 0.3 7:21.79 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47268 3936 S 75.0 0.3 7:42.81 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 645148 47328 3936 S 68.8 0.3 12:40.12 glusterfs after b1 was brought up using vol force start: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 778272 49512 4028 S 93.8 0.3 13:55.20 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 778272 49516 4032 S 106.2 0.3 14:26.07 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 778272 49516 4032 S 100.0 0.3 14:56.36 glusterfs at end of file create complete 10gb file PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 776220 49512 4040 S 0.0 0.3 20:48.97 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 776220 49512 4040 S 0.0 0.3 20:48.97 glusterfs PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2985 root 20 0 776220 49512 4040 S 0.0 0.3 20:48.97 glusterfs brick process(the replica brick b2 which was alive) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1117984 48020 4024 S 37.5 0.6 0:01.36 glusterfsd Tue Dec 27 12:53:45 IST 2016 ============>started file1 create PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1117984 48272 4164 S 43.8 0.6 0:11.84 glusterfsd Tue Dec 27 12:54:15 IST 2016 ===>killing b1 after about 3gb created PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48520 4260 S 37.5 0.6 2:44.61 glusterfsd Tue Dec 27 13:01:48 IST 2016 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48784 4264 S 56.2 0.6 3:00.91 glusterfsd Tue Dec 27 13:02:18 IST 2016 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48784 4264 S 62.5 0.6 3:19.55 glusterfsd PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48784 4264 S 56.2 0.6 7:58.98 glusterfsd Tue Dec 27 13:10:21 IST 2016 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48784 4264 S 52.9 0.6 8:17.68 glusterfsd Tue Dec 27 13:10:51 IST 2016 ===>i dont see any mem leak even after 6gb completed hence restarting brick PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1250084 48636 4116 S 62.5 0.6 8:36.46 glusterfsd Tue Dec 27 13:11:21 IST 2016 ====>started vol using force PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1316648 48848 4132 S 61.1 0.6 8:59.89 glusterfsd Tue Dec 27 13:11:52 IST 2016 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1316648 48848 4132 S 25.0 0.6 9:20.06 glusterfsd Tue Dec 27 13:12:22 IST 2016 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28670 root 20 0 1316648 49072 4152 S 37.5 0.6 9:33.38 glusterfsd Tue Dec 27 13:12:52 IST 2016 So there was minimal mem consumption when compared to before fix Hence moving the testcase to pass, ie the bz to verified Created attachment 1235486 [details]
on_qa top output validation log server and client
[root@dhcp35-37 ~]# gluster v heal comp info Brick 10.70.35.37:/rhs/brick4/comp Status: Connected Number of entries: 0 Brick 10.70.35.116:/rhs/brick4/comp /dir1/file1 Status: Connected Number of entries: 1 [root@dhcp35-37 ~]# gluster v heal comp info Brick 10.70.35.37:/rhs/brick4/comp Status: Connected Number of entries: 0 Brick 10.70.35.116:/rhs/brick4/comp Status: Connected Number of entries: 0 [root@dhcp35-37 ~]# gluster v info comp Volume Name: comp Type: Replicate Volume ID: 1dae5832-1625-4287-b478-3be79be62d68 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.70.35.37:/rhs/brick4/comp Brick2: 10.70.35.116:/rhs/brick4/comp Options Reconfigured: cluster.use-compound-fops: on transport.address-family: inet performance.readdir-ahead: on nfs.disable: on nfs-ganesha: enable cluster.enable-shared-storage: enable [root@dhcp35-37 ~]# gluster v status comp Status of volume: comp Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.35.37:/rhs/brick4/comp 49155 0 Y 23814 Brick 10.70.35.116:/rhs/brick4/comp 49155 0 Y 28670 Self-heal Daemon on localhost N/A N/A Y 23834 Self-heal Daemon on dhcp35-239.lab.eng.blr. redhat.com N/A N/A Y 17583 Self-heal Daemon on dhcp35-196.lab.eng.blr. redhat.com N/A N/A Y 22197 Self-heal Daemon on dhcp35-116.lab.eng.blr. redhat.com N/A N/A Y 20028 Self-heal Daemon on dhcp35-135.lab.eng.blr. redhat.com N/A N/A Y 16450 Self-heal Daemon on dhcp35-8.lab.eng.blr.re dhat.com N/A N/A Y 16470 Task Status of Volume comp ------------------------------------------------------------------------------ There are no active volume tasks [root@dhcp35-37 ~]# Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2017-0486.html |