Bug 1496204

(In reply to boo kheng khoo from comment #0)
> Version-Release number of selected component (if applicable):
> RHEL 7.3 (Maipo)
> NetworkManager 1.4.0-14.el7_3

A bit of a blind shot (haven't really looked in detail) but judging only by the version number this might be a duplicate: Bug 1436650 fixed a leak in 7.3's NM.

Can you try with NetworkManager-1.4.0-18.el7_3 or later and see if the problem persists?

If so, there's also bug 1461643 with an ongoing investigation of memory consumption by NM.

To me this sounds like a duplicate of bug 1461643.

Can you provide the requested information from  https://bugzilla.redhat.com/show_bug.cgi?id=1461643#c27

Thanks.

(In reply to Josep 'Pep' Turro Mauri from comment #2)
> (In reply to boo kheng khoo from comment #0)
> > Version-Release number of selected component (if applicable):
> > RHEL 7.3 (Maipo)
> > NetworkManager 1.4.0-14.el7_3
> 
> A bit of a blind shot (haven't really looked in detail) but judging only by
> the version number this might be a duplicate: Bug 1436650 fixed a leak in
> 7.3's NM.
> 
> Can you try with NetworkManager-1.4.0-18.el7_3 or later and see if the
> problem persists?
> 
> If so, there's also bug 1461643 with an ongoing investigation of memory
> consumption by NM.

I could not find NetworkManager version 1.4.0-18.el7_3 anywhere. The yum repo is a private mirror; version 1.8.0-9.el7 is available, and it has the same issue as version 1.4.0-14.el7_3.

(In reply to Thomas Haller from comment #3)
> To me this sounds like a duplicate of bug 1461643.
> 
> Can you provide the requested information from 
> https://bugzilla.redhat.com/show_bug.cgi?id=1461643#c27
> 
> Thanks.

This bug does seems like 1461643.

Attaching the log file from valgrind requested; the log is captured when i am starting and stopping docker container in a loop.

two files:
* log.txt
* proc-status.txt

Created attachment 1331225 [details]
valgrind log

Created attachment 1331226 [details]
additional info when running valgrind

Hi,

the latest NM package for RHEL 7.3 is 1.4.0-20, which is available in
the following repositories:

https://access.redhat.com/downloads/content/NetworkManager/1.4.0-20.el7_3/x86_64/fd431d51/package

and it resolves the memory leak reported in bug 1436650. Please try
upgrading to that version and check if the issue still happens.

If it does, then probably this is a duplicate of bug 1461643, which in
under investigation. It would be useful if you could run the valgrind
massif tool following the steps reported here:

https://bugzilla.redhat.com/show_bug.cgi?id=1461643#c20

(not the steps from previous comments) and attach the output, thanks!

loaded NM version 1.4.0-20, and tested starting/stop docker containers for about 30min.

when running NM as service, not seeing NM increasing memory consumption continuously, memory usage reached 8340kb and stayed there during the 30min test:
[root@ip-1-70-141-110 ~]# top -p $(pgrep NetworkManager -o) -b -n 1
top - 14:31:32 up  3:01,  3 users,  load average: 1.93, 3.00, 3.24
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.7 us, 18.6 sy,  0.0 ni, 54.7 id, 13.9 wa,  0.0 hi,  0.1 si,  0.0 st
KiB Mem :  3881936 total,   158132 free,   206632 used,  3517172 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  3118388 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 5847 root      20   0  429264   8340   6088 S   0.0  0.2   0:43.54 NetworkManager

----

when running with valgrind, memory usage is much higher, starting at ~90mb, then slowly increasing to ~124mb at the end of the test run. if i have leave the test running, i think the memory usage would probably continue to increase slowly. the rate of increase is much lower than NM version 1.4.0-14, the increase is probably due to valgrind, not NM.

attaching the valgrind log.

Created attachment 1331553 [details]
more valgrind log

(In reply to boo kheng khoo from comment #9)
> loaded NM version 1.4.0-20, and tested starting/stop docker containers for
> about 30min.
>
> when running NM as service, not seeing NM increasing memory consumption
> continuously, memory usage reached 8340kb and stayed there during the 30min
> test:

If NetworkManager-1.4.0-20 does not show the leak, I think we can
close this bug as duplicate of bug 1436650.

> when running with valgrind, memory usage is much higher, starting at ~90mb,
> then slowly increasing to ~124mb at the end of the test run. if i have leave
> the test running, i think the memory usage would probably continue to
> increase slowly. the rate of increase is much lower than NM version
> 1.4.0-14, the increase is probably due to valgrind, not NM.

Yes, valgrind needs its own memory for internal bookkeeping, which can
change over time.

> attaching the valgrind log.
>
> ==12199== Memcheck, a memory error detector
> ==12199== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
> ==12199== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
> ==12199== Command: /sbin/NetworkManager -d --log-level=TRACE

Thanks. This is the output of valgrind-memcheck, which does memory
access checking and tries to detect leaks, but unfortunately it
doesn't detect all possible leaks. For this reason I requested in
comment 8 to use another command, which launches valgrind-massif.

valgrind-massif simply takes snapshots of the allocations on the heap
at regular intervals and saves them to a file that can be later
analyzed. Anyway, since the memory consumption is stable with the last
NM package, there is no need for more logs.

Hi,

I believe this is a duplicate of bug 1461643 and the patch there should fix this leak.

*** This bug has been marked as a duplicate of bug 1461643 ***