Bug 1296972

Summary: memory leak in 4.3.3.fc23 and 4.4-rc8.rawhide
Product: [Fedora] Fedora Reporter: Thorsten Leemhuis <fedora>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED WORKSFORME QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: andrej, gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, mchehab, stuart
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1297188 (view as bug list) Environment:
Last Closed: 2016-02-09 16:11:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
'fpaste --sysinfo --printonly' on the system where it used to happen none

Description Thorsten Leemhuis 2016-01-08 15:20:53 UTC 
Description of problem:
I'm seeing memory leaks in 4.3.3-300.fc23.x86_64 as well as in 4.4.0-0.rc8.git0.1.fc24.x86_64 and 4.4.0-0.rc8.git1.1.fc24.x86_64; 4.2.8-300.fc23.x86_64 otoh works fine. 

A minute or two after boot I see this in slabtop with 4.3.3-300.fc23.x86_64
  11900  11842  99%    0,56K    425       28      6800K skbuff_head_cache  
About 25 Minutes later a lot more memory is used already:
 125804 125764  99%    0,56K   4493       28     71888K skbuff_head_cache
From experiences with earlier boots I assume the kernel will run out of memory after a few hours and panic. The problem doesn't show up in 4.2.8-300.fc23.x86_64; with 4.4.0-0.rc8 it's a different slab-name where the memory is leaking slowly (kmalloc-256 iirc)

Additional info:
I consider this mostly a JFYI bug to say "I'm seeing a regression when going from 4.2 to 4.3" and to make tracking easier in case other see this, too. I guess it's something that is pretty specific to my configuration, as I'd assume other would have noticed the problem already (I'm seeing it only on one machine). I'll try to investigate further next week. Any hints how to find the root cause are appreciated.
Comment 1 Stuart D Gathman 2016-02-01 18:18:53 UTC 
This prevents f23 systems from staying up more than a few hours.  Likely a duplicate: https://bugzilla.redhat.com/show_bug.cgi?id=1303270

The network DoS is interesting twist for the OOM panic
Comment 2 Stuart D Gathman 2016-02-01 18:29:08 UTC 
This prevents f23 systems from staying up more than a few hours.  Likely a duplicate: 

https://bugzilla.redhat.com/show_bug.cgi?id=1303270

The network DoS is interesting twist for the OOM panic

Could you post fpaste --sysinfo --printonly  ?

Not everyone sees this leak, and the rate varies among those who do.
Comment 3 Josh Boyer 2016-02-03 14:29:44 UTC 
Thorsten, what kind of machine is this and what modules are loaded?  Also, is this using ethernet for networking or wifi?
Comment 4 Thorsten Leemhuis 2016-02-03 14:40:31 UTC 
Created attachment 1120802 [details]
'fpaste --sysinfo --printonly' on the system where it used to happen

(In reply to Stuart D Gathman from comment #2)
> Could you post fpaste --sysinfo --printonly  ?
here it is
Comment 5 Thorsten Leemhuis 2016-02-03 14:45:07 UTC 
(In reply to Josh Boyer from comment #3)
> Thorsten, what kind of machine is this and what modules are loaded?
I just attached some infos.

> Also, is this using ethernet for networking or wifi?

It's a desktop machine connected to the ethernet. But it has a wifi chip connected via USB.

Problem is: I haven't seen the problem since some time (I think it was around the time when 4.3 got into updates proper for f23). I'm not totally sure, but it seems to me that I changed something in the configuration that made it vanish, as it suddenly was gone on kernels where it iirc had shown up earlier. But I'm not totally sure and I have no idea what change might be responsible. :-/
Comment 6 Stuart D Gathman 2016-02-09 15:58:47 UTC 
The USB Wifi was the issue.  This scratch build fixes the USB Wifi leak:

http://koji.fedoraproject.org/koji/taskinfo?taskID=12806302

Upstream problem description:
https://bugzilla.redhat.com/show_bug.cgi?id=1303270#c36
Comment 7 Josh Boyer 2016-02-09 16:11:59 UTC 
I'm going to close this report out then.  Thorsten, if you can recreate it at some point, please reopen.