Bug 1432219

Summary: dnf in rawhide uses much more RAM than dnf in Fedora 25
Product: [Fedora] Fedora Reporter: Richard W.M. Jones <rjones>
Component: dnfAssignee: rpm-software-management
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 27CC: jmracek, packaging-team-maint, rjones, rpm-software-management, vmukhame
Target Milestone: ---Keywords: Performance, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-07 11:06:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 910269, 1432213    

Description Richard W.M. Jones 2017-03-14 19:45:16 UTC 
Description of problem:

... causing this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1432213

libguestfs runs dnf in an appliance with a mere 512 MB of RAM.  Previously that worked
fine, but in current Rawhide, everything fails with errors like:

[  204.467753] dnf invoked oom-killer: gfp_mask=0x24201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=0, order=0, oom_score_adj=0
[  204.470038] dnf cpuset=/ mems_allowed=0
[  204.470849] CPU: 0 PID: 619 Comm: dnf Tainted: G      D W       4.9.13-201.fc25.x86_64+debug #1
[  204.472509] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-1.fc25589] Mem-Info:
[  312.006153] active_anon:102189 inactive_anon:19 isolated_anon:0
[  312.006153]  active_file:35 inactive_file:37 isolated_file:0
[  312.006153]  unevictable:0 dirty:0 writeback:0 unstable:0
[  312.006153]  slab_reclaimable:5224 slab_unreclaimable:5739
[  312.006153]  mapped:48 shmem:29 pagetables:459 bounce:0
[  312.006153]  free:1117 free_pcp:174 free_cma:0
[  312.010844] Node 0 active_anon:408756kB inactive_anon:76kB active_file:140kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:192kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 116kB writeback_tmp:0kB unstable:0kB pages_scanned:34 all_unreclaimable? no
[  312.014950] Node 0 DMA free:1844kB min:92kB low:112kB high:132kB active_anon:13736kB inactive_anon:0kB active_file:12kB inactive_file:4kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB slab_reclaimable:48kB slab_unreclaimable:24kB kernel_stack:16kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[  312.019434] lowmem_reserve[]: 0 438 438 438 438
[  312.020250] Node 0 DMA32 free:2624kB min:2628kB low:3284kB high:3940kB active_anon:395020kB inactive_anon:76kB active_file:152kB inactive_file:120kB unevictable:0kB writepending:0kB present:495608kB managed:454312kB mlocked:0kB slab_reclaimable:20848kB slab_unreclaimable:22932kB kernel_stack:1136kB pagetables:1836kB bounce:0kB free_pcp:696kB local_pcp:696kB free_cma:0kB
[  312.024943] lowmem_reserve[]: 0 0 0 0 0
[  312.025628] Node 0 DMA: 3*4kB (U) 5*8kB (UM) 4*16kB (UM) 4*32kB (U) 1*64kB (U) 0*128kB 0*256kB 1*512kB (M) 1*1024kB (U) 0*2048kB 0*4096kB = 1844kB
[  312.027975] Node 0 DMA32: 84*4kB (UMEH) 21*8kB (UMEH) 6*16kB (UMEH) 2*32kB (M) 1*64kB (H) 1*128kB (M) 1*256kB (M) 1*512kB (H) 1*1024kB (H) 0*2048kB 0*4096kB = 2648kB
[  312.030588] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[  312.031834] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[  312.033067] 96 total pagecache pages
[  312.033603] 0 pages in swap cache
[  312.034102] Swap cache stats: add 0, delete 0, find 0/0
[  312.034868] Free swap  = 0kB
[  312.035296] Total swap = 0kB
[  312.035711] 127900 pages RAM
[  312.036151] 0 pages HighMem/MovableOnly
[  312.036716] 10345 pages reserved
[  312.037191] 0 pages cma reserved
[  312.037667] 0 pages hwpoisoned
[  312.038126] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
[  312.039361] [  400]     0   400    11585      188      24       3        0         -1000 systemd-udevd
[  312.040734] [  479]     0   479    22299     3077      42       3        0             0 dhclient
[  312.042018] [  497]     0   497    54191    21799      80       4        0             0 guestfsd
[  312.043315] [  500]     0   500    31598       86      27       3        0             0 lvmetad
[  312.044585] [  619]     0   619   189475    76957     274       3        0             0 dnf
[  312.045829] Out of memory: Kill process 619 (dnf) score 637 or sacrifice child
[  312.046867] Killed process 619 (dnf) total-vm:757900kB, anon-rss:307828kB, file-rss:0kB, shmem-rss:0kB
commandrvf: stdout=n stderr=n flags=0x0
commandrvf: umount /sysroot/sys
[  312.089002] oom_reaper: reaped process 619 (dnf), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

and:

Running transaction
  Installing  : gdk-pixbuf2-2.36.5-1.fc26.x86_64                          1/157 
error: Couldn't fork %post(gdk-pixbuf2-2.36.5-1.fc26.x86_64): Cannot allocate memory
  Installing  : libjpeg-turbo-1.5.1-0.fc26.x86_64                         2/157 
error: Couldn't fork %post(libjpeg-turbo-1.5.1-0.fc26.x86_64): Cannot allocate memory
  Installing  : libwayland-client-1.13.0-1.fc26.x86_64                    3/157 
error: Couldn't fork %post(libwayland-client-1.13.0-1.fc26.x86_64): Cannot allocate memory
  Installing  : atk-2.23.4-1.fc26.x86_64                                  4/157 
error: Couldn't fork %post(atk-2.23.4-1.fc26.x86_64): Cannot allocate memory
  Installing  : libwayland-server-1.13.0-1.fc26.x86_64                    5/157 

Version-Release number of selected component (if applicable):

Current Rawhide version of dnf.

How reproducible:

100%

Steps to Reproduce:
1. Run dnf in a machine with 512MB of RAM and no swap.
Comment 1 Richard W.M. Jones 2017-03-14 19:57:48 UTC 
FWIW increasing the RAM to 1 GB (from 512 MB) works.
Comment 2 Honza Silhan 2017-03-22 12:44:31 UTC 
We should try to reproduce it inside isolated cgroup with limited RAM.
Comment 3 Jan Kurik 2017-08-15 06:57:03 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.
Comment 4 Jaroslav Mracek 2017-11-07 10:34:26 UTC 
I have fixed several memory-leaks in dnf, libdnf. Please can you confirm that the issue is still reproducible with "dnf-0:2.7.5-1.fc26.noarch" and "libdnf-0.11.1"?
Comment 5 Richard W.M. Jones 2017-11-07 11:06:36 UTC 
A (long) one-line reproducer is:

$ virt-builder \
    --install fedora-repos-rawhide \
    --edit '/etc/yum.repos.d/fedora-rawhide.repo: s/^enabled=0/enabled=1/' \
    --edit '/etc/yum.repos.d/fedora-rawhide.repo: s/^gpgcheck=1/gpgcheck=0/' \
    --run-command 'dnf -y update dnf dnf-plugins-core libdnf' \
    --install bash,emacs,mlocate,net-tools,nfs-utils,ntp,sudo,telnet \
    fedora-26

To see what versions of dnf, libdnf were installed you have to grep
the /tmp/builder.log file inside the final image:

$ virt-cat fedora-26.img /tmp/builder.log | less

With Rawhide today I happened to get:

dnf 2.7.5-2.fc28
libdnf 0.11.1-2.fc28

These versions do appear to work in the restricted 500 MB environment
of the appliance, and so I am closing this bug as fixed in rawhide.