1202278 – [atomic] OOM with 1024 MB RAM (squashfs includes ostree content)

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1202278 - [atomic] OOM with 1024 MB RAM (squashfs includes ostree content)

Summary: [atomic] OOM with 1024 MB RAM (squashfs includes ostree content)

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	lorax
Sub Component:
Version:	7.1
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Radek Vykydal
QA Contact:	Release Test Team
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1205794 1235558
TreeView+	depends on / blocked

Reported:	2015-03-16 09:45 UTC by Alexander Todorov
Modified:	2015-11-19 09:40 UTC (History)
CC List:	5 users (show)
Fixed In Version:	lorax-19.6.56-1
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Clones:	1235558 (view as bug list)
Environment:
Last Closed:	2015-11-19 09:40:34 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
rdsosreport.txt (83.36 KB, text/plain) 2015-03-19 10:23 UTC, Alexander Todorov	no flags	Details
journalctl.txt (73.86 KB, text/plain) 2015-03-19 10:23 UTC, Alexander Todorov	no flags	Details
console.log from test with Linux/Unknown (55.10 KB, text/plain) 2015-03-20 12:39 UTC, Alexander Todorov	no flags	Details
console.log from ISO test with Unknown/Unknown (36.70 KB, text/plain) 2015-03-20 12:51 UTC, Alexander Todorov	no flags	Details
journalctl.txt from ISO test with Unknown/Unknown (90.16 KB, text/plain) 2015-03-20 12:51 UTC, Alexander Todorov	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHEA-2015:2289	0	normal	SHIPPED_LIVE	lorax bug fix and enhancement update	2015-11-19 09:44:13 UTC

Description Alexander Todorov 2015-03-16 09:45:35 UTC

Description of problem:

During stage1 anaconda dies because there isn't enough memory. I usually see dhclient killed by the kernel. 


Version-Release number of selected component (if applicable):
anaconda 19.31.79-1 for RHEL Atomic Host 7


How reproducible:
Always

Steps to Reproduce:
1. Start a KVM guest in virt-manager
2. Select HTTP install method and configure OS type/variant to Linux/RHEL7 (this is important)
3. Proceed with the default 1024 MB RAM that virt-manager proposes
4. Select a disk image (mine is 20 GB).
5. Boot the installer

Actual results:
The installer starts and early during the process I see dhclient killed by the kernel.

Expected results:


Additional info:
A regular RHEL 7 install with the same amount of RAM is able to proceed to stage 2 and complete the installation. This is the minimum recommended in the docs. 


Also if I'm installing from an ISO or don't specify OS type/variant and Linux/RHEL7 (but Unknown/Unknown) I don't seem to get this error.

Comment 1 Brian Lane 2015-03-18 23:59:11 UTC

What does /proc/cmdline look like?

And the output from journalctl might be helpful.

Comment 2 Alexander Todorov 2015-03-19 10:22:46 UTC

Here's the console log. 

# virsh console rhel7.0-17
Connected to domain rhel7.0-17
Escape character is ^]
100  536M  100  536M    0     0  32.3M      0  0:00:16  0:00:16 --:--:-- 59.7M:--:-- --:--:--     0
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0:--:-- --:--:--     0
dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found
dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/updates.img' failed!
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0:--:-- --:--:--     0
dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found
dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/product.img' failed!
[   28.279696] loop: module loaded
[  OK  ] Started dracut initqueue hook.
         Starting dracut pre-mount hook...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Started dracut pre-mount hook.
[  OK  ] Reached target Initrd Root File System.
         Starting Reload Configuration from the Real Root...
[  OK  ] Started Reload Configuration from the Real Root.
[  OK  ] Reached target Initrd File Systems.
         Starting dracut mount hook...
[   28.468323] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[  OK  ] Started dracut mount hook.
[  OK  ] Reached target Initrd Default Target.
[   38.315862] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
[   38.316954] gzip cpuset=/ mems_allowed=0
[   38.317516] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1
[   38.318481] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[   38.319284]  ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36
[   38.320396]  ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18
[   38.321504]  ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803
[   38.322698] Call Trace:
[   38.323069]  [<ffffffff81603f36>] dump_stack+0x19/0x1b
[   38.323809]  [<ffffffff815feeff>] dump_header+0x8e/0x214
[   38.324587]  [<ffffffff81117803>] ? delayacct_end+0x73/0xb0
[   38.325392]  [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0
[   38.326236]  [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30
[   38.327147]  [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0
[   38.327928]  [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90
[   38.328833]  [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140
[   38.329710]  [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0
[   38.330603]  [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120
[   38.331482]  [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90
[   38.332775]  [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0
[   38.333718]  [<ffffffff811700c8>] shmem_write_begin+0x38/0x40
[   38.334572]  [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290
[   38.335569]  [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0
[   38.336502]  [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0
[   38.337384]  [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0
[   38.338179]  [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0
[   38.338899]  [<ffffffff811c7028>] SyS_write+0x58/0xb0
[   38.340039]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
[   38.340968] Mem-Info:
[   38.341314] Node 0 DMA per-cpu:
[   38.341783] CPU    0: hi:    0, btch:   1 usd:   0
[   38.342799] Node 0 DMA32 per-cpu:
[   38.343359] CPU    0: hi:  186, btch:  31 usd:  68
[   38.344084] active_anon:4440 inactive_anon:57419 isolated_anon:0
[   38.344084]  active_file:10 inactive_file:4 isolated_file:0
[   38.344084]  unevictable:169812 dirty:6 writeback:0 unstable:0
[   38.344084]  free:12228 slab_reclaimable:2870 slab_unreclaimable:2144
[   38.344084]  mapped:2011 shmem:57460 pagetables:269 bounce:0
[   38.344084]  free_cma:0
[   38.348812] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:312kB inactive_anon:2432kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2440kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[   38.354781] lowmem_reserve[]: 0 976 976 976
[   38.355486] Node 0 DMA32 free:44308kB min:44348kB low:55432kB high:66520kB active_anon:17448kB inactive_anon:227244kB active_file:40kB inactive_file:16kB unevictable:670976kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:24kB writeback:0kB mapped:8044kB shmem:227400kB slab_reclaimable:11476kB slab_unreclaimable:8564kB kernel_stack:1280kB pagetables:1076kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:40917 all_unreclaimable? yes
[   38.361570] lowmem_reserve[]: 0 0 0 0
[   38.362254] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB
[   38.365021] Node 0 DMA32: 75*4kB (UEM) 37*8kB (UEM) 16*16kB (UE) 6*32kB (EM) 4*64kB (UE) 2*128kB (UM) 3*256kB (UM) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44308kB
[   38.367961] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   38.369171] 226952 total pagecache pages
[   38.370077] 0 pages in swap cache
[   38.370603] Swap cache stats: add 0, delete 0, find 0/0
[   38.371368] Free swap  = 0kB
[   38.371777] Total swap = 0kB
[   38.372200] 262044 pages RAM
[   38.372609] 0 pages HighMem/MovableOnly
[   38.373161] 7738 pages reserved
[   38.373605] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   38.374711] [   85]     0    85    10296      543      23        0             0 systemd-journal
[   38.375942] [  129]     0   129    76032     1278      29        0         -1000 multipathd
[   38.377239] [  396]    32   396     9977      295      22        0             0 rpcbind
[   38.378417] [  400]     0   400    11121      424      25        0             0 rpc.statd
[   38.379570] [  405]     0   405     5853      108      17        0             0 rpc.idmapd
[   38.380740] [  427]     0   427     8831      460      21        0         -1000 systemd-udevd
[   38.381945] [  587]     0   587     7809      254      18        0             0 plymouthd
[   38.383105] [  686]     0   686    25934     3176      47        0             0 dhclient
[   38.384250] [  940]     0   940     3067      556      12        0             0 dracut-pre-pivo
[   38.385490] [  978]     0   978     2292      232      10        0             0 grep
[   38.386618] [  979]     0   979     1106      156       8        0             0 cpio
[   38.387715] [  980]     0   980     1143      153       7        0             0 gzip
[   38.388814] Out of memory: Kill process 686 (dhclient) score 12 or sacrifice child
[   38.389879] Killed process 686 (dhclient) total-vm:103736kB, anon-rss:12404kB, file-rss:300kB
[   38.981379] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
[   38.983461] gzip cpuset=/ mems_allowed=0
[   38.984089] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1
[   38.985153] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[   38.985958]  ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36
[   38.987097]  ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18
[   38.988233]  ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803
[   38.989422] Call Trace:
[   38.989775]  [<ffffffff81603f36>] dump_stack+0x19/0x1b
[   38.990512]  [<ffffffff815feeff>] dump_header+0x8e/0x214
[   38.991259]  [<ffffffff81117803>] ? delayacct_end+0x73/0xb0
[   38.992035]  [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0
[   38.993287]  [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30
[   38.994251]  [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0
[   38.995042]  [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90
[   38.995934]  [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140
[   38.996742]  [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0
[   38.997548]  [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120
[   38.998424]  [<ffffffff812f4b41>] ? __percpu_counter_add+0x51/0x70
[   38.999310]  [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90
[   39.000600]  [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0
[   39.001467]  [<ffffffff811700c8>] shmem_write_begin+0x38/0x40
[   39.002293]  [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290
[   39.003270]  [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0
[   39.004300]  [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0
[   39.005222]  [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0
[   39.005979]  [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0
[   39.006714]  [<ffffffff811c7028>] SyS_write+0x58/0xb0
[   39.007438]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
[   39.008290] Mem-Info:
[   39.008621] Node 0 DMA per-cpu:
[   39.009097] CPU    0: hi:    0, btch:   1 usd:   0
[   39.009799] Node 0 DMA32 per-cpu:
[   39.010302] CPU    0: hi:  186, btch:  31 usd:  72
[   39.010974] active_anon:1340 inactive_anon:60574 isolated_anon:0
[   39.010974]  active_file:6 inactive_file:7 isolated_file:0
[   39.010974]  unevictable:169815 dirty:0 writeback:0 unstable:0
[   39.010974]  free:12224 slab_reclaimable:2884 slab_unreclaimable:2141
[   39.010974]  mapped:1972 shmem:60605 pagetables:222 bounce:0
[   39.010974]  free_cma:0
[   39.015829] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:132kB inactive_anon:2608kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2608kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[   39.021533] lowmem_reserve[]: 0 976 976 976
[   39.022297] Node 0 DMA32 free:44292kB min:44348kB low:55432kB high:66520kB active_anon:5228kB inactive_anon:239688kB active_file:24kB inactive_file:28kB unevictable:670988kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:0kB writeback:0kB mapped:7888kB shmem:239812kB slab_reclaimable:11532kB slab_unreclaimable:8552kB kernel_stack:1264kB pagetables:888kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:38789 all_unreclaimable? yes
[   39.028944] lowmem_reserve[]: 0 0 0 0
[   39.029892] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB
[   39.032790] Node 0 DMA32: 115*4kB (UEM) 39*8kB (UEM) 18*16kB (UEM) 5*32kB (E) 5*64kB (UEM) 2*128kB (UM) 2*256kB (U) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44292kB
[   39.035500] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   39.036730] 230105 total pagecache pages
[   39.037309] 0 pages in swap cache
[   39.037823] Swap cache stats: add 0, delete 0, find 0/0
[   39.038636] Free swap  = 0kB
[   39.039070] Total swap = 0kB
[   39.039488] 262044 pages RAM
[   39.039896] 0 pages HighMem/MovableOnly
[   39.040451] 7738 pages reserved
[   39.040895] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   39.042144] [   85]     0    85    10296      543      23        0             0 systemd-journal
[   39.043468] [  129]     0   129    76032     1278      29        0         -1000 multipathd
[   39.044641] [  396]    32   396     9977      295      22        0             0 rpcbind
[   39.045854] [  400]     0   400    11121      424      25        0             0 rpc.statd
[   39.047037] [  405]     0   405     5853      108      17        0             0 rpc.idmapd
[   39.048296] [  427]     0   427     8831      460      21        0         -1000 systemd-udevd
[   39.049519] [  587]     0   587     7809      254      18        0             0 plymouthd
[   39.050667] [  940]     0   940     3067      556      12        0             0 dracut-pre-pivo
[   39.051874] [  978]     0   978     2292      232      10        0             0 grep
[   39.052951] [  979]     0   979     1106      156       8        0             0 cpio
[   39.054125] [  980]     0   980     1143      153       7        0             0 gzip
[   39.055275] Out of memory: Kill process 940 (dracut-pre-pivo) score 2 or sacrifice child
[   39.056405] Killed process 978 (grep) total-vm:9168kB, anon-rss:220kB, file-rss:708kB
[   39.058272] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
[   39.059998] gzip cpuset=/ mems_allowed=0
[   39.060603] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1
[   39.061584] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[   39.062416]  ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36
[   39.063537]  ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18
[   39.064657]  ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803
[   39.065909] Call Trace:
[   39.066331]  [<ffffffff81603f36>] dump_stack+0x19/0x1b
[   39.067092]  [<ffffffff815feeff>] dump_header+0x8e/0x214
[   39.067849]  [<ffffffff81117803>] ? delayacct_end+0x73/0xb0
[   39.068655]  [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0
[   39.069469]  [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30
[   39.070352]  [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0
[   39.071124]  [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90
[   39.072089]  [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140
[   39.072939]  [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0
[   39.073734]  [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120
[   39.074615]  [<ffffffff812f4b41>] ? __percpu_counter_add+0x51/0x70
[   39.075489]  [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90
[   39.076383]  [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0
[   39.077235]  [<ffffffff811700c8>] shmem_write_begin+0x38/0x40
[   39.078100]  [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290
[   39.079106]  [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0
[   39.080018]  [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0
[   39.080884]  [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0
[   39.081642]  [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0
[   39.082360]  [<ffffffff811c7028>] SyS_write+0x58/0xb0
[   39.083063]  [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b
[   39.083961] Mem-Info:
[   39.084331] Node 0 DMA per-cpu:
[   39.084831] CPU    0: hi:    0, btch:   1 usd:   0
[   39.085501] Node 0 DMA32 per-cpu:
[   39.085981] CPU    0: hi:  186, btch:  31 usd: 157
[   39.086746] active_anon:1277 inactive_anon:60574 isolated_anon:0
[   39.086746]  active_file:6 inactive_file:7 isolated_file:0
[   39.086746]  unevictable:169815 dirty:0 writeback:0 unstable:0
[   39.086746]  free:12224 slab_reclaimable:2884 slab_unreclaimable:2141
[   39.086746]  mapped:1959 shmem:60605 pagetables:209 bounce:0
[   39.086746]  free_cma:0
[   39.091510] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:132kB inactive_anon:2608kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2608kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[   39.097293] lowmem_reserve[]: 0 976 976 976
[   39.097972] Node 0 DMA32 free:44292kB min:44348kB low:55432kB high:66520kB active_anon:4976kB inactive_anon:239688kB active_file:24kB inactive_file:28kB unevictable:670988kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:0kB writeback:0kB mapped:7836kB shmem:239812kB slab_reclaimable:11532kB slab_unreclaimable:8552kB kernel_stack:1264kB pagetables:836kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:38815 all_unreclaimable? yes
[   39.104117] lowmem_reserve[]: 0 0 0 0
[   39.104731] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB
[   39.107227] Node 0 DMA32: 115*4kB (UEM) 39*8kB (UEM) 18*16kB (UEM) 5*32kB (E) 5*64kB (UEM) 2*128kB (UM) 2*256kB (U) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44292kB
[   39.109803] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   39.110964] 230105 total pagecache pages
[   39.111522] 0 pages in swap cache
[   39.111982] Swap cache stats: add 0, delete 0, find 0/0
[   39.112756] Free swap  = 0kB
[   39.113214] Total swap = 0kB
[   39.113657] 262044 pages RAM
[   39.114083] 0 pages HighMem/MovableOnly
[   39.114617] 7738 pages reserved
[   39.115064] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   39.116531] [   85]     0    85    10296      543      23        0             0 systemd-journal
[   39.117843] [  129]     0   129    76032     1278      29        0         -1000 multipathd
[   39.119343] [  396]    32   396     9977      295      22        0             0 rpcbind
[   39.120923] [  400]     0   400    11121      424      25        0             0 rpc.statd
[   39.122095] [  405]     0   405     5853      108      17        0             0 rpc.idmapd
[   39.123273] [  427]     0   427     8831      460      21        0         -1000 systemd-udevd
[   39.124821] [  587]     0   587     7809      254      18        0             0 plymouthd
[   39.126087] [  940]     0   940     3067      556      12        0             0 dracut-pre-pivo
[   39.127331] [  979]     0   979     1106      156       8        0             0 cpio
[   39.128452] [  980]     0   980     1143      153       7        0             0 gzip
[   39.129670] Out of memory: Kill process 940 (dracut-pre-pivo) score 2 or sacrifice child
[   39.130824] Killed process 979 (cpio) total-vm:4424kB, anon-rss:96kB, file-rss:528kB
dracut-pre-pivot[940]: Warning: Break before switch_root
[  OK  ] Started Show Plymouth Boot Screen.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Basic System.
dracut-initqueue[584]: RTNETLINK answers: File exists
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
100   458  100   458    0     0   1329      0 --:--:-- --:--:-- --:--:--  1327:--:-- --:--:--     0
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
100  536M  100  536M    0     0  32.3M      0  0:00:16  0:00:16 --:--:-- 59.7M:--:-- --:--:--     0
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0:--:-- --:--:--     0
dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found
dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/updates.img' failed!
dracut-initqueue[584]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[584]: Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0:--:-- --:--:--     0
dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found
dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/product.img' failed!
[  OK  ] Started dracut initqueue hook.
         Starting dracut pre-mount hook...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Started dracut pre-mount hook.
[  OK  ] Reached target Initrd Root File System.
         Starting Reload Configuration from the Real Root...
[  OK  ] Started Reload Configuration from the Real Root.
[  OK  ] Reached target Initrd File Systems.
         Starting dracut mount hook...
[  OK  ] Started dracut mount hook.
[  OK  ] Reached target Initrd Default Target.
dracut-pre-pivot[940]: Warning: Break before switch_root

Generating "/run/initramfs/rdsosreport.txt"


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


switch_root:/# 





# cat /proc/cmdline 
method=http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer/ console=ttyS0 rd.break rd.shell

Comment 3 Alexander Todorov 2015-03-19 10:23:18 UTC

Created attachment 1003777 [details]
rdsosreport.txt

Comment 4 Alexander Todorov 2015-03-19 10:23:37 UTC

Created attachment 1003778 [details]
journalctl.txt

Comment 5 Brian Lane 2015-03-19 19:07:35 UTC

What's it look like with the other variant?

What I think is happening is you're running out of memory. A http/PXE installation downloads the stage2 squashfs.img into memory and requires more than the normal amount needed when booting from the installer media.

What I don't understand is why it would work with one variant setting and not another, as long as the memory setting is the same it should just fail (or maybe be on the edge of failing, so you'd see it work sometimes and not others).

Comment 6 Alexander Todorov 2015-03-20 12:38:18 UTC

I think you may be right. Next logs are with settings os type/variant Linux/Unknown in virt manager. 

The system started VNC b/c I'm attached to the serial console. However I'm not able to connect to the VNC server and when switched to tty2 to collect some logs I've noticed the system was incredibly slow. At the end it showed:

# free -m
              total        used     free      shared  buff/cache   available
Mem:            993         104       62         237         826          16
Swap:             0           0        0

Comment 7 Alexander Todorov 2015-03-20 12:39:10 UTC

Created attachment 1004477 [details]
console.log from test with Linux/Unknown

Comment 8 Alexander Todorov 2015-03-20 12:40:52 UTC

After 5 minutes of waiting for scp to connect and present me with a password prompt I just give up. Let me know if this file is necessary from this scenario and I'll seek another way to grab it.

Comment 9 Alexander Todorov 2015-03-20 12:50:54 UTC

The next set of logs is from Atomic Host installation via ISO (previous was via HTTP) and OS type/Version in virt-manager left at Unknown/Unknown .

I've done VNC install and the system appeared responsive enough. Installation completed without errors.

# free -m
              total        used     free      shared  buff/cache   available
Mem:            993         215       64           7         712         605
Swap:           923           0      923

^^^ this is after installation has started and swap has already been created (but not in use as it seems).

Comment 10 Alexander Todorov 2015-03-20 12:51:25 UTC

Created attachment 1004478 [details]
console.log from ISO test with Unknown/Unknown

Comment 11 Alexander Todorov 2015-03-20 12:51:48 UTC

Created attachment 1004479 [details]
journalctl.txt from ISO test with Unknown/Unknown

Comment 12 Alexander Todorov 2015-03-20 12:55:49 UTC

(In reply to bcl from comment #5)
> What's it look like with the other variant?
> 
> What I think is happening is you're running out of memory. A http/PXE
> installation downloads the stage2 squashfs.img into memory and requires more
> than the normal amount needed when booting from the installer media.
> 

Indeed for Atomic  squashfs.img  is  537M, nearly twice as much as the regular RHEL variant. 

I guess this is related to how the distro is put together.

Comment 13 Colin Walters 2015-03-20 13:13:05 UTC

(In reply to Alexander Todorov from comment #12)

> Indeed for Atomic  squashfs.img  is  537M, nearly twice as much as the
> regular RHEL variant. 

Yeah, it's because the squashfs includes the ostree content.

> I guess this is related to how the distro is put together.

I have been working on fixing this upstream this week.  See:
https://github.com/rhinstaller/lorax/pull/8
https://fedorahosted.org/rel-eng/ticket/6119

Comment 15 Radek Vykydal 2015-06-16 12:38:30 UTC

Patches ported to rhel7-branch sent for review.

Comment 16 Radek Vykydal 2015-06-16 14:14:20 UTC

As Colin pointed out we should reconsider if we really want to remove ostree content from squashfs image as it would break PXE network installations using http, requiring argument similar to inst.repo= pointing to the content. We should check also if nfsiso PXE installation will break, the nfsiso case should be fixable without need for additional parameter though.

Comment 17 Colin Walters 2015-06-16 14:31:27 UTC

See https://fedorahosted.org/cloud/ticket/104

Comment 19 Radek Vykydal 2015-06-25 07:37:07 UTC

We've added the support in lorax to be ready for whatever approach we decide to follow. I'll clone this bz for rhel-server-atomic to track the issue further (on the level of iso compose, and anaconda support for ostree content outside installer image eventually)

Comment 23 errata-xmlrpc 2015-11-19 09:40:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2015-2289.html

Note You need to log in before you can comment on or make changes to this bug.