Bug 1202278
| Summary: | [atomic] OOM with 1024 MB RAM (squashfs includes ostree content) | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Alexander Todorov <atodorov> | ||||||||||||
| Component: | lorax | Assignee: | Radek Vykydal <rvykydal> | ||||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Release Test Team <release-test-team-automation> | ||||||||||||
| Severity: | high | Docs Contact: | |||||||||||||
| Priority: | high | ||||||||||||||
| Version: | 7.1 | CC: | atodorov, fdeutsch, jeder, mbanas, walters | ||||||||||||
| Target Milestone: | rc | ||||||||||||||
| Target Release: | --- | ||||||||||||||
| Hardware: | Unspecified | ||||||||||||||
| OS: | Unspecified | ||||||||||||||
| Whiteboard: | |||||||||||||||
| Fixed In Version: | lorax-19.6.56-1 | Doc Type: | Bug Fix | ||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||
| Clone Of: | |||||||||||||||
| : | 1235558 (view as bug list) | Environment: | |||||||||||||
| Last Closed: | 2015-11-19 09:40:34 UTC | Type: | Bug | ||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||
| Documentation: | --- | CRM: | |||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||
| Embargoed: | |||||||||||||||
| Bug Depends On: | |||||||||||||||
| Bug Blocks: | 1205794, 1235558 | ||||||||||||||
| Attachments: |
|
||||||||||||||
|
Description
Alexander Todorov
2015-03-16 09:45:35 UTC
What does /proc/cmdline look like? And the output from journalctl might be helpful. Here's the console log. # virsh console rhel7.0-17 Connected to domain rhel7.0-17 Escape character is ^] 100 536M 100 536M 0 0 32.3M 0 0:00:16 0:00:16 --:--:-- 59.7M:--:-- --:--:-- 0 dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0:--:-- --:--:-- 0 dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/updates.img' failed! dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0:--:-- --:--:-- 0 dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/product.img' failed! [ 28.279696] loop: module loaded [ OK ] Started dracut initqueue hook. Starting dracut pre-mount hook... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ OK ] Started dracut pre-mount hook. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. Starting dracut mount hook... [ 28.468323] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Started dracut mount hook. [ OK ] Reached target Initrd Default Target. [ 38.315862] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0 [ 38.316954] gzip cpuset=/ mems_allowed=0 [ 38.317516] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1 [ 38.318481] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 38.319284] ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36 [ 38.320396] ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18 [ 38.321504] ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803 [ 38.322698] Call Trace: [ 38.323069] [<ffffffff81603f36>] dump_stack+0x19/0x1b [ 38.323809] [<ffffffff815feeff>] dump_header+0x8e/0x214 [ 38.324587] [<ffffffff81117803>] ? delayacct_end+0x73/0xb0 [ 38.325392] [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0 [ 38.326236] [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30 [ 38.327147] [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0 [ 38.327928] [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90 [ 38.328833] [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140 [ 38.329710] [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0 [ 38.330603] [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120 [ 38.331482] [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90 [ 38.332775] [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0 [ 38.333718] [<ffffffff811700c8>] shmem_write_begin+0x38/0x40 [ 38.334572] [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290 [ 38.335569] [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0 [ 38.336502] [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0 [ 38.337384] [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0 [ 38.338179] [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0 [ 38.338899] [<ffffffff811c7028>] SyS_write+0x58/0xb0 [ 38.340039] [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b [ 38.340968] Mem-Info: [ 38.341314] Node 0 DMA per-cpu: [ 38.341783] CPU 0: hi: 0, btch: 1 usd: 0 [ 38.342799] Node 0 DMA32 per-cpu: [ 38.343359] CPU 0: hi: 186, btch: 31 usd: 68 [ 38.344084] active_anon:4440 inactive_anon:57419 isolated_anon:0 [ 38.344084] active_file:10 inactive_file:4 isolated_file:0 [ 38.344084] unevictable:169812 dirty:6 writeback:0 unstable:0 [ 38.344084] free:12228 slab_reclaimable:2870 slab_unreclaimable:2144 [ 38.344084] mapped:2011 shmem:57460 pagetables:269 bounce:0 [ 38.344084] free_cma:0 [ 38.348812] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:312kB inactive_anon:2432kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2440kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [ 38.354781] lowmem_reserve[]: 0 976 976 976 [ 38.355486] Node 0 DMA32 free:44308kB min:44348kB low:55432kB high:66520kB active_anon:17448kB inactive_anon:227244kB active_file:40kB inactive_file:16kB unevictable:670976kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:24kB writeback:0kB mapped:8044kB shmem:227400kB slab_reclaimable:11476kB slab_unreclaimable:8564kB kernel_stack:1280kB pagetables:1076kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:40917 all_unreclaimable? yes [ 38.361570] lowmem_reserve[]: 0 0 0 0 [ 38.362254] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB [ 38.365021] Node 0 DMA32: 75*4kB (UEM) 37*8kB (UEM) 16*16kB (UE) 6*32kB (EM) 4*64kB (UE) 2*128kB (UM) 3*256kB (UM) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44308kB [ 38.367961] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 38.369171] 226952 total pagecache pages [ 38.370077] 0 pages in swap cache [ 38.370603] Swap cache stats: add 0, delete 0, find 0/0 [ 38.371368] Free swap = 0kB [ 38.371777] Total swap = 0kB [ 38.372200] 262044 pages RAM [ 38.372609] 0 pages HighMem/MovableOnly [ 38.373161] 7738 pages reserved [ 38.373605] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [ 38.374711] [ 85] 0 85 10296 543 23 0 0 systemd-journal [ 38.375942] [ 129] 0 129 76032 1278 29 0 -1000 multipathd [ 38.377239] [ 396] 32 396 9977 295 22 0 0 rpcbind [ 38.378417] [ 400] 0 400 11121 424 25 0 0 rpc.statd [ 38.379570] [ 405] 0 405 5853 108 17 0 0 rpc.idmapd [ 38.380740] [ 427] 0 427 8831 460 21 0 -1000 systemd-udevd [ 38.381945] [ 587] 0 587 7809 254 18 0 0 plymouthd [ 38.383105] [ 686] 0 686 25934 3176 47 0 0 dhclient [ 38.384250] [ 940] 0 940 3067 556 12 0 0 dracut-pre-pivo [ 38.385490] [ 978] 0 978 2292 232 10 0 0 grep [ 38.386618] [ 979] 0 979 1106 156 8 0 0 cpio [ 38.387715] [ 980] 0 980 1143 153 7 0 0 gzip [ 38.388814] Out of memory: Kill process 686 (dhclient) score 12 or sacrifice child [ 38.389879] Killed process 686 (dhclient) total-vm:103736kB, anon-rss:12404kB, file-rss:300kB [ 38.981379] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0 [ 38.983461] gzip cpuset=/ mems_allowed=0 [ 38.984089] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1 [ 38.985153] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 38.985958] ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36 [ 38.987097] ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18 [ 38.988233] ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803 [ 38.989422] Call Trace: [ 38.989775] [<ffffffff81603f36>] dump_stack+0x19/0x1b [ 38.990512] [<ffffffff815feeff>] dump_header+0x8e/0x214 [ 38.991259] [<ffffffff81117803>] ? delayacct_end+0x73/0xb0 [ 38.992035] [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0 [ 38.993287] [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30 [ 38.994251] [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0 [ 38.995042] [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90 [ 38.995934] [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140 [ 38.996742] [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0 [ 38.997548] [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120 [ 38.998424] [<ffffffff812f4b41>] ? __percpu_counter_add+0x51/0x70 [ 38.999310] [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90 [ 39.000600] [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0 [ 39.001467] [<ffffffff811700c8>] shmem_write_begin+0x38/0x40 [ 39.002293] [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290 [ 39.003270] [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0 [ 39.004300] [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0 [ 39.005222] [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0 [ 39.005979] [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0 [ 39.006714] [<ffffffff811c7028>] SyS_write+0x58/0xb0 [ 39.007438] [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b [ 39.008290] Mem-Info: [ 39.008621] Node 0 DMA per-cpu: [ 39.009097] CPU 0: hi: 0, btch: 1 usd: 0 [ 39.009799] Node 0 DMA32 per-cpu: [ 39.010302] CPU 0: hi: 186, btch: 31 usd: 72 [ 39.010974] active_anon:1340 inactive_anon:60574 isolated_anon:0 [ 39.010974] active_file:6 inactive_file:7 isolated_file:0 [ 39.010974] unevictable:169815 dirty:0 writeback:0 unstable:0 [ 39.010974] free:12224 slab_reclaimable:2884 slab_unreclaimable:2141 [ 39.010974] mapped:1972 shmem:60605 pagetables:222 bounce:0 [ 39.010974] free_cma:0 [ 39.015829] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:132kB inactive_anon:2608kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2608kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [ 39.021533] lowmem_reserve[]: 0 976 976 976 [ 39.022297] Node 0 DMA32 free:44292kB min:44348kB low:55432kB high:66520kB active_anon:5228kB inactive_anon:239688kB active_file:24kB inactive_file:28kB unevictable:670988kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:0kB writeback:0kB mapped:7888kB shmem:239812kB slab_reclaimable:11532kB slab_unreclaimable:8552kB kernel_stack:1264kB pagetables:888kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:38789 all_unreclaimable? yes [ 39.028944] lowmem_reserve[]: 0 0 0 0 [ 39.029892] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB [ 39.032790] Node 0 DMA32: 115*4kB (UEM) 39*8kB (UEM) 18*16kB (UEM) 5*32kB (E) 5*64kB (UEM) 2*128kB (UM) 2*256kB (U) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44292kB [ 39.035500] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 39.036730] 230105 total pagecache pages [ 39.037309] 0 pages in swap cache [ 39.037823] Swap cache stats: add 0, delete 0, find 0/0 [ 39.038636] Free swap = 0kB [ 39.039070] Total swap = 0kB [ 39.039488] 262044 pages RAM [ 39.039896] 0 pages HighMem/MovableOnly [ 39.040451] 7738 pages reserved [ 39.040895] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [ 39.042144] [ 85] 0 85 10296 543 23 0 0 systemd-journal [ 39.043468] [ 129] 0 129 76032 1278 29 0 -1000 multipathd [ 39.044641] [ 396] 32 396 9977 295 22 0 0 rpcbind [ 39.045854] [ 400] 0 400 11121 424 25 0 0 rpc.statd [ 39.047037] [ 405] 0 405 5853 108 17 0 0 rpc.idmapd [ 39.048296] [ 427] 0 427 8831 460 21 0 -1000 systemd-udevd [ 39.049519] [ 587] 0 587 7809 254 18 0 0 plymouthd [ 39.050667] [ 940] 0 940 3067 556 12 0 0 dracut-pre-pivo [ 39.051874] [ 978] 0 978 2292 232 10 0 0 grep [ 39.052951] [ 979] 0 979 1106 156 8 0 0 cpio [ 39.054125] [ 980] 0 980 1143 153 7 0 0 gzip [ 39.055275] Out of memory: Kill process 940 (dracut-pre-pivo) score 2 or sacrifice child [ 39.056405] Killed process 978 (grep) total-vm:9168kB, anon-rss:220kB, file-rss:708kB [ 39.058272] gzip invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0 [ 39.059998] gzip cpuset=/ mems_allowed=0 [ 39.060603] CPU: 0 PID: 980 Comm: gzip Not tainted 3.10.0-229.el7.x86_64 #1 [ 39.061584] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 39.062416] ffff88003e870000 000000003df9817c ffff88001179f798 ffffffff81603f36 [ 39.063537] ffff88001179f828 ffffffff815feeff ffff880011795d00 ffff880011795d18 [ 39.064657] ffffffff00000206 ffffffff00000000 0000000000000000 ffffffff81117803 [ 39.065909] Call Trace: [ 39.066331] [<ffffffff81603f36>] dump_stack+0x19/0x1b [ 39.067092] [<ffffffff815feeff>] dump_header+0x8e/0x214 [ 39.067849] [<ffffffff81117803>] ? delayacct_end+0x73/0xb0 [ 39.068655] [<ffffffff8115a07e>] oom_kill_process+0x24e/0x3b0 [ 39.069469] [<ffffffff8107bd0e>] ? has_capability_noaudit+0x1e/0x30 [ 39.070352] [<ffffffff8115a8a6>] out_of_memory+0x4b6/0x4f0 [ 39.071124] [<ffffffff81160a65>] __alloc_pages_nodemask+0xa95/0xb90 [ 39.072089] [<ffffffff811a24aa>] alloc_pages_vma+0x9a/0x140 [ 39.072939] [<ffffffff8116d4de>] shmem_alloc_page+0x6e/0xc0 [ 39.073734] [<ffffffff8126ec0b>] ? cred_has_capability+0x6b/0x120 [ 39.074615] [<ffffffff812f4b41>] ? __percpu_counter_add+0x51/0x70 [ 39.075489] [<ffffffff812f4c55>] ? percpu_counter_compare+0x25/0x90 [ 39.076383] [<ffffffff8116fce4>] shmem_getpage_gfp+0x404/0x7b0 [ 39.077235] [<ffffffff811700c8>] shmem_write_begin+0x38/0x40 [ 39.078100] [<ffffffff8115648d>] generic_file_buffered_write+0x11d/0x290 [ 39.079106] [<ffffffff811585f5>] __generic_file_aio_write+0x1d5/0x3e0 [ 39.080018] [<ffffffff8115885d>] generic_file_aio_write+0x5d/0xc0 [ 39.080884] [<ffffffff811c5e3d>] do_sync_write+0x8d/0xd0 [ 39.081642] [<ffffffff811c65dd>] vfs_write+0xbd/0x1e0 [ 39.082360] [<ffffffff811c7028>] SyS_write+0x58/0xb0 [ 39.083063] [<ffffffff81613da9>] system_call_fastpath+0x16/0x1b [ 39.083961] Mem-Info: [ 39.084331] Node 0 DMA per-cpu: [ 39.084831] CPU 0: hi: 0, btch: 1 usd: 0 [ 39.085501] Node 0 DMA32 per-cpu: [ 39.085981] CPU 0: hi: 186, btch: 31 usd: 157 [ 39.086746] active_anon:1277 inactive_anon:60574 isolated_anon:0 [ 39.086746] active_file:6 inactive_file:7 isolated_file:0 [ 39.086746] unevictable:169815 dirty:0 writeback:0 unstable:0 [ 39.086746] free:12224 slab_reclaimable:2884 slab_unreclaimable:2141 [ 39.086746] mapped:1959 shmem:60605 pagetables:209 bounce:0 [ 39.086746] free_cma:0 [ 39.091510] Node 0 DMA free:4604kB min:704kB low:880kB high:1056kB active_anon:132kB inactive_anon:2608kB active_file:0kB inactive_file:0kB unevictable:8272kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:2608kB slab_reclaimable:4kB slab_unreclaimable:12kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [ 39.097293] lowmem_reserve[]: 0 976 976 976 [ 39.097972] Node 0 DMA32 free:44292kB min:44348kB low:55432kB high:66520kB active_anon:4976kB inactive_anon:239688kB active_file:24kB inactive_file:28kB unevictable:670988kB isolated(anon):0kB isolated(file):0kB present:1032184kB managed:1001316kB mlocked:5472kB dirty:0kB writeback:0kB mapped:7836kB shmem:239812kB slab_reclaimable:11532kB slab_unreclaimable:8552kB kernel_stack:1264kB pagetables:836kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:38815 all_unreclaimable? yes [ 39.104117] lowmem_reserve[]: 0 0 0 0 [ 39.104731] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (M) 1*128kB (U) 1*256kB (M) 2*512kB (M) 1*1024kB (U) 1*2048kB (R) 0*4096kB = 4604kB [ 39.107227] Node 0 DMA32: 115*4kB (UEM) 39*8kB (UEM) 18*16kB (UEM) 5*32kB (E) 5*64kB (UEM) 2*128kB (UM) 2*256kB (U) 0*512kB 1*1024kB (M) 0*2048kB 10*4096kB (MR) = 44292kB [ 39.109803] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 39.110964] 230105 total pagecache pages [ 39.111522] 0 pages in swap cache [ 39.111982] Swap cache stats: add 0, delete 0, find 0/0 [ 39.112756] Free swap = 0kB [ 39.113214] Total swap = 0kB [ 39.113657] 262044 pages RAM [ 39.114083] 0 pages HighMem/MovableOnly [ 39.114617] 7738 pages reserved [ 39.115064] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [ 39.116531] [ 85] 0 85 10296 543 23 0 0 systemd-journal [ 39.117843] [ 129] 0 129 76032 1278 29 0 -1000 multipathd [ 39.119343] [ 396] 32 396 9977 295 22 0 0 rpcbind [ 39.120923] [ 400] 0 400 11121 424 25 0 0 rpc.statd [ 39.122095] [ 405] 0 405 5853 108 17 0 0 rpc.idmapd [ 39.123273] [ 427] 0 427 8831 460 21 0 -1000 systemd-udevd [ 39.124821] [ 587] 0 587 7809 254 18 0 0 plymouthd [ 39.126087] [ 940] 0 940 3067 556 12 0 0 dracut-pre-pivo [ 39.127331] [ 979] 0 979 1106 156 8 0 0 cpio [ 39.128452] [ 980] 0 980 1143 153 7 0 0 gzip [ 39.129670] Out of memory: Kill process 940 (dracut-pre-pivo) score 2 or sacrifice child [ 39.130824] Killed process 979 (cpio) total-vm:4424kB, anon-rss:96kB, file-rss:528kB dracut-pre-pivot[940]: Warning: Break before switch_root [ OK ] Started Show Plymouth Boot Screen. [ OK ] Reached target Paths. [ OK ] Reached target Basic System. dracut-initqueue[584]: RTNETLINK answers: File exists dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 100 458 100 458 0 0 1329 0 --:--:-- --:--:-- --:--:-- 1327:--:-- --:--:-- 0 dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 100 536M 100 536M 0 0 32.3M 0 0:00:16 0:00:16 --:--:-- 59.7M:--:-- --:--:-- 0 dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0:--:-- --:--:-- 0 dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/updates.img' failed! dracut-initqueue[584]: % Total % Received % Xferd Average Speed Time Time Time Current dracut-initqueue[584]: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0:--:-- --:--:-- 0 dracut-initqueue[584]: curl: (22) The requested URL returned error: 404 Not Found dracut-initqueue[584]: Warning: Downloading 'http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer//images/product.img' failed! [ OK ] Started dracut initqueue hook. Starting dracut pre-mount hook... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ OK ] Started dracut pre-mount hook. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. Starting dracut mount hook... [ OK ] Started dracut mount hook. [ OK ] Reached target Initrd Default Target. dracut-pre-pivot[940]: Warning: Break before switch_root Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report. switch_root:/# # cat /proc/cmdline method=http://download.eng.bos.redhat.com/rel-eng/Atomic/7/trees/GA.brew/images/20150217.0/installer/ console=ttyS0 rd.break rd.shell Created attachment 1003777 [details]
rdsosreport.txt
Created attachment 1003778 [details]
journalctl.txt
What's it look like with the other variant? What I think is happening is you're running out of memory. A http/PXE installation downloads the stage2 squashfs.img into memory and requires more than the normal amount needed when booting from the installer media. What I don't understand is why it would work with one variant setting and not another, as long as the memory setting is the same it should just fail (or maybe be on the edge of failing, so you'd see it work sometimes and not others). I think you may be right. Next logs are with settings os type/variant Linux/Unknown in virt manager.
The system started VNC b/c I'm attached to the serial console. However I'm not able to connect to the VNC server and when switched to tty2 to collect some logs I've noticed the system was incredibly slow. At the end it showed:
# free -m
total used free shared buff/cache available
Mem: 993 104 62 237 826 16
Swap: 0 0 0
Created attachment 1004477 [details]
console.log from test with Linux/Unknown
After 5 minutes of waiting for scp to connect and present me with a password prompt I just give up. Let me know if this file is necessary from this scenario and I'll seek another way to grab it. The next set of logs is from Atomic Host installation via ISO (previous was via HTTP) and OS type/Version in virt-manager left at Unknown/Unknown .
I've done VNC install and the system appeared responsive enough. Installation completed without errors.
# free -m
total used free shared buff/cache available
Mem: 993 215 64 7 712 605
Swap: 923 0 923
^^^ this is after installation has started and swap has already been created (but not in use as it seems).
Created attachment 1004478 [details]
console.log from ISO test with Unknown/Unknown
Created attachment 1004479 [details]
journalctl.txt from ISO test with Unknown/Unknown
(In reply to bcl from comment #5) > What's it look like with the other variant? > > What I think is happening is you're running out of memory. A http/PXE > installation downloads the stage2 squashfs.img into memory and requires more > than the normal amount needed when booting from the installer media. > Indeed for Atomic squashfs.img is 537M, nearly twice as much as the regular RHEL variant. I guess this is related to how the distro is put together. (In reply to Alexander Todorov from comment #12) > Indeed for Atomic squashfs.img is 537M, nearly twice as much as the > regular RHEL variant. Yeah, it's because the squashfs includes the ostree content. > I guess this is related to how the distro is put together. I have been working on fixing this upstream this week. See: https://github.com/rhinstaller/lorax/pull/8 https://fedorahosted.org/rel-eng/ticket/6119 Patches ported to rhel7-branch sent for review. As Colin pointed out we should reconsider if we really want to remove ostree content from squashfs image as it would break PXE network installations using http, requiring argument similar to inst.repo= pointing to the content. We should check also if nfsiso PXE installation will break, the nfsiso case should be fixable without need for additional parameter though. We've added the support in lorax to be ready for whatever approach we decide to follow. I'll clone this bz for rhel-server-atomic to track the issue further (on the level of iso compose, and anaconda support for ostree content outside installer image eventually) Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-2289.html |