Bug 1594447

Summary: systemd-239-1.fc29 doesn't boot correctly on aarch64
Product: [Fedora] Fedora Reporter: Kevin Fenzi <kevin>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: airlied, bskeggs, ewk, hdegoede, ichavero, itamar, jarodwilson, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, lnykryn, mchehab, mjg59, msekleta, pbrobinson, pwhalen, ssahani, s, steved, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.18.0-0.rc2.git1.1.fc29.aarch64 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-26 16:21:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 245418    

Description Kevin Fenzi 2018-06-23 01:49:08 UTC
Trying to compose the Base-Cloud image on aarch64 fails: 

https://koji.fedoraproject.org/koji/taskinfo?taskID=27794913

Firing off a test compose using the same repos/parameters and watching the console I see: 

...
[  OK  ] Listening on Journal Socket.
         Starting Setup Virtual Console...
[  OK  ] Listening on Journal Socket (/dev/log).
         Starting Journal Service...
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Reached target Sockets.
         Starting Create list of required st…ce nodes for the current kernel...
[   10.771321] Internal error: SP/PC alignment exception: 8a000000 [#1] SMP
[   10.772489] Modules linked in:
[   10.773012] CPU: 0 PID: 198 Comm: (journald) Tainted: G        W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   10.774644] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   10.775791] pstate: 60400005 (nZCv daif +PAN -UAO)
[   10.776594] pc : 0xffff4cb953c693bb
[   10.777185] lr : seccomp_run_filters+0x4c/0xb0
[   10.778099] sp : ffff00000d83bd10
[   10.778858] x29: ffff00000d83bd10 x28: ffff8f16e986cc80 
[   10.778863] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   10.778867] x25: 0000000000000125 x24: 0000000000000015 
[   10.780943] x23: 0000000000000000 x22: ffff00000d83bdc8 
[   10.780947] x21: ffff00000d83bd40 x20: 000000007fff0000 
[   10.780951] x19: ffff8f16ebd55b80 x18: 000000000000007f 
[   10.780954] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   10.780958] x15: 0000000000000001 x14: 0000000000000000 
[   10.780962] x13: 00000029002b0015 x12: 0000002a002c0015 
[   10.780972] x11: 0000002400260015 x10: 0000002500270015 
[   10.788901] x9 : 0000002600280015 x8 : 0000000000000115 
[   10.788904] x7 : 0000000000000038 x6 : 0000aaaae6319a70 
[   10.788908] x5 : 0000000000000007 x4 : 0000000000000000 
[   10.788912] x3 : ffff8f16e986cc80 x2 : ffff4cb953c693bb 
[   10.788915] x1 : ffff0000087b5038 x0 : ffff00000d83bd40 
[   10.788936] Process (journald) (pid: 198, stack limit = 0x00000000aea0c8f5)
[   10.788939] Call trace:
[   10.788944]  0xffff4cb953c693bb
[   10.788948]  __seccomp_filter+0x3c/0x420
[   10.788952]  __secure_computing+0x4c/0xc0
[   10.788958]  syscall_trace_enter+0x100/0x2f0
[   10.788962]  __sys_trace+0x18/0x48
[   10.788971] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   10.788976] ---[ end trace 259714942391020a ]---
         Starting dracut cmdline hook...
         Starting Apply Kernel Variables...
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Started Create list of required sta…vice nodes for the current kernel.
[  OK  ] Started Setup Virtual Console.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Stopped Journal Service.
         Starting Journal Service...
[   10.866261] Internal error: SP/PC alignment exception: 8a000000 [#2] SMP
[   10.867431] Modules linked in:
[   10.868114] CPU: 0 PID: 211 Comm: (journald) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   10.870223] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   10.871758] pstate: 60400005 (nZCv daif +PAN -UAO)
[   10.872823] pc : 0xffff4cb953c6f247
[   10.872823] pc : 0xffff4cb953c6f247
[   10.873609] lr : seccomp_run_filters+0x4c/0xb0
[   10.874592] sp : ffff00000d893d10
[   10.875331] x29: ffff00000d893d10 x28: ffff8f16e95acc80 
[   10.876512] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   10.877695] x25: 0000000000000125 x24: 0000000000000015 
[   10.878895] x23: 0000000000000000 x22: ffff00000d893dc8 
[   10.880078] x21: ffff00000d893d40 x20: 000000007fff0000 
[   10.881256] x19: ffff8f16e981ad00 x18: 000000000000007f 
[   10.882436] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   10.883619] x15: 0000000000000001 x14: 0000000000000000 
[   10.884798] x13: 00000029002b0015 x12: 0000002a002c0015 
[   10.885977] x11: 0000002400260015 x10: 0000002500270015 
[   10.887156] x9 : 0000002600280015 x8 : 0000000000000115 
[   10.888352] x7 : 0000000000000038 x6 : 0000aaaae6317080 
[   10.889533] x5 : 0000000000000007 x4 : 0000000000000000 
[   10.890714] x3 : ffff8f16e95acc80 x2 : ffff4cb953c6f247 
[   10.891891] x1 : ffff00000d23d038 x0 : ffff00000d893d40 
[   10.893076] Process (journald) (pid: 211, stack limit = 0x00000000fea1050f)
[   10.894615] Call trace:
[   10.895163]  0xffff4cb953c6f247
[   10.895864]  __seccomp_filter+0x3c/0x420
[   10.896737]  __secure_computing+0x4c/0xc0
[   10.897629]  syscall_trace_enter+0x100/0x2f0
[   10.898595]  __sys_trace+0x18/0x48
[   10.899363] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   10.900719] ---[ end trace 259714942391020b ]---
         Starting Create Static Device Nodes in /dev...
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Stopped Journal Service.
         Starting Journal Service...
[   10.948989] Internal error: SP/PC alignment exception: 8a000000 [#3] SMP
[   10.950094] Modules linked in:
[   10.950592] CPU: 1 PID: 219 Comm: (journald) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   10.952143] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   10.953228] pstate: 60400005 (nZCv daif +PAN -UAO)
[   10.953989] pc : 0xffff4cb953c75993
[   10.954546] lr : seccomp_run_filters+0x4c/0xb0
[   10.955254] sp : ffff00000d8ebd10
[   10.955780] x29: ffff00000d8ebd10 x28: ffff8f16e9888000 
[   10.956623] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   10.957474] x25: 0000000000000125 x24: 0000000000000015 
[   10.958607] x23: 0000000000000000 x22: ffff00000d8ebdc8 
[   10.959692] x21: ffff00000d8ebd40 x20: 000000007fff0000 
[   10.960776] x19: ffff8f16e97a4180 x18: 000000000000007f 
[   10.961861] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   10.962946] x15: 0000000000000001 x14: 0000000000000000 
[   10.964030] x13: 00000029002b0015 x12: 0000002a002c0015 
[   10.965112] x11: 0000002400260015 x10: 0000002500270015 
[   10.966196] x9 : 0000002600280015 x8 : 0000000000000115 
[   10.967277] x7 : 0000000000000038 x6 : 0000aaaae6317080 
[   10.968367] x5 : 0000000000000007 x4 : 0000000000000000 
[   10.969443] x3 : ffff8f16e9888000 x2 : ffff4cb953c75993 
[   10.970526] x1 : ffff00000d275038 x0 : ffff00000d8ebd40 
[   10.971609] Process (journald) (pid: 219, stack limit = 0x0000000047c39a82)
[   10.973019] Call trace:
[   10.973520]  0xffff4cb953c75993
[   10.974161]  __seccomp_filter+0x3c/0x420
[   10.974956]  __secure_computing+0x4c/0xc0
[   10.975774]  syscall_trace_enter+0x100/0x2f0
[   10.976643]  __sys_trace+0x18/0x48
[   10.977345] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   10.978599] ---[ end trace 259714942391020c ]---
[  OK  ] Started Create Static Device Nodes in /dev.
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Stopped Journal Service.
         Starting Journal Service...
[   11.009089] Internal error: SP/PC alignment exception: 8a000000 [#4] SMP
[   11.010262] Modules linked in:
[   11.010790] CPU: 0 PID: 240 Comm: (journald) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   11.012426] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   11.013577] pstate: 60400005 (nZCv daif +PAN -UAO)
[   11.014379] pc : 0xffff4cb953c7b11f
[   11.014973] lr : seccomp_run_filters+0x4c/0xb0
[   11.015721] sp : ffff00000d8ebd10
[   11.016278] x29: ffff00000d8ebd10 x28: ffff8f16e988a640 
[   11.017172] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   11.018321] x25: 0000000000000125 x24: 0000000000000015 
[   11.019660] x23: 0000000000000000 x22: ffff00000d8ebdc8 
[   11.020896] x21: ffff00000d8ebd40 x20: 000000007fff0000 
[   11.022052] x19: ffff8f16e9814800 x18: 000000000000007f 
[   11.023196] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   11.024339] x15: 0000000000000001 x14: 0000000000000000 
[   11.025484] x13: 00000029002b0015 x12: 0000002a002c0015 
[   11.026630] x11: 0000002400260015 x10: 0000002500270015 
[   11.027783] x9 : 0000002600280015 x8 : 0000000000000115 
[   11.028925] x7 : 0000000000000038 x6 : 0000aaaae631b950 
[   11.030062] x5 : 0000000000000007 x4 : 0000000000000000 
[   11.031201] x3 : ffff8f16e988a640 x2 : ffff4cb953c7b11f 
[   11.032346] x1 : ffff00000d705038 x0 : ffff00000d8ebd40 
[   11.033492] Process (journald) (pid: 240, stack limit = 0x0000000047c39a82)
[   11.035071] Call trace:
[   11.035614]  0xffff4cb953c7b11f
[   11.036305]  __seccomp_filter+0x3c/0x420
[   11.037152]  __secure_computing+0x4c/0xc0
[   11.038052]  syscall_trace_enter+0x100/0x2f0
[   11.038983]  __sys_trace+0x18/0x48
[   11.039730] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   11.041044] ---[ end trace 259714942391020d ]---
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Stopped Journal Service.
         Starting Journal Service...
[   11.068355] Internal error: SP/PC alignment exception: 8a000000 [#5] SMP
[   11.069544] Modules linked in:
[   11.070069] CPU: 0 PID: 252 Comm: (journald) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   11.071708] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   11.072861] pstate: 60400005 (nZCv daif +PAN -UAO)
[   11.073671] pc : 0xffff4cb953c818a3
[   11.074264] lr : seccomp_run_filters+0x4c/0xb0
[   11.075011] sp : ffff00000d903d10
[   11.075576] x29: ffff00000d903d10 x28: ffff8f16e9888000 
[   11.076477] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   11.077377] x25: 0000000000000125 x24: 0000000000000015 
[   11.078515] x23: 0000000000000000 x22: ffff00000d903dc8 
[   11.079653] x21: ffff00000d903d40 x20: 000000007fff0000 
[   11.080840] x19: ffff8f16e96c5500 x18: 000000000000007f 
[   11.082025] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   11.083213] x15: 0000000000000001 x14: 0000000000000000 
[   11.084400] x13: 00000029002b0015 x12: 0000002a002c0015 
[   11.085590] x11: 0000002400260015 x10: 0000002500270015 
[   11.086779] x9 : 0000002600280015 x8 : 0000000000000115 
[   11.087977] x7 : 0000000000000038 x6 : 0000aaaae631b860 
[   11.089254] x5 : 0000000000000007 x4 : 0000000000000000 
[   11.090541] x3 : ffff8f16e9888000 x2 : ffff4cb953c818a3 
[   11.091787] x1 : ffff00000d73d038 x0 : ffff00000d903d40 
[   11.093031] Process (journald) (pid: 252, stack limit = 0x000000007e58bde5)
[   11.094591] Call trace:
[   11.095169]  0xffff4cb953c818a3
[   11.095888]  __seccomp_filter+0x3c/0x420
[   11.096781]  __secure_computing+0x4c/0xc0
[   11.097688]  syscall_trace_enter+0x100/0x2f0
[   11.098674]  __sys_trace+0x18/0x48
[   11.099444] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   11.100803] ---[ end trace 259714942391020e ]---
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Stopped Journal Service.
[FAILED] Failed to start Journal Service.
See 'systemctl status systemd-journald.service' for details.
[  OK  ] Started dracut cmdline hook.
         Starting dracut pre-udev hook...[   14.446951] systemd[438]: dracut-pre-udev.service: Failed to connect stdout
 to the journal socket, ignoring: Connection refused

[   14.449594] systemd[438]: dracut-pre-udev.service: Failed to connect stderr to the journal socket, ignoring: Connect
ion refused
[   14.605374] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[   15.007486] raid6: int64x1  gen()  1436 MB/s
[   15.177489] raid6: int64x1  xor()  1132 MB/s
[   15.347474] raid6: int64x2  gen()  2052 MB/s
[   15.517490] raid6: int64x2  xor()  1407 MB/s
[   15.687487] raid6: int64x4  gen()  2787 MB/s
[   15.857488] raid6: int64x4  xor()  1597 MB/s
[   16.027478] raid6: int64x8  gen()  2866 MB/s
[   16.197485] raid6: int64x8  xor()  1650 MB/s
[   16.367489] raid6: neonx1   gen()  2998 MB/s
[   16.537482] raid6: neonx1   xor()  2557 MB/s
[   16.707480] raid6: neonx2   gen()  3064 MB/s
[   16.877479] raid6: neonx2   xor()  2571 MB/s
[   17.047479] raid6: neonx4   gen()  3058 MB/s
[   17.217473] raid6: neonx4   xor()  2612 MB/s
[   17.387485] raid6: neonx8   gen()  2922 MB/s
[   17.557471] raid6: neonx8   xor()  2492 MB/s
[   17.558175] raid6: using algorithm neonx2 gen() 3064 MB/s
[   17.559053] raid6: .... xor() 2571 MB/s, rmw enabled
[   17.559841] raid6: using neon recovery algorithm
[   17.563614] xor: measuring software checksum speed
[   17.657476]    8regs     :  8497.600 MB/sec
[   17.757475]    8regs_prefetch:  7372.400 MB/sec
[   17.857475]    32regs    :  8612.400 MB/sec
[   17.957475]    32regs_prefetch:  7723.600 MB/sec
[   17.958302] xor: using function: 32regs (8612.400 MB/sec)
[   17.961896] async_tx: api initialized (async)
[   18.316532] device-mapper: multipath round-robin: version 1.2.0 loaded
[   18.590564] RPC: Registered named UNIX socket transport module.
[   18.591670] RPC: Registered udp transport module.
[   18.592469] RPC: Registered tcp transport module.
[   18.593255] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   18.815448] systemd[1]: Started dracut pre-udev hook.
[  OK  ] Started dracut pre-udev hook.
[   18.824968] systemd[1]: Starting udev Kernel Device Manager...
         Starting udev Kernel Device Manager...
[   18.829002] systemd[666]: systemd-udevd.service: Failed to connect stdout to the journal socket, ignoring: Connectio
n refused
[   18.834906] Internal error: SP/PC alignment exception: 8a000000 [#6] SMP
[   18.836037] Modules linked in: sunrpc lrw dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_rec
ov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 squashfs zstd_decompress xxhash cramfs
[   18.840010] CPU: 1 PID: 666 Comm: (md-udevd) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   18.842041] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   18.843461] pstate: 60400005 (nZCv daif +PAN -UAO)
[   18.844458] pc : 0xffff4cb953c93ceb
[   18.845190] lr : seccomp_run_filters+0x4c/0xb0
[   18.846111] sp : ffff000008b73d10
[   18.846801] x29: ffff000008b73d10 x28: ffff8f16e986cc80 
[   18.847905] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   18.849016] x25: 0000000000000125 x24: 0000000000000015 
[   18.850119] x23: 0000000000000000 x22: ffff000008b73dc8 
[   18.851222] x21: ffff000008b73d40 x20: 000000007fff0000 
[   18.852326] x19: ffff8f16e99bae80 x18: 000000000000007f 
[   18.853432] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   18.854539] x15: 0000000000000001 x14: 0000000000000000 
[   18.855645] x13: 0000002900290015 x12: 0000002a002a0015 
[   18.856752] x11: 0000002400240015 x10: 0000002500250015 
[   18.857856] x9 : 0000002600260015 x8 : 0000000000000115 
[   18.858966] x7 : 0000000000000038 x6 : 0000aaaae631f340 
[   18.860071] x5 : 0000000000000007 x4 : 0000000000000000 
[   18.861176] x3 : ffff8f16e986cc80 x2 : ffff4cb953c93ceb 
[   18.862280] x1 : ffff00000878d038 x0 : ffff000008b73d40 
[   18.863389] Process (md-udevd) (pid: 666, stack limit = 0x0000000099a9da5e)
[   18.864828] Call trace:
[   18.865343]  0xffff4cb953c93ceb
[   18.865999]  __seccomp_filter+0x3c/0x420
[   18.866815]  __secure_computing+0x4c/0xc0
[   18.867652]  syscall_trace_enter+0x100/0x2f0
[   18.868591]  __sys_trace+0x18/0x48
[   18.869305] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   18.870565] ---[ end trace 259714942391020f ]---
[   18.908175] systemd[1]: systemd-udevd.service: Main process exited, code=killed, status=11/SEGV
[   18.910306] systemd[1]: systemd-udevd.service: Failed with result 'signal'.
[   18.915198] systemd[1]: Failed to start udev Kernel Device Manager.
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
[   18.919813] systemd[1]: systemd-udevd.service: Service has no hold-off time, scheduling restart.
[   18.921453] systemd[1]: systemd-udevd.service: Scheduled restart job, restart counter is at 1.
[   18.923003] systemd[1]: Stopped udev Kernel Device Manager.
[  OK  ] Stopped udev Kernel Device Manager.
[   18.931649] systemd[1]: Starting udev Kernel Device Manager...
[   18.932464] systemd[667]: systemd-udevd.service: Failed to connect stdout to the journal socket, ignoring: Connectio
n refused
         Starting udev Kernel Device Manager...
[   18.938730] Internal error: SP/PC alignment exception: 8a000000 [#7] SMP
[   18.939856] Modules linked in: sunrpc lrw dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_rec
ov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 squashfs zstd_decompress xxhash cramfs
[   18.943232] CPU: 1 PID: 667 Comm: (md-udevd) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   18.944818] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   18.945920] pstate: 60400005 (nZCv daif +PAN -UAO)
[   18.946703] pc : 0xffff4cb953cd559f
[   18.947271] lr : seccomp_run_filters+0x4c/0xb0
[   18.948161] sp : ffff000008c83d10
[   18.948870] x29: ffff000008c83d10 x28: ffff8f16ebcc4c80 
[   18.949993] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   18.951105] x25: 0000000000000125 x24: 0000000000000015 
[   18.952204] x23: 0000000000000000 x22: ffff000008c83dc8 
[   18.953314] x21: ffff000008c83d40 x20: 000000007fff0000 
[   18.954415] x19: ffff8f16e9b27580 x18: 000000000000007f 
[   18.955529] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   18.956637] x15: 0000000000000001 x14: 0000000000000000 
[   18.957746] x13: 0000002900290015 x12: 0000002a002a0015 
[   18.958853] x11: 0000002400240015 x10: 0000002500250015 
[   18.959971] x9 : 0000002600260015 x8 : 0000000000000115 
[   18.961078] x7 : 0000000000000038 x6 : 0000aaaae631f340 
[   18.962178] x5 : 0000000000000007 x4 : 0000000000000000 
[   18.963294] x3 : ffff8f16ebcc4c80 x2 : ffff4cb953cd559f 
[   18.964407] x1 : ffff0000087c5038 x0 : ffff000008c83d40 
[   18.965515] Process (md-udevd) (pid: 667, stack limit = 0x00000000e5261d39)
[   18.966969] Call trace:
[   18.967490]  0xffff4cb953cd559f
[   18.968150]  __seccomp_filter+0x3c/0x420
[   18.968983]  __secure_computing+0x4c/0xc0
[   18.969827]  syscall_trace_enter+0x100/0x2f0
[   18.970723]  __sys_trace+0x18/0x48
[   18.971450] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   18.972719] ---[ end trace 2597149423910210 ]---
[   19.008205] systemd[1]: systemd-udevd.service: Main process exited, code=killed, status=11/SEGV
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
[  OK  ] Stopped udev Kernel Device Manager.
         Starting udev Kernel Device Manager...[   19.025365] systemd[670]: systemd-udevd.service: Failed to connect st
dout to the journal socket, ignoring: Connection refused
[   19.031160] Internal error: SP/PC alignment exception: 8a000000 [#8] SMP
[   19.032362] Modules linked in: sunrpc lrw dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_rec
ov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 squashfs zstd_decompress xxhash cramfs
[   19.035987] CPU: 0 PID: 670 Comm: (md-udevd) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   19.037784] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   19.039405] pstate: 60400005 (nZCv daif +PAN -UAO)
[   19.040531] pc : 0xffff4cb953cef04b
[   19.041354] lr : seccomp_run_filters+0x4c/0xb0
[   19.042395] sp : ffff000008b73d10
[   19.043170] x29: ffff000008b73d10 x28: ffff8f16e986cc80 
[   19.044388] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   19.045618] x25: 0000000000000125 x24: 0000000000000015 
[   19.046848] x23: 0000000000000000 x22: ffff000008b73dc8 
[   19.048081] x21: ffff000008b73d40 x20: 000000007fff0000 
[   19.049348] x19: ffff8f16e9b6ca00 x18: 000000000000007f 
[   19.050584] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   19.051810] x15: 0000000000000001 x14: 0000000000000000 
[   19.053037] x13: 0000002900290015 x12: 0000002a002a0015 
[   19.054259] x11: 0000002400240015 x10: 0000002500250015 
[   19.055487] x9 : 0000002600260015 x8 : 0000000000000115 
[   19.056711] x7 : 0000000000000038 x6 : 0000aaaae631f340 
[   19.057941] x5 : 0000000000000007 x4 : 0000000000000000 
[   19.059190] x3 : ffff8f16e986cc80 x2 : ffff4cb953cef04b 
[   19.060424] x1 : ffff000008b05038 x0 : ffff000008b73d40 
[   19.061658] Process (md-udevd) (pid: 670, stack limit = 0x0000000099a9da5e)
[   19.063253] Call trace:
[   19.063829]  0xffff4cb953cef04b
[   19.064564]  __seccomp_filter+0x3c/0x420
[   19.065482]  __secure_computing+0x4c/0xc0
[   19.066418]  syscall_trace_enter+0x100/0x2f0
[   19.067408]  __sys_trace+0x18/0x48
[   19.068208] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   19.069638] ---[ end trace 2597149423910211 ]---
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
[  OK  ] Stopped udev Kernel Device Manager.
         Starting udev Kernel Device Manager...[   19.123871] systemd[671]: systemd-udevd.service: Failed to connect st
dout to the journal socket, ignoring: Connection refused

[   19.129902] Internal error: SP/PC alignment exception: 8a000000 [#9] SMP
[   19.131110] Modules linked in: sunrpc lrw dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_rec
ov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 squashfs zstd_decompress xxhash cramfs
[   19.134742] CPU: 0 PID: 671 Comm: (md-udevd) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   19.136455] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   19.137734] pstate: 60400005 (nZCv daif +PAN -UAO)
[   19.138828] pc : 0xffff4cb953d18ad3
[   19.139542] lr : seccomp_run_filters+0x4c/0xb0
[   19.140443] sp : ffff000008c6bd10
[   19.141115] x29: ffff000008c6bd10 x28: ffff8f16ebcc4c80 
[   19.142190] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   19.143262] x25: 0000000000000125 x24: 0000000000000015 
[   19.144333] x23: 0000000000000000 x22: ffff000008c6bdc8 
[   19.145405] x21: ffff000008c6bd40 x20: 000000007fff0000 
[   19.146478] x19: ffff8f16e9c4b680 x18: 000000000000007f 
[   19.147550] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   19.148633] x15: 0000000000000001 x14: 0000000000000000 
[   19.149706] x13: 0000002900290015 x12: 0000002a002a0015 
[   19.150780] x11: 0000002400240015 x10: 0000002500250015 
[   19.151854] x9 : 0000002600260015 x8 : 0000000000000115 
[   19.152924] x7 : 0000000000000038 x6 : 0000aaaae631f340 
[   19.153999] x5 : 0000000000000007 x4 : 0000000000000000 
[   19.155075] x3 : ffff8f16ebcc4c80 x2 : ffff4cb953d18ad3 
[   19.156156] x1 : ffff000008b6d038 x0 : ffff000008c6bd40 
[   19.157240] Process (md-udevd) (pid: 671, stack limit = 0x00000000b7886861)
[   19.158648] Call trace:
[   19.159148]  0xffff4cb953d18ad3
[   19.159790]  __seccomp_filter+0x3c/0x420
[   19.160584]  __secure_computing+0x4c/0xc0
[   19.161397]  syscall_trace_enter+0x100/0x2f0
[   19.162262]  __sys_trace+0x18/0x48
[   19.162958] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   19.164191] ---[ end trace 2597149423910212 ]---
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
[  OK  ] Stopped udev Kernel Device Manager.
         Starting udev Kernel Device Manager...[   19.214502] systemd[672]: systemd-udevd.service: Failed to connect st
dout to the journal socket, ignoring: Connection refused

[   19.220403] Internal error: SP/PC alignment exception: 8a000000 [#10] SMP
[   19.221508] Modules linked in: sunrpc lrw dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_rec
ov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 squashfs zstd_decompress xxhash cramfs
[   19.224794] CPU: 0 PID: 672 Comm: (md-udevd) Tainted: G      D W         4.18.0-0.rc1.git3.1.fc29.aarch64 #1
[   19.226326] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[   19.227407] pstate: 60400005 (nZCv daif +PAN -UAO)
[   19.228381] pc : 0xffff4cb953d29853
[   19.229119] lr : seccomp_run_filters+0x4c/0xb0
[   19.230026] sp : ffff000008cdbd10
[   19.230701] x29: ffff000008cdbd10 x28: ffff8f16e986cc80 
[   19.231786] x27: ffff4cba3a1d2000 x26: 0000000000000115 
[   19.232869] x25: 0000000000000125 x24: 0000000000000015 
[   19.233948] x23: 0000000000000000 x22: ffff000008cdbdc8 
[   19.235026] x21: ffff000008cdbd40 x20: 000000007fff0000 
[   19.236109] x19: ffff8f16e9b6c900 x18: 000000000000007f 
[   19.237183] x17: 0000ffff99967cb0 x16: 0000000000000800 
[   19.238267] x15: 0000000000000001 x14: 0000000000000000 
[   19.239354] x13: 0000002900290015 x12: 0000002a002a0015 
[   19.240436] x11: 0000002400240015 x10: 0000002500250015 
[   19.241515] x9 : 0000002600260015 x8 : 0000000000000115 
[   19.242598] x7 : 0000000000000038 x6 : 0000aaaae631f340 
[   19.243708] x5 : 0000000000000007 x4 : 0000000000000000 
[   19.244810] x3 : ffff8f16e986cc80 x2 : ffff4cb953d29853 
[   19.245920] x1 : ffff000008c7d038 x0 : ffff000008cdbd40 
[   19.247025] Process (md-udevd) (pid: 672, stack limit = 0x000000008551ec94)
[   19.248482] Call trace:
[   19.249002]  0xffff4cb953d29853
[   19.249663]  __seccomp_filter+0x3c/0x420
[   19.250483]  __secure_computing+0x4c/0xc0
[   19.251317]  syscall_trace_enter+0x100/0x2f0
[   19.252209]  __sys_trace+0x18/0x48
[   19.252923] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[   19.254183] ---[ end trace 2597149423910213 ]---
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
[  OK  ] Stopped udev Kernel Device Manager.
[FAILED] Failed to start udev Kernel Device Manager.
See 'systemctl status systemd-udevd.service' for details.
         Starting dracut pre-trigger hook...[   19.308532] systemd[673]: dracut-pre-trigger.service: Failed to connect 
stdout to the journal socket, ignoring: Connection refused

[   19.310596] systemd[673]: dracut-pre-trigger.service: Failed to connect stderr to the journal socket, ignoring: Conn
ection refused
[  OK  ] Started dracut pre-trigger hook.
         Starting udev Coldplug all Devices...[   19.769403] systemd[749]: systemd-udev-trigger.service: Failed to conn
ect stdout to the journal socket, ignoring: Connection refused

[   19.961672] systemd[750]: systemd-udev-trigger.service: Failed to connect stdout to the journal socket, ignoring: Co
nnection refused
[  OK  ] Started udev Coldplug all Devices.
         Starting dracut initqueue hook...
[   20.213563] systemd[751]: dracut-initqueue.service: Failed to connect stdout to the journal socket, ignoring: Connec
tion refused
[   20.215827] systemd[751]: dracut-initqueue.service: Failed to connect stderr to the journal socket, ignoring: Connec
tion refused
         Starting Show Plymouth Boot Screen...
[   20.226541] systemd[752]: plymouth-start.service: Failed to connect stdout to the journal socket, ignoring: Connecti
on refused
         Starting udev Wait for Complete Device Initialization...
[   20.242431] systemd[753]: systemd-udev-settle.service: Failed to connect stdout to the journal socket, ignoring: Con
nection refused
[   20.267844] systemd[760]: plymouth-start.service: Failed to connect stdout to the journal socket, ignoring: Connecti
on refused
[  OK  ] Started udev Wait for Complete Device Initialization.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Show Plymouth Boot Screen.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Local Encrypted Volumes.
[  OK  ] Started Forward Password Requests to Plymouth Directory Watch.
[   20.300802] systemd[763]: multipathd.service: Failed to connect stdout to the journal socket, ignoring: Connection r
efused
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
[   20.323082] systemd[768]: systemd-tmpfiles-setup.service: Failed to connect stdout to the journal socket, ignoring: 
Connection refused
[   20.323353] systemd[766]: multipathd.service: Failed to connect stdout to the journal socket, ignoring: Connection r
efused
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[  227.651212] systemd[1]: Starting Setup Virtual Console...
         Starting Setup Virtual Console...
[  227.654842] systemd[4094]: systemd-vconsole-setup.service: Failed to connect stdout to the journal socket, ignoring: Connection refused
[  227.673537] systemd-vconsole-setup[4094]: KD_FONT_OP_GET failed while trying to get the font metadata: Function not implemented
[  227.675369] systemd-vconsole-setup[4094]: Fonts will not be copied to remaining consoles
[  227.685308] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Setup Virtual Console.
[  227.693160] systemd[1]: Starting Dracut Emergency Shell...
         Starting Dracut Emergency Shell...
[  227.732757] systemd[1]: Received SIGRTMIN+21 from PID 759 (plymouthd).
Warning: /dev/root does not exist

Generating "/run/initramfs/rdsosreport.txt"


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


dracut:/#

I have untagged 239-1.fc29 for now so we can get a compose tonight/tomorrow. 
It's possible this is a kernel issue, but starting here. 

Happy to collect more info, etc.

Comment 1 Kevin Fenzi 2018-06-23 17:35:41 UTC
So, we got a pretty much identical failure today, which makes me think this is kernel related. 

On the 20th, kernel-4.18.0-0.rc1.git2.1.fc29 worked
On the 21st we didn't have a compose for reasons, but kernel-4.18.0-0.rc1.git3.1.fc29 was built.
On the 22nd and today we have kernel-4.18.0-0.rc1.git4.1.fc29 which failed this way. 

I guess I will untag kernel-4.18.0-0.rc1.git4.1.fc29 and see if we get a compose with git3.1? Or just untag them both and confirm that git2.1 still works...

Comment 2 Peter Robinson 2018-06-25 14:16:13 UTC
yes, so seeing this on a F-28 userspace as well so it's not systemd

[    4.573522] Internal error: SP/PC alignment exception: 8a000000 [#1] SMP
[    4.580196] Modules linked in:
[    4.583239] CPU: 7 PID: 236 Comm: (journald) Tainted: G        W         4.18.0-0.rc1.git4.1.fc29.aarch64 #1
[    4.593021] Hardware name: AppliedMicro X-Gene Mustang Board/X-Gene Mustang Board, BIOS 3.06.21 Sep 27 2016
[    4.593024] pstate: 60400005 (nZCv daif +PAN -UAO)
[    4.593030] pc : 0xffff000000b7a347
[    4.593038] lr : seccomp_run_filters+0x4c/0xb0
[    4.615373] sp : ffff00000e9dbd10
[    4.615374] x29: ffff00000e9dbd10 x28: ffff8003e6908000 
[    4.615378] x27: ffff000008b72000 x26: 0000000000000115 
         [    4.629243] x25: 0000000000000125 x24: 0000000000000015 
Starting Create list of required st…ce nodes for the current k[    4.629247] x23: 0000000000000000 x22: ffff00000e9dbdc8 
ernel...
[    4.646115] x21: ffff00000e9dbd40 x20: 000000007fff0000 
[    4.652261] x19: ffff8003e6d74580 x18: 000000000000004f 
[    4.657548] x17: 0000ffff9a3c27c0 x16: 0000000000000800 
[    4.662835] x15: 000000000000270f x14: 0000000000000000 
[    4.668121] x13: 0000000000000000 x12: 0000000000000001 
[    4.673408] x11: 0000000000000002 x10: 0000000000000000 
[    4.678696] x9 : 0000ffff99b6f700 x8 : 0000000000000115 
[    4.678700] x7 : 0000000000000038 x6 : 0000aaab0a21f490 
[    4.678703] x5 : 0000000000000007 x4 : 0000000000000000 
[    4.694555] x3 : ffff8003e6908000 x2 : ffff000000b7a347 
[[    4.699841] x1 : ffff00000c35d038 x0 : ffff00000e9dbd40 
  OK  [   ] Reached target Timers.
 4.705212] Process (journald) (pid: 236, stack limit = 0x(____ptrval____))
[    4.715849] Call trace:
[    4.718282]  0xffff000000b7a347
[    4.721406]  __seccomp_filter+0x3c/0x420
[    4.725308]  __secure_computing+0x4c/0xc0
[    4.729299]  syscall_trace_enter+0x100/0x2f0
[    4.733546]  __sys_trace+0x18/0x48
[    4.736931] Code: 202000d4 202000d4 202000d4 202000d4 (a9bf7bfd) 
[    4.742997] ---[ end trace cef19c546f74b884 ]---

Comment 3 Peter Robinson 2018-06-25 14:20:12 UTC
This *might* be fixed in the net merge (bpf fixes) that has already landed upstream

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6f0d349d922ba44e4348a17a78ea51b7135965b1

Comment 4 Peter Robinson 2018-06-25 22:42:56 UTC
Boots on mustang with 4.18.0-0.rc2.git1.1.fc29.aarch64 for me. I'll test others before closing this out.

Comment 5 Peter Robinson 2018-06-25 23:12:09 UTC
Right tested this on and they all work at least to boot to text with network:
* DragonBoard 410c
* RPi 3B+
* Pine64

Kevin I'll leave you to close this out once you're happy the compose looks good.

Comment 6 Kevin Fenzi 2018-06-26 16:21:35 UTC
Indeed things composed today. Unfortunately the compose as a whole failed, but it was not due to this issue anymore. ;)