Bug 2035812 - systemd-journald doesn't start on armv7 minimal initramfs
Summary: systemd-journald doesn't start on armv7 minimal initramfs
Keywords:
Status: CLOSED DUPLICATE of bug 2035802
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-27 18:56 UTC by Kevin Fenzi
Modified: 2021-12-28 17:08 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-12-28 17:08:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Kevin Fenzi 2021-12-27 18:56:08 UTC
[  459.432051] systemd[1]: Received SIGRTMIN+21 from PID 1021 (plymouthd).                           
[  459.441992] systemd[1]: Received SIGRTMIN+21 from PID 1021 (plymouthd).                           
Warning: /dev/root does not exist                                                                    
                                                                                                     
Generating "/run/initramfs/rdsosreport.txt"                                                          
                                                                                                     
                                                                                                     
Entering emergency mode. Exit the shell to continue.                                                 
Type "journalctl" to view system logs.                                                               
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot                      
after mounting them and attach it to a bug report.                                                   
                                                                                                     
                                                                                                     
Press Enter for maintenance                                                                          
(or press Control-D to continue):                                                                    
sh-5.1# systemctl --failed                                                                           
  UNIT                            LOAD   ACTIVE SUB    DESCRIPTION             >                     
● dbus-broker.service             loaded failed failed D-Bus System Message Bus                      
● systemd-journald.service        loaded failed failed Journal Service                               
● systemd-udevd.service           loaded failed failed Rule-based Manager for D>                     
● dbus.socket                     loaded failed failed D-Bus System Message Bus>                     
● systemd-journald-audit.socket   loaded failed failed Journal Audit Socket                          
● systemd-journald-dev-log.socket loaded failed failed Journal Socket (/dev/log)                     ● systemd-journald.socket         loaded failed failed Journal Socket                                
● systemd-udevd-control.socket    loaded failed failed udev Control Socket                           
● systemd-udevd-kernel.socket     loaded failed failed udev Kernel Socket                            
                                                                                                     
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
9 loaded units listed.
sh-5.1# systemctl start systemd-journald
[  489.575031] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[  489.579092] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[  489.583430] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[  489.623466] systemd[1]: Starting systemd-journald.service - Journal Service...
[  489.924593] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[  489.928950] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  489.932935] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  490.273924] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
Job for systemd-journald.service failed because the control process exited with error code.
See "systemctl status systemd-journald.service" and "journalctl -xeu systemd-journald.service" for de
tails.[  490.514987] systemd[1]: Stopped systemd-journald.service - Journal Service.

sh-5.1# [  491.151715] systemd[1]: Starting systemd-journald.service - Journal Service...
[  491.333804] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[  491.338217] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  491.342465] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  491.357716] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 2.
[  491.360560] systemd[1]: Stopped systemd-journald.service - Journal Service.
[  492.423290] systemd[1]: Starting systemd-journald.service - Journal Service...
[  492.852990] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[  492.857113] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  492.861107] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  492.909079] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 3.
[  492.912039] systemd[1]: Stopped systemd-journald.service - Journal Service.
[  493.953034] systemd[1]: Starting systemd-journald.service - Journal Service...
[  494.391760] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[  494.395917] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  494.399989] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  494.448774] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 4.
[  494.451436] systemd[1]: Stopped systemd-journald.service - Journal Service.
[  495.493953] systemd[1]: Starting systemd-journald.service - Journal Service...
[  495.921995] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[  495.926219] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  495.930347] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  495.955936] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 5.
[  495.959004] systemd[1]: Stopped systemd-journald.service - Journal Service.
[  495.961019] systemd[1]: systemd-journald.service: Start request repeated too quickly.
[  495.962766] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.
[  496.089975] systemd[1]: Failed to start systemd-journald.service - Journal Service.
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.
[  496.367170] systemd[1]: systemd-journald.socket: Failed with result 'service-start-limit-hit'.
[  496.700123] systemd[1]: systemd-journald-audit.socket: Failed with result 'service-start-limit-hit
'.
[  496.911368] systemd[1]: systemd-journald-dev-log.socket: Failed with result 'service-start-limit-h
it'.

sh-5.1# systemctl status systemd-journald.service
× systemd-journald.service - Journal Service
     Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static)
     Active: failed (Result: exit-code) since Thu 2021-12-23 00:08:04 UTC; 32s >
TriggeredBy: × systemd-journald.socket
             × systemd-journald-audit.socket
             × systemd-journald-dev-log.socket
       Docs: man:systemd-journald.service(8)
             man:journald.conf(5)
    Process: 5552 ExecStart=/usr/lib/systemd/systemd-journald (code=exited, sta>
   Main PID: 5552 (code=exited, status=127)
        CPU: 490ms
sh-5.1# systemctl status systemd-journald.service | cat
× systemd-journald.service - Journal Service
     Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static)
     Active: failed (Result: exit-code) since Thu 2021-12-23 00:08:04 UTC; 43s ago
TriggeredBy: × systemd-journald.socket
             × systemd-journald-audit.socket
             × systemd-journald-dev-log.socket
       Docs: man:systemd-journald.service(8)
             man:journald.conf(5)
    Process: 5552 ExecStart=/usr/lib/systemd/systemd-journald (code=exited, status=127)
   Main PID: 5552 (code=exited, status=127)
        CPU: 490ms
sh-5.1# /usr/lib/systemd/systemd-journald                                                            
                                                                                                     
[1]+  Stopped(SIGTSTP)        /usr/lib/systemd/systemd-journald                                      
sh-5.1# bg
[1] /usr/lib/systemd/systemd-journald &
sh-5.1#

Dec 23 00:09:22 fedora systemd[1]: System time before build time, advancing clock.                   
Dec 23 00:09:22 fedora systemd[1]: systemd v250-1.fc36 running in system mode (+PAM +AUDIT +SELINUX $
APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN 
-IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD 
-BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)                                 
Dec 23 00:09:22 fedora systemd[1]: Detected virtualization kvm.                                      
Dec 23 00:09:22 fedora systemd[1]: Detected architecture arm.                                        
Dec 23 00:09:22 fedora systemd[1]: Running in initial RAM disk.                                      
Dec 23 00:09:22 fedora systemd[1]: No hostname configured, using default hostname.                   
Dec 23 00:09:22 fedora systemd[1]: Hostname set to <fedora>.                                         
Dec 23 00:09:22 fedora systemd[1]: Initializing machine ID from VM UUID.                             
Dec 23 00:09:22 fedora kernel: hrtimer: interrupt took 9985925 ns                                    
Dec 23 00:09:22 fedora systemd-gpt-auto-generator[213]: EFI loader partition unknown, exiting.       
Dec 23 00:09:22 fedora systemd-gpt-auto-generator[213]: (The boot loader did not set EFI variable Lo$
derDevicePartUUID.)                                                                                  
Dec 23 00:09:22 fedora systemd[1]: Queued start job for default target initrd.target.                
Dec 23 00:09:22 fedora systemd[1]: Reached target initrd-root-device.target - Initrd Root Device.    
Dec 23 00:09:22 fedora systemd[1]: Reached target initrd-usr-fs.target - Initrd /usr File System.    
Dec 23 00:09:22 fedora systemd[1]: Reached target slices.target - Slice Units.                       
Dec 23 00:09:22 fedora systemd[1]: Reached target swap.target - Swaps.                               
Dec 23 00:09:22 fedora systemd[1]: Reached target timers.target - Timer Units.                       
Dec 23 00:09:22 fedora systemd[1]: Listening on dbus.socket - D-Bus System Message Bus Socket.       
Dec 23 00:09:22 fedora systemd[1]: Listening on iscsid.socket - Open-iSCSI iscsid Socket.            
Dec 23 00:09:22 fedora systemd[1]: Listening on iscsiuio.socket - Open-iSCSI iscsiuio Socket.        
Dec 23 00:09:22 fedora systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket$
Dec 23 00:09:22 fedora systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/d$
v/log).                                                                                              
Dec 23 00:09:22 fedora systemd[1]: Listening on systemd-journald.socket - Journal Socket.            
Dec 23 00:09:22 fedora systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.  
Dec 23 00:09:22 fedora systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.    
Dec 23 00:09:22 fedora systemd[1]: Reached target sockets.target - Socket Units.                     
Dec 23 00:09:22 fedora systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device 
Nodes...                                                                                             
Dec 23 00:09:22 fedora systemd[1]: Started rngd.service - Hardware RNG Entropy Gatherer Daemon.      
Dec 23 00:09:22 fedora systemd[1]: Starting systemd-journald.service - Journal Service...            
Dec 23 00:09:22 fedora systemd[1]: systemd-modules-load.service - Load Kernel Modules was skipped be$
ause all trigger condition checks failed.                                                            
Dec 23 00:09:22 fedora systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...       
Dec 23 00:09:22 fedora systemd[1]: systemd-sysusers.service - Create System Users was skipped becaus$
 of a failed condition check (ConditionNeedsUpdate=/etc).                                            
Dec 23 00:09:22 fedora systemd[1]: Starting systemd-vconsole-setup.service - Setup Virtual Console..$
Dec 23 00:09:22 fedora systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device 
Nodes.                                                                                               
Dec 23 00:09:22 fedora systemd[1]: systemd-journald.service: Main process exited, code=exited, statu$
=127/n/a                                                                                             
Dec 23 00:09:22 fedora systemd[1]: systemd-journald.service: Failed with result 'exit-code'.         
Dec 23 00:09:22 fedora systemd[1]: Failed to start systemd-journald.service - Journal Service.

sh-5.1# systemctl restart systemd-journald                                                          
sh-5.1# systemctl restart systemd-journald                                                          
[ 1261.285237] audit: type=1335 audit(1640218850.269:4): pid=1 uid=0 auid=4294967295 tty=(none) ses=4294967295 subj=kernel comm="systemd" exe="/usr/lib/systemd/systemd" nl-mcgrp=1 op=connect res=1     
[ 1261.286427] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.       
[ 1261.304161] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 1261.309605] systemd[1]: Listening on systemd-journald.socket - Journal Socket.                   
[ 1261.330161] audit: type=1334 audit(1640218850.314:5): prog-id=59 op=LOAD                         
[ 1261.334745] audit: type=1334 audit(1640218850.315:6): prog-id=60 op=LOAD                         
[ 1261.337490] audit: type=1334 audit(1640218850.321:7): prog-id=61 op=LOAD                         
[ 1261.339154] audit: type=1334 audit(1640218850.322:8): prog-id=0 op=UNLOAD                        
[ 1261.340619] audit: type=1334 audit(1640218850.322:9): prog-id=0 op=UNLOAD                        
[ 1261.347775] systemd[1]: Starting systemd-journald.service - Journal Service...                   
[ 1261.650355] systemd[1]: systemd-journald.service: Main process exited, code=exited, status=127/n/a
[ 1261.664306] systemd[1]: systemd-journald.service: Failed with result 'exit-code'.                
[ 1261.668587] systemd[1]: Failed to start systemd-journald.service - Journal Service.              
[FAILED] Failed to start systemd-jo…ald.service - Journal Service.                                  
[ 1261.673393] audit: type=1130 audit(1640218850.658:10): pid=1 uid=0 auid=4294967295 ses=4294967295
subj=kernel msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[ 1262.352273] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
Job for systemd-journald.service failed because the control process exited with error code.

So, it seems to run ok manually, but not from the unit?

Also, this only seems to be happening on 32bit arm?

Comment 1 Kevin Fenzi 2021-12-27 20:32:04 UTC
I isolated this some more: 

20211222.n.0 rawhide - ok
20211223.n.0 rawhide - problem shows up. 

The only thing that looks like it could at all be related in the updated packages set is libcap-ng (which systemd links to), but I can't tell if the change there would matter any. 

I'm trying another compose with that libcap-ng untagged (and glibc for the ldconfig bug #2034715).

Comment 2 Gary Buhrmaster 2021-12-28 02:17:05 UTC
FWIW, it seems this is the libzstd 1.5.1-1 update (1.5.0-2 seems to work) for armv7.

Comment 3 Kevin Fenzi 2021-12-28 03:04:41 UTC
Yeah, libcap-ng wasn't it. :) Thanks, will try zstd.

Comment 4 Gary Buhrmaster 2021-12-28 03:47:06 UTC
libzstd 1.5.1 failing on armv7hl was also mentioned in BZ#2035802

Comment 5 Zbigniew Jędrzejewski-Szmek 2021-12-28 17:08:01 UTC

*** This bug has been marked as a duplicate of bug 2035802 ***


Note You need to log in before you can comment on or make changes to this bug.