Bug 859721 - Constant segv on boot with systemd-191-2.fc18
Constant segv on boot with systemd-191-2.fc18
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: systemd (Show other bugs)
18
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: systemd-maint
Fedora Extras Quality Assurance
:
: 859648 859696 861130 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-23 10:24 EDT by Mamoru TASAKA
Modified: 2012-10-07 18:35 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-10-07 18:35:19 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
FreeDesktop.org 55213 None None None 2012-09-24 17:11:22 EDT

  None (edit)
Description Mamoru TASAKA 2012-09-23 10:24:11 EDT
Description of problem:
With systemd-191-2.fc18, systemd process (process 1) constantly segfaults on boot, and boot sequence cannot proceed.
This segv happens at very early stage and no logs is left for this segv.


Version-Release number of selected component (if applicable):
systemd-191-2.fc18

How reproducible:
100%

Steps to Reproduce:
1. boot
2.
3.
  
Actual results:
process 1 segfaults, boot sequence stops.

Expected results:
No systemd segv

Additional info:
Had to downgrade to 188-3.fc18
Comment 1 Mamoru TASAKA 2012-09-23 10:50:13 EDT
selinux is disabled.
Comment 2 Sergei LITVINENKO 2012-09-23 16:02:06 EDT
Updating systemd with its dependencies cause system problems

Выполнение сценария
  Обновление  : systemd-libs-191-2.fc18.i686		1/8 
  Обновление  : systemd-191-2.fc18.i686			2/8 

Message from syslogd@localhost at Sep 23 22:25:10 ...
 kernel:[  153.790792] systemd[1]: segfault at 7d ip b7442447 sp bfecf628 error 4 in libc-2.16.so[b73c2000+1af000]
  Обновление  : systemd-sysv-191-2.fc18.i686		3/8 
  Обновление  : libgudev1-191-2.fc18.i686		4/8 
  Очистка     : libgudev1-188-3.fc18.i686		5/8 
  Очистка     : systemd-sysv-188-3.fc18.i686		6/8 
  Очистка     : systemd-libs-188-3.fc18.i686		7/8 
  Очистка     : systemd-188-3.fc18.i686			8/8 
  Проверка    : systemd-191-2.fc18.i686			1/8 
  Проверка    : systemd-libs-191-2.fc18.i686		2/8 
  Проверка    : libgudev1-191-2.fc18.i686		3/8 
  Проверка    : systemd-sysv-191-2.fc18.i686		4/8 
  Проверка    : systemd-188-3.fc18.i686			5/8 
  Проверка    : systemd-libs-188-3.fc18.i686		6/8 
  Проверка    : libgudev1-188-3.fc18.i686		7/8 
  Проверка    : systemd-sysv-188-3.fc18.i686		8/8 

Обновлено:
  systemd.i686 0:191-2.fc18

Обновлены зависимости:
  libgudev1.i686 0:191-2.fc18  systemd-libs.i686 0:191-2.fc18 systemd-sysv.i686 0:191-2.fc18

---------------------------------------------------------------------------
After updating it is not possible to shutdown system because /dev/initctl

[root@localhost ~]# shutdown -h now
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.

In case if system will be switched off, it will be not able to boot.

Downgrading help.
After downgrading systemd, systemd-libs, systemd-sysv, libgudev1 to version 0:188-3.fc18 it is still not possible to shutdown, because /dev/initctl, but switched off system can boot again and after work as expected.

PS: I can not identify which exactly: systemd or libgudev1 or systemd-libs or systemd-sysv is buggy.
Comment 3 Mamoru TASAKA 2012-09-24 09:04:33 EDT
Well, getting backtrace when reboot with systemd-191-2.fc18 is very difficult... because this segv occurs at very early stage.

By the way, (although not sure the following issue is the same as the one on boot), like Sergei's comment 2 segv also happens when just upgrading systemd from 188-3.fc18 to 191-2.fc18 like:

[root@localhost systemd-new]# rpm -Fvh *rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:systemd-libs-191-2.fc18          ################################# [ 10%]
   2:systemd-191-2.fc18               ################################# [ 20%]

Message from syslogd@localhost at Sep 24 21:52:50 ...
 kernel:[82425.955159] systemd[1]: segfault at 7d ip b7449447 sp bff385a8 error 4 in libc-2.16.so[b73c9000+1af000]
   3:systemd-analyze-191-2.fc18       ################################# [ 30%]
   4:systemd-sysv-191-2.fc18          ################################# [ 40%]
   5:libgudev1-191-2.fc18             ################################# [ 50%]
Cleaning up / removing...
   6:libgudev1-188-3.fc18             ################################# [ 60%]
   7:systemd-sysv-188-3.fc18          ################################# [ 70%]
   8:systemd-analyze-188-3.fc18       ################################# [ 80%]
   9:systemd-libs-188-3.fc18          ################################# [ 90%]
  10:systemd-188-3.fc18               ################################# [100%]

/var/log/messages says:
Sep 24 21:52:34 localhost dbus[1974]: [system] Reloaded configuration
Sep 24 21:52:47 localhost systemd[1]: Reexecuting.
Sep 24 21:52:48 localhost systemd[1]: systemd 191 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ; fedora)
Sep 24 21:52:49 localhost systemd[1]: Caught <SEGV>, dumped core as pid 5261.
Sep 24 21:52:49 localhost systemd[1]: Freezing execution.
Sep 24 21:52:50 localhost kernel: [82425.955159] systemd[1]: segfault at 7d ip b7449447 sp bff385a8 error 4 in libc-2.16.so[b73c9000+1af000]
Sep 24 21:52:49 localhost systemd[1]: Caught <SEGV>, dumped core as pid 5261.
Sep 24 21:52:49 localhost systemd[1]: Freezing execution.

and for this segv:
# gdb systemd /core.5261
GNU gdb (GDB) Fedora (7.5-19.fc18)
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/systemd...Reading symbols from /usr/lib/debug/usr/lib/systemd/systemd.debug...done.
done.
[New LWP 5261]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
Core was generated by `/usr/lib/systemd/systemd --system --deserialize 21'.
Program terminated with signal 11, Segmentation fault.
#0  0xb76a8424 in __kernel_vsyscall ()
Missing separate debuginfos, use: debuginfo-install audit-libs-2.2.1-2.fc18.i686 kmod-libs-10-1.fc18.i686 libattr-2.4.46-7.fc18.i686 libcap-2.22-3.fc18.i686 libgcc-4.7.2-2.fc18.i686 libselinux-2.1.12-1.fc18.i686 pam-1.1.6-1.fc18.i686 pcre-8.31-2.fc18.i686 tcp_wrappers-libs-7.6-70.fc18.i686 xz-libs-5.1.2-2alpha.fc18.i686
(gdb) bt
#0  0xb76a8424 in __kernel_vsyscall ()
#1  0xb73f7b5f in __GI_raise (sig=sig@entry=11) at ../nptl/sysdeps/unix/sysv/linux/raise.c:63
#2  0xb76da7d0 in crash (sig=11) at src/core/main.c:134
#3  <signal handler called>
#4  strchrnul () at ../sysdeps/i386/strchrnul.S:67
#5  0xb740c400 in __find_specmb (format=0x7d <Address 0x7d out of bounds>) at printf-parse.h:109
#6  _IO_vfprintf_internal (s=s@entry=0xbff38b50, format=format@entry=0x7d <Address 0x7d out of bounds>, 
    ap=ap@entry=0xbff39548 "I") at vfprintf.c:1288
#7  0xb74d63c1 in __GI___vasprintf_chk (result_ptr=result_ptr@entry=0xbff38c64, flags=flags@entry=1, 
    format=format@entry=0x7d <Address 0x7d out of bounds>, args=0xbff39548 "I") at vasprintf_chk.c:66
#8  0xb7731679 in vasprintf (__ap=<optimized out>, __fmt=0x7d <Address 0x7d out of bounds>, __ptr=0xbff38c64)
    at /usr/include/bits/stdio2.h:210
#9  log_struct_internal (level=30, level@entry=6, file=file@entry=0xb77892b4 "src/core/unit.c", 
    line=line@entry=1029, 
    func=func@entry=0xb7789f00 <__func__.12455> "unit_status_log_starting_stopping_reloading", 
    format=0x7d <Address 0x7d out of bounds>, 
    format@entry=0xb7758994 "MESSAGE_ID=%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x")
    at src/shared/log.c:709
#10 0xb7745c9d in unit_status_log_starting_stopping_reloading (u=u@entry=0xb8f0b708, t=t@entry=JOB_START)
    at src/core/unit.c:1025
#11 0xb7749731 in unit_start (u=0xb8f0b708) at src/core/unit.c:1072
#12 0xb774fc8f in job_run_and_invalidate (j=0xb8ee2858) at src/core/job.c:500
#13 0xb76dd52f in manager_dispatch_run_queue (m=m@entry=0xb8edb480) at src/core/manager.c:909
#14 0xb76e0848 in manager_loop (m=0xb8edb480) at src/core/manager.c:1456
#15 0xb76d78c1 in main (argc=4, argv=0xbff3a414) at src/core/main.c:1696
(gdb) quit

Using i686. selinux disabled. Again not sure this segv is the same as the one occuring on boot.
Comment 4 Mamoru TASAKA 2012-09-24 09:08:02 EDT
Using glibc-2.16-15.fc18.i686
Comment 5 Sergei LITVINENKO 2012-09-24 13:35:39 EDT
It looks like bug is in mainstream, because not only fedora package is affected.

https://bbs.archlinux.org/viewtopic.php?pid=1164825
Comment 6 Mamoru TASAKA 2012-09-24 17:11:22 EDT
Looks like this is
https://bugs.freedesktop.org/show_bug.cgi?id=55213
Comment 7 Lennart Poettering 2012-09-24 19:20:52 EDT
Fixed in git, will update bodhi soon.
Comment 8 Lennart Poettering 2012-09-24 19:26:21 EDT
*** Bug 859648 has been marked as a duplicate of this bug. ***
Comment 9 Lennart Poettering 2012-09-24 19:27:41 EDT
*** Bug 859696 has been marked as a duplicate of this bug. ***
Comment 10 Fedora Update System 2012-09-25 18:31:18 EDT
glibc-2.16-17.fc18, systemd-192-1.fc18, selinux-policy-3.11.1-23.fc18, rtkit-0.11-3.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/selinux-policy-3.11.1-23.fc18,rtkit-0.11-3.fc18,systemd-192-1.fc18,glibc-2.16-17.fc18
Comment 11 Lennart Poettering 2012-09-27 12:40:07 EDT
*** Bug 861130 has been marked as a duplicate of this bug. ***
Comment 12 Fedora Update System 2012-09-27 20:14:43 EDT
Package glibc-2.16-17.fc18, systemd-192-1.fc18, selinux-policy-3.11.1-23.fc18, rtkit-0.11-3.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing glibc-2.16-17.fc18 systemd-192-1.fc18 selinux-policy-3.11.1-23.fc18 rtkit-0.11-3.fc18'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/selinux-policy-3.11.1-23.fc18,rtkit-0.11-3.fc18,systemd-192-1.fc18,glibc-2.16-17.fc18
then log in and leave karma (feedback).
Comment 13 Sergei LITVINENKO 2012-10-01 13:59:27 EDT
[root@f18a ~]# rpm -qa | grep glibc
glibc-2.16-17.fc18.i686
glibc-common-2.16-17.fc18.i686

[root@f18a ~]# ll
total 6032
-rw-r--r--  1 root root 4349060 Sep 21 23:05 glibc-2.16-17.fc18.i686.rpm
-rw-r--r--  1 root root   33336 Sep 28 03:48 libgudev1-193-1.fc18.i686.rpm
-rw-r--r--  1 root root   48976 Sep 14 19:19 rtkit-0.11-3.fc18.i686.rpm
-rw-r--r--  1 root root 1526300 Sep 28 03:48 systemd-193-1.fc18.i686.rpm
-rw-r--r--  1 root root   22736 Sep 28 03:48 systemd-analyze-193-1.fc18.i686.rpm
-rw-r--r--  1 root root  127544 Sep 28 03:48 systemd-libs-193-1.fc18.i686.rpm
-rw-r--r--  1 root root   27660 Sep 28 03:48 systemd-python-193-1.fc18.i686.rpm
-rw-r--r--  1 root root   21216 Sep 28 03:48 systemd-sysv-193-1.fc18.i686.rpm

[root@f18a ~]# yum update *.rpm
...

Dependency Installed:
  libmicrohttpd.i686 0:0.9.22-1.fc18

Updated:
 libgudev1.i686 0:193-1.fc18 systemd.i686 0:193-1.fc18 systemd-libs.i686 0:193-1.fc18 systemd-sysv.i686 0:193-1.fc18          

Complete!


So, system shutdown without error and boot after.

[root@f18a ~]# rpm -qa | grep -E 'glibc|rtkit|systemd'
systemd-libs-193-1.fc18.i686
glibc-2.16-17.fc18.i686
systemd-sysv-193-1.fc18.i686
glibc-common-2.16-17.fc18.i686
systemd-193-1.fc18.i686
rtkit-0.11-3.fc18.i686
Comment 14 Fedora Update System 2012-10-01 16:05:08 EDT
Package glibc-2.16-17.fc18, rtkit-0.11-3.fc18, systemd-193-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing glibc-2.16-17.fc18 rtkit-0.11-3.fc18 systemd-193-1.fc18'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/rtkit-0.11-3.fc18,systemd-193-1.fc18,glibc-2.16-17.fc18
then log in and leave karma (feedback).

Note You need to log in before you can comment on or make changes to this bug.