Bug 1244570

Summary: [abrt] systemd: journal_file_find_data_object_with_hash(): systemd-journald killed by SIGABRT
Product: [Fedora] Fedora Reporter: Trevor Davenport <trevor.davenport>
Component: systemdAssignee: systemd-maint
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: htl10, johannbg, jsynacek, lnykryn, msekleta, omega13a, s, systemd-maint, vsapronenko, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
URL: https://retrace.fedoraproject.org/faf/reports/bthash/c593d6da1cf33aa98e8e61cb83adeb24d1023194
Whiteboard: abrt_hash:1a9ff5226321e90417d71606e9fc8aaa132caa85
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-20 13:10:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
File: backtrace
none
File: cgroup
none
File: core_backtrace
none
File: dso_list
none
File: environ
none
File: limits
none
File: maps
none
File: mountinfo
none
File: namespaces
none
File: open_fds
none
File: proc_pid_status none

Description Trevor Davenport 2015-07-20 03:04:30 UTC
Version-Release number of selected component:
systemd-219-19.fc22

Additional info:
reporter:       libreport-2.6.0
backtrace_rating: 4
cmdline:        /usr/lib/systemd/systemd-journald
crash_function: journal_file_find_data_object_with_hash
executable:     /usr/lib/systemd/systemd-journald
global_pid:     489
kernel:         4.0.7-300.fc22.x86_64
runlevel:       N 5
type:           CCpp
uid:            0

Truncated backtrace:
Thread no. 1 (10 frames)
 #0 journal_file_find_data_object_with_hash at ../src/journal/journal-file.c:876
 #1 journal_file_append_data.lto_priv.143 at ../src/journal/journal-file.c:1032
 #2 journal_file_append_entry.constprop.104 at ../src/journal/journal-file.c:1414
 #3 write_to_journal at ../src/journal/journald-server.c:512
 #4 dispatch_message_real.lto_priv.228 at ../src/journal/journald-server.c:819
 #5 server_dispatch_message at ../src/journal/journald-server.c:917
 #6 server_process_syslog_message at ../src/journal/journald-syslog.c:375
 #7 server_process_datagram at ../src/journal/journald-server.c:1205
 #8 source_dispatch at ../src/libsystemd/sd-event/sd-event.c:2115
 #9 sd_event_dispatch at ../src/libsystemd/sd-event/sd-event.c:2472

Comment 1 Trevor Davenport 2015-07-20 03:04:33 UTC
Created attachment 1053710 [details]
File: backtrace

Comment 2 Trevor Davenport 2015-07-20 03:04:34 UTC
Created attachment 1053711 [details]
File: cgroup

Comment 3 Trevor Davenport 2015-07-20 03:04:35 UTC
Created attachment 1053712 [details]
File: core_backtrace

Comment 4 Trevor Davenport 2015-07-20 03:04:36 UTC
Created attachment 1053713 [details]
File: dso_list

Comment 5 Trevor Davenport 2015-07-20 03:04:37 UTC
Created attachment 1053714 [details]
File: environ

Comment 6 Trevor Davenport 2015-07-20 03:04:37 UTC
Created attachment 1053715 [details]
File: limits

Comment 7 Trevor Davenport 2015-07-20 03:04:38 UTC
Created attachment 1053716 [details]
File: maps

Comment 8 Trevor Davenport 2015-07-20 03:04:39 UTC
Created attachment 1053717 [details]
File: mountinfo

Comment 9 Trevor Davenport 2015-07-20 03:04:40 UTC
Created attachment 1053718 [details]
File: namespaces

Comment 10 Trevor Davenport 2015-07-20 03:04:41 UTC
Created attachment 1053719 [details]
File: open_fds

Comment 11 Trevor Davenport 2015-07-20 03:04:42 UTC
Created attachment 1053720 [details]
File: proc_pid_status

Comment 12 Hin-Tak Leung 2015-08-04 11:32:24 UTC
# abrt-cli list
id 257bc925df1aba3c65739edd5bc03b878d002670
reason:         systemd-journald killed by SIGABRT
time:           Tue 04 Aug 2015 00:43:52 BST
cmdline:        /usr/lib/systemd/systemd-journald
package:        systemd-219-20.fc22
uid:            0 (root)
count:          1
Directory:      /var/spool/abrt/ccpp-2015-08-04-00:43:52-24112

# uname -r
4.1.3-200.fc22.x86_64

(gdb) bt
#0  0x000056555556639f in journal_file_find_data_object_with_hash ()
#1  0x0000565555567273 in journal_file_append_data.lto_priv ()
#2  0x0000565555572961 in journal_file_append_entry.constprop ()
#3  0x000056555555fc4b in dispatch_message_real.lto_priv ()
#4  0x000056555558756c in server_dispatch_message ()
#5  0x0000565555561fc3 in server_read_dev_kmsg.lto_priv ()
#6  0x000056555555a2f9 in main ()

looks similar enough.

Comment 13 Hin-Tak Leung 2015-08-04 11:54:44 UTC
#tac /var/log/messages | grep journal |more
...
Aug  4 00:52:04 localhost systemd: systemd-journald.service watchdog timeout (limit 1min)!
Aug  4 00:52:04 localhost journal: Journal started
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service failed.
Aug  4 00:52:04 localhost systemd[1]: Unit systemd-journald.service entered failed state.
Aug  4 00:52:04 localhost systemd[1]: Job systemd-journal-flush.service/start failed with result 'dependency'.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service still around after final SIGKILL. Entering failed mode.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service stop-final-sigterm timed out. Killing.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service still around after SIGKILL. Ignoring.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service stop-sigterm timed out. Killing.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service stop-sigabrt timed out. Terminating.
Aug  4 00:52:04 localhost systemd[1]: systemd-journald.service watchdog timeout (limit 1min)!
Aug  4 00:51:31 localhost journal: Permanent journal is using 2.1G (max allowed 4.0G, trying to leave 4.0G free of 206.5M available → 
current limit 2.1G).
Aug  4 00:43:49 localhost systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
Aug  4 00:43:49 localhost systemd[1]: systemd-journald.service failed.
Aug  4 00:43:49 localhost systemd[1]: Unit systemd-journald.service entered failed state.
Aug  4 00:43:49 localhost systemd[1]: systemd-journald.service: main process exited, code=dumped, status=6/ABRT
Aug  4 00:43:49 localhost systemd[1]: systemd-journald.service stop-sigabrt timed out. Terminating.
Aug  4 00:42:58 localhost journal: Permanent journal is using 2.1G (max allowed 4.0G, trying to leave 4.0G free of 215.1M available → 
current limit 2.1G).

is this disk space greediness relevant?

Comment 14 Jan Synacek 2016-01-20 13:10:30 UTC

*** This bug has been marked as a duplicate of bug 1300212 ***