Bug 1036720

Summary:	systemd-journald causes massive slowdown of filesystem operations
Product:	[Fedora] Fedora	Reporter:	Ali Akcaagac <aliakc>
Component:	systemd	Assignee:	systemd-maint
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	20	CC:	johannbg, lnykryn, mschmidt, msekleta, plautrba, systemd-maint, vpavlin, zbyszek
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-05 14:13:59 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Ali Akcaagac 2013-12-02 14:07:31 UTC

I just switched from Fedora 18 to Fedora 20 (skipped Fedora 19) and found out that operations with files (read or write) started to crawl on my system. After some investigations I figured out that systemd-journald is the cause of the problem. Afer entirely disabling systemd-journald the system performs quite similar to Fedora 18.

Sadly this is a huge issue. Why ?

I run Fedora since Fedora 16 on a RAGE XT 64gb USB Stick. This is one of the fastest USB Sticks around for USB2.0 and even performs faster on USB 3.0. I receive nearly 26mb on larger files and around 15mb with smaller files. So never had any issues so far.

When updating to Fedora 20 I realized that everything. Regardless what: Everything started to crawl. Yum updates slow. Playing movies usually get out of sync and and and.

So I investigated what might cause the problems and figured out that journald seems to cause something to the filesystem that causes - specially on usb sticks - the file operating to crawl. Maybe this is paid no attention by users running it on Harddisks or SDD Disks but with a USB disk everything crawls.

systemctl disable systemd-journald
systemctl disable systemd-journald.socket
systemctl stop systemd-journald
systemctl stop systemd-journald.socket

and a verification by using ps -A | grep -i "journal" ensured that the process is removed and not running anymore. Now the overall file operation is normal again.

You might investigate into this issue before final Fedora 20 is released.

Comment 1 Michal Schmidt 2013-12-05 09:38:06 UTC

Is there some process running that produces lots of logs? That's the only way I can imagine systemd-journald could keep the filesystem busy.
Have you looked at what was being stored in the journal before you disabled it?

Comment 2 Ali Akcaagac 2013-12-05 14:00:48 UTC

I can't really say whether there is a process which causes a lot of ouput but I must admit that I returned back to Fedora 18 because of quite some other annoying issues that I also detected with Fedora 20.

But I can tell for sure that the /var/log/journald directory (or how's it called) had some binary blobs inside with roughly 37-39 mb of size. Quite a lot for my taste for just 1-2 hour of "experimenting".

Comment 3 Michal Schmidt 2013-12-05 14:08:01 UTC

(In reply to Ali Akcaagac from comment #2)
> But I can tell for sure that the /var/log/journald directory (or how's it
> called) had some binary blobs inside with roughly 37-39 mb of size. Quite a
> lot for my taste for just 1-2 hour of "experimenting".

That seems to support the hypothesis that some excessive logging was going on.

Comment 4 Michal Schmidt 2013-12-05 14:13:59 UTC

(In reply to Ali Akcaagac from comment #2)
> I can't really say whether there is a process which causes a lot of ouput
> but I must admit that I returned back to Fedora 18 because of quite some
> other annoying issues that I also detected with Fedora 20.

OK, closing as INSUFFICIENT_DATA then. Anyone feel free to reopen if you can reproduce.