Bug 1236422

Summary: Certain "abrt-action-generate-core-backtrace" processes uses, each, 100% of 1 CPU core and takes a while to finish
Product: [Fedora] Fedora Reporter: Diogo Campos <diogocamposwd>
Component: abrtAssignee: abrt <abrt-devel-list>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: abrt-devel-list, diogocamposwd, dvlasenk, eraygezer.94, iprikryl, jfilak, mhabrnal, michal.toman, mmilata
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: abrt-2.6.1-2.fc22 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-30 00:50:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Diogo Campos 2015-06-28 22:17:15 UTC
# Description of problem:

--> First of all: I have a "Pentium B950 (Mobile, SandyBridge)" and it is underclocked because of problems with overheating.

The problem is that, frequently, something (probably a crash with "webkitgtk4", I think) starts one or two "abrt-action-generate-core-backtrace" processes which uses, each one, 100% of one CPU core. And seems to never stop.

The result, as you can guess, is a unusable machine, until I manually kill the said processes. And, to be fair, I never waited more than 2 minutes (that I remember) before killing the processes. Because reasons.

# Version-Release number of selected component (if applicable):

abrt-addon-ccpp-2.6.0-1.fc22.x86_64

# How reproducible:

Always that something triggers "abrt-action-generate-core-backtrace".

# Steps to Reproduce:

1. Just trigger "abrt-action-generate-core-backtrace", I think.

# Actual results:

Each "abrt-action-generate-core-backtrace" process uses 100% of each CPU core.

# Expected results:

Should it be easy on CPU, or should it end quickly.
A person who don't know what's happening, when this to happen, simply will conclude that Fedora is *scarily* slow.

# Additional info:
There is some other information I can give?
There is something I can do to prevent this, besides removing ABRT?

Comment 1 Diogo Campos 2015-06-29 04:37:21 UTC
So, this time I tried to wait for this thing to finish, and it, indeed, finished; but just after 40 minutes of intense processing.

Last time I looked, it was using 250MB+ of RAM.

It seems to always happen because a crash with "webkitgtk4", the trigger seems to be a close of a (long time opened - 1~2+ hours) GNOME Web's "WebApp".

This is the crash: https://retrace.fedoraproject.org/faf/reports/bthash/e40663c3f76810b827e59250c34621f236788115

Comment 2 Jakub Filak 2015-06-29 12:57:41 UTC
(In reply to Diogo Campos from comment #1)
> So, this time I tried to wait for this thing to finish, and it, indeed,
> finished; but just after 40 minutes of intense processing.
> 
> Last time I looked, it was using 250MB+ of RAM.
> 
> It seems to always happen because a crash with "webkitgtk4", the trigger
> seems to be a close of a (long time opened - 1~2+ hours) GNOME Web's
> "WebApp".
> 
> This is the crash:
> https://retrace.fedoraproject.org/faf/reports/bthash/
> e40663c3f76810b827e59250c34621f236788115

Thank you for the report! Would you mind sharing the coredump? Or is it possible to reproduce the crash?

Comment 3 Diogo Campos 2015-06-29 20:14:58 UTC
I reported to Bugzilla through GNOME ABRT and selected to include the coredump.

The bad news is that seems that GA just ignored my selection, and do not uploaded the coredump together.

The good news is that the offending crash shouldn't happen anymore in the next release of "webkitgtk4".

This is the bug: https://bugzilla.redhat.com/show_bug.cgi?id=1236587

---

I don't know if this is:

1- A bad behaviour of "abrt-ccpp".
2- A weird crash from "webkitgtk4".
3- Not a bug. Just a "non supported processor slowness".
4- None of the above.

But sure is a (really) bad experience :(

So, if there is still interest in seeing the coredump, sure I can manually upload it. Just tell me.

Comment 4 Jakub Filak 2015-06-30 15:45:35 UTC
Sure, we want to learn more about your issue and eventually fix the root cause of it, so please upload the core dump.

Comment 5 Diogo Campos 2015-06-30 19:50:07 UTC
:)

https://drive.google.com/file/d/0B7DZ4M94xvduZ1BqQ2NsaWczX1U/view?usp=sharing

(it's on GDrive because it's bigger than the attachment size limit)

Comment 6 Diogo Campos 2015-07-03 20:17:12 UTC
I did a "systemctl disable abrt-ccpp" to get rid of this annoyance.
Turns out that seems that the same crash starts the process "systemd-coredump", who behaves the same way :P

Comment 7 Jakub Filak 2015-07-06 05:23:45 UTC
I opened the coredump and I found that /usr/libexec/webkit2gtk-4.0/WebKitWebProcess fell into an infinite recursion which causes all unwinders to consume 100% of your CPU. ABRT (satyr) tries to find the beginning of the loop and report only first 256 frames because the top frames are useless. Unfortunately, there is not much we can do about it. We need to go through all frames, so we can only try to decrease scheduling priority of "abrt-action-generate-core-backtrace".

Comment 8 Diogo Campos 2015-07-06 15:13:54 UTC
Thank you a lot for looking, Jakub.

And don't worry, as seen, the crash should not happen anymore in the next release of WebKitGtk4 (and I can handle this thing for now, of course).

Anyway, not sure if I understood everything, so, I will ask, just in case: a "coredump" generated by "systemd-coredump" (or any other information) can be useful for something? Because I kinda can reproduce the crash, and provide the whatever thing.

If not, can I close the bug, then?

Comment 9 Jakub Filak 2015-07-07 07:21:05 UTC
(In reply to Diogo Campos from comment #8)
> Anyway, not sure if I understood everything, so, I will ask, just in case: a
> "coredump" generated by "systemd-coredump" (or any other information) can be
> useful for something? Because I kinda can reproduce the crash, and provide
> the whatever thing.
> 
> If not, can I close the bug, then?

Thank you for taking the care and time! In general, a core dump file might be useful, for example, if a maintainer solving a crash needs to get more information than ABRT attaches to its bug reports (e.g. values of global variables). In the case of the crash of WebKitGtk4, the maintainer had everything to solve the bug, so you can remove everything related to the crash.

I am not closing this bug yet, because we should add a commit like this:
diff --git a/src/plugins/ccpp_event.conf b/src/plugins/ccpp_event.conf
index d5a8db2..8551474 100644
--- a/src/plugins/ccpp_event.conf
+++ b/src/plugins/ccpp_event.conf
@@ -16,7 +16,7 @@ EVENT=post-create type=CCpp
         fi
         # Try generating backtrace, if it fails we can still use
         # the hash generated by abrt-action-analyze-c
-        [ ! -e core_backtrace ] && abrt-action-generate-core-backtrace
+        [ ! -e core_backtrace ] && (nice abrt-action-generate-core-backtrace)
         # Run GDB plugin to see if crash looks exploitable
         [ -r coredump ] && abrt-action-analyze-vulnerability
         # Generate hash


Could you please re-enable abrt-ccpp, replace abrt-action-generate-core-backtrace with (nice abrt-action-generate-core-backtrace) in /etc/libreport/events.d/ccpp_event.conf and reproduce the crash? It will take more time to finish the processing but it should not consume too much resources and your machine should be more reactive.

Comment 10 Diogo Campos 2015-07-07 18:24:16 UTC
I tried to do a "sandbox testing" with the two options, in two different boots, with a workflow consisting of Geary, Liferea, webpages on GNOME Web, online videos on Firefox, and constantly monitoring of numbers in GNOME System Monitor.

In the 2 runs, "abrt-action-generate-core-backtrace" used mostly 50% of CPU while active, was using 3XX MB of RAM near to the end, and took around 60 minutes to finish.

The perceived technical difference was that, in the run "with nice", the CPU usage of "abrt-action-generate-core-backtrace" easily dropped to around 25%; instead of hardly dropping below 40% in the run "without nice".

The perceived visual difference was that, in the run "with nice", the whole UI was "smooth"; while in the run "without nice" the UI "stutters" are frequent (especially notable on GNOME Shell's transitions).

So, for me, this change is a "go".

Keep in mind, however, that this machine is severely underclocked (from 2.1 GHz, to 0.8 GHz). If this means that the whole case is, then, highly relevant or highly irrelevant, is up to you, of course.

Comment 11 Jakub Filak 2015-07-15 10:50:51 UTC
I have opened an upstream pull request that makes entire abrtd post-processing less resource demanding:
https://github.com/abrt/abrt/pull/1012

Comment 12 Fedora Update System 2015-07-21 16:53:00 UTC
abrt-2.6.1-2.fc22,libreport-2.6.2-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/abrt-2.6.1-2.fc22,libreport-2.6.2-1.fc22

Comment 13 Fedora Update System 2015-07-29 01:41:49 UTC
Package abrt-2.6.1-2.fc22, libreport-2.6.2-1.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing abrt-2.6.1-2.fc22 libreport-2.6.2-1.fc22'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-11974/abrt-2.6.1-2.fc22,libreport-2.6.2-1.fc22
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2015-07-30 00:50:08 UTC
abrt-2.6.1-2.fc22, libreport-2.6.2-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Eray Erdin 2022-10-03 14:07:47 UTC
I'm experiencing this issue with Fedora 36.

It is related to [this issue](https://bugs.kde.org/show_bug.cgi?id=459838) in KDE.

When the desktop crashes, `abrt-action-generate-core-backtrace` process launches.

After a while, process is steadily using %25 percent of CPU. My machine is 4-core 4-thread. So, it's actually using a core %100 percent of a single core.

The process does not finish. The memory starts from 20 mb. After running for 10 minutes it has raised to 64 mb, which indicates there's a memory leak.

Comment 16 Eray Erdin 2022-10-03 14:08:42 UTC
(In reply to Eray Erdin from comment #15)
> I'm experiencing this issue with Fedora 36.
> 
> It is related to [this issue](https://bugs.kde.org/show_bug.cgi?id=459838)
> in KDE.
> 
> When the desktop crashes, `abrt-action-generate-core-backtrace` process
> launches.
> 
> After a while, process is steadily using %25 percent of CPU. My machine is
> 4-core 4-thread. So, it's actually using a core %100 percent of a single
> core.
> 
> The process does not finish. The memory starts from 20 mb. After running for
> 10 minutes it has raised to 64 mb, which indicates there's a memory leak.

My system information:

```
Operating System: Fedora Linux 36
KDE Plasma Version: 5.25.5
KDE Frameworks Version: 5.98.0
Qt Version: 5.15.6
Kernel Version: 5.19.12-200.fc36.x86_64 (64-bit)
Graphics Platform: X11
Processors: 4 × Intel® Core™ i5-2400 CPU @ 3.10GHz
Memory: 15,6 GiB of RAM
Graphics Processor: AMD OLAND
```