Bug 183898 - beagle causing cpu usage spikes
beagle causing cpu usage spikes
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: beagle (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Alexander Larsson
:
: 183958 (view as bug list)
Depends On:
Blocks: FC5Target
  Show dependency treegraph
 
Reported: 2006-03-03 10:19 EST by Dennis Jacobfeuerborn
Modified: 2007-11-30 17:11 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-08-11 08:42:47 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
CPU hogging beagled-generated log file (13.18 KB, text/plain)
2006-06-28 07:54 EDT, Gilboa Davara
no flags Details

  None (edit)
Description Dennis Jacobfeuerborn 2006-03-03 10:19:31 EST
For a while now I notice audio hiccups every few minutes when playing back video
and now I identified beagle as the source of the problem. "ps ax" reveals that
the mono-beagled-helper uses quite a bit of cpu time and a "top" shows mono
appearing at the top every few seconds eating 80% of the cpu.
Comment 1 Dave Malcolm 2006-03-07 19:38:00 EST
I'm seeing similar but worse issues: CPU usage goes to 100% but for periods of
about 8 seconds within every 10 second period; looking at top shows mono to  the
offender.

"ps ax | grep mono" shows it to be the index helper:
 1561 ?        Sl    32:35 mono-beagled-helper --debug
/usr/lib/beagle/IndexHelper.exe

This is with rawhide, beagle-0.2.1-12

(am uninstalling beagle to make machine usable, alas)
Comment 2 Rahul Sundaram 2006-03-08 09:18:15 EST
*** Bug 183958 has been marked as a duplicate of this bug. ***
Comment 3 Ray Strode [halfline] 2006-03-08 13:22:09 EST
This one seems like a pretty bad bug.  Unfortunately, we may not get it fixed in
time for FC5, so I've turned beagle off by default.
Comment 4 Ray Strode [halfline] 2006-03-20 03:29:30 EST
Hi Dennis,

Just out of curiosity, what kind of sound card do you own?
Comment 5 Ray Strode [halfline] 2006-03-20 03:40:47 EST
Also, we're probably going to push beagle 0.2.3 to updates-testing later today
or tomorrow.  It fixes a large memory leak which may also be the cause of this
problem (because of the garbage collector).
Comment 6 Dennis Jacobfeuerborn 2006-03-20 07:28:39 EST
I'm using a M-Audio Audiophile 24/96 card (which uses the ice1712 driver).

I see these audio hiccups whenever I put some pressure on the machine, usually
installing a large rpm package is enough to trigger this.
Comment 7 Daniel Thompson 2006-03-25 17:17:24 EST
(In reply to comment #5)
> Also, we're probably going to push beagle 0.2.3 to updates-testing later today
> or tomorrow.  It fixes a large memory leak which may also be the cause of this
> problem (because of the garbage collector).

I'm using 0.2.3 from the updates (no longer testing I guess). My beagled often
eats up several hundred megabytes leaving the machine almost unusable until it
is killed.

Comment 8 Matthias Clasen 2006-04-18 23:23:02 EDT
Is this (the memory consumption) still a problem with 0.2.4 ?
Comment 9 Brad Kittredge 2006-04-20 11:19:32 EDT
(In reply to comment #8)
> Is this (the memory consumption) still a problem with 0.2.4 ?

Apparently, it is.  

[brad@AVL-kittreb .beagle]$ rpm -qa | grep beagle
libbeagle-0.2.4-1.fc5.1
beagle-0.2.4-1.fc5.1

Here's the details from top:

top - 08:15:19 up 17:58,  2 users,  load average: 1.90, 1.66, 1.32
Tasks: 128 total,   1 running, 126 sleeping,   0 stopped,   1 zombie
Cpu(s): 50.7% us,  0.2% sy,  0.0% ni, 49.2% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:   1031004k total,  1007708k used,    23296k free,     1672k buffers
Swap:  2096472k total,  2096468k used,        4k free,    52532k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4640 brad      16   0 2782m 758m 2832 S  100 75.3 890:50.78 beagled-helper
 2499 root      15   0 65556  27m 4428 S    1  2.7  20:23.25 Xorg
13124 brad      15   0 38460 9508 5492 S    0  0.9   0:02.49 gnome-terminal
13333 brad      16   0  2128 1048  796 R    0  0.1   0:00.21 top

The problem is still afoot.
Comment 10 Brad Kittredge 2006-04-24 12:50:48 EDT
Beagle was updated to version 2.5 via yum, and the problem still persisted.  I
removed .beagle/ (rm -rf .beagle/) and restarted beagle, and now the system is
behaving apropriately.
Comment 11 Andrea Dell'Amico 2006-06-01 07:58:14 EDT
I'm using 0.2.6 from updates, and the option "--debug" is still hardwired in all
of the beagle scripts.
If I remove it, cpu and memory usage drop down to somewhat acceptable levels,
for me.
Comment 12 Alexander Larsson 2006-06-21 05:25:08 EDT
I'm removing the --debug flags in rawhide.
Comment 13 Joe Shaw 2006-06-21 11:21:24 EDT
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  4640 brad      16   0 2782m 758m 2832 S  100 75.3 890:50.78 beagled-helper

This implies that either (a) some file is causing problems with a filter and it
is eating all memory or (b) that some catastrophic exception is happening and
(probably) looping infinitely.

In general, beagled-helper will restart itself after it passes some memory
threshold.  Check (or attach) the IndexHelper log from ~/.beagle/Log to see
which it is.
Comment 14 Gilboa Davara 2006-06-21 11:43:27 EDT
At least in my case, I get empty logs and beagle eating 100% CPU.
(Though lucky for me, I've got enough CPU cores to go around ;)).

Restarting beagle doesn't seem to help. I need to delete the .beagle directory
and start from scratch.

Oh, I'm running under kde, if it means anything.
Comment 15 Joe Shaw 2006-06-21 12:38:47 EDT
You may need to run beagled with the --debug option.  Unfortunately a patch
currently in FC5 (although being fixed in an errata) causes the Beagle log
itself to always be empty, regardless of --debug.

Brief periods of CPU usage are to be expected -- it does have to read and
process all of your data after all :) -- but extended periods are bugs.  It
doesn't matter that you're running KDE (although that might help track things
down), but it is important to note which process (beagled or beagled-helper) is
actually eating the CPU.
Comment 16 Gilboa Davara 2006-06-21 12:52:33 EDT
At least in my case, beagled is the CPU eater. When I say CPU eater, I mean:
100% CPU for more then 10 minutes.
I'm not if it's relevant, just before all hell breaks lose, beagled-helper gets
marked as "defunct" (or zombie).

Oh... here's the weird part.
I've got two identical FC5/x86_64 workstations, one at home and other at work.
In both cases I index the same data (my home directory is synced daily between
both machines)
While I beagle spikes on my work machines, it doesn't rarely fails on my home
workstation.

Cheers,
Comment 17 Joe Shaw 2006-06-21 13:21:36 EDT
(In reply to comment #16)
> At least in my case, beagled is the CPU eater. When I say CPU eater, I mean:
> 100% CPU for more then 10 minutes.

Yeah, this looks like a bug.  Once the errata is out, I'd like to see what your
beagled log looks like.

> I'm not if it's relevant, just before all hell breaks lose, beagled-helper gets
> marked as "defunct" (or zombie).

This was recently fixed in the upstream 0.2.7 release.

> Oh... here's the weird part.
> While I beagle spikes on my work machines, it doesn't rarely fails on my home
> workstation.

Most likely something happened to get your index into a funky state, and beagle
isn't liking something.  Half of the potential for bugs is the data itself.  The
other half is circumstance. :)
Comment 18 Gilboa Davara 2006-06-21 13:30:40 EDT
> Most likely something happened to get your index into a funky state, and beagle
> isn't liking something.  Half of the potential for bugs is the data itself.
> The other half is circumstance. :)

A couple of weeks ago I decided to delete the .beagle directory on both machines
just for the hell of it; while my private workstation still works just fine, my
work machine gone nuts after a week. Since then, it goes bonkers ~4-6 days after
each .beagle delete.

Did I mention that both machines are 99.5% identical. (The private machien has 2
x Opteron 270s, while my work machine has 2 x Opteron 275... if it means
anything...)
Should I call for a shaman?
Comment 19 Andrea Dell'Amico 2006-06-21 13:44:16 EDT
(In reply to comment #18)

> A couple of weeks ago I decided to delete the .beagle directory on both machines
> just for the hell of it; while my private workstation still works just fine, my
> work machine gone nuts after a week. Since then, it goes bonkers ~4-6 days after
> each .beagle delete.
> 
> Did I mention that both machines are 99.5% identical. (The private machien has 2
> x Opteron 270s, while my work machine has 2 x Opteron 275... if it means
> anything...)
> Should I call for a shaman?
> 

For what I've seen so far, the file system does a lot of difference. A local
file system with xattr enabled is doing good, my home on an nfs file system
works well if I use static indexes, my laptop with encfs is a tragedy. I didn't
try static indexes on that.
Comment 20 Gilboa Davara 2006-06-21 14:12:04 EDT
Both machines index has SCSI based software RAID5, with /home sitting on
separate LVM partition with xattr activated...

It's pure magic, I tell you! pure magic! ;)
Comment 21 Gilboa Davara 2006-06-21 14:13:46 EDT
argh... I mean, both machine index my home, which is sitting on a SCSI... well,
you get the rest.
Comment 22 Gilboa Davara 2006-06-28 07:51:38 EDT
No go.
Latest -testing beagle doesn't solve the problem.
I'm now running beagled with --debug (which works just fine, thanks). After the
last line in current-Beagle log, beagled continues to eats 100%, but no new
lines are added to the log.

Comment 23 Gilboa Davara 2006-06-28 07:54:19 EDT
Created attachment 131659 [details]
CPU hogging beagled-generated log file
Comment 24 Gilboa Davara 2006-06-28 07:59:01 EDT
... After looking at the logs, it seems that beagle goes bonkers in line 151:
149: 060628 1453196245 32498 Beagle DEBUG: Overall percent is 36
150: 060628 1453196245 32498 Beagle DEBUG: Inbox/Mailing/Linux-IL: indexed 0
messages
151: ---> 060628 1453201310 32498 Beagle DEBUG: Opening mbox Logs
152: 060628 1459475671 32498 Beagle DEBUG: Saw event in '/home/gilboa'
153: 060628 1459475673 32498 Beagle DEBUG: Saw event in '/home/gilboa'
Comment 25 Joe Shaw 2006-06-28 09:15:22 EDT
Looks probably like a bug in the KMail backend.  Do you have a mbox named "Logs"
on your system?  It would probably be in your ~/Mail directory.  How big is that
file?  Is it really a valid mbox?
Comment 26 Gilboa Davara 2006-06-28 10:39:02 EDT
Ummm... I use evolution, but I once tried to import my evolution mboxes to
kmail... Seems that this is the source of the problem.
I'll delete the kmail configuration/mboxes and start over.

Thanks!
Comment 27 Alexander Larsson 2006-08-11 08:42:47 EDT
This bug seems to be overused for a number of bugreports. Hopefully things is
faster now with the latest rawhide version. If you have something more specific
to report, open a new bug.

Note You need to log in before you can comment on or make changes to this bug.