Bug 183898

I'm seeing similar but worse issues: CPU usage goes to 100% but for periods of
about 8 seconds within every 10 second period; looking at top shows mono to  the
offender.

"ps ax | grep mono" shows it to be the index helper:
 1561 ?        Sl    32:35 mono-beagled-helper --debug
/usr/lib/beagle/IndexHelper.exe

This is with rawhide, beagle-0.2.1-12

(am uninstalling beagle to make machine usable, alas)

*** Bug 183958 has been marked as a duplicate of this bug. ***

This one seems like a pretty bad bug.  Unfortunately, we may not get it fixed in
time for FC5, so I've turned beagle off by default.

Hi Dennis,

Just out of curiosity, what kind of sound card do you own?

Also, we're probably going to push beagle 0.2.3 to updates-testing later today
or tomorrow.  It fixes a large memory leak which may also be the cause of this
problem (because of the garbage collector).

I'm using a M-Audio Audiophile 24/96 card (which uses the ice1712 driver).

I see these audio hiccups whenever I put some pressure on the machine, usually
installing a large rpm package is enough to trigger this.

(In reply to comment #5)
> Also, we're probably going to push beagle 0.2.3 to updates-testing later today
> or tomorrow.  It fixes a large memory leak which may also be the cause of this
> problem (because of the garbage collector).

I'm using 0.2.3 from the updates (no longer testing I guess). My beagled often
eats up several hundred megabytes leaving the machine almost unusable until it
is killed.

Is this (the memory consumption) still a problem with 0.2.4 ?

(In reply to comment #8)
> Is this (the memory consumption) still a problem with 0.2.4 ?

Apparently, it is.  

[brad@AVL-kittreb .beagle]$ rpm -qa | grep beagle
libbeagle-0.2.4-1.fc5.1
beagle-0.2.4-1.fc5.1

Here's the details from top:

top - 08:15:19 up 17:58,  2 users,  load average: 1.90, 1.66, 1.32
Tasks: 128 total,   1 running, 126 sleeping,   0 stopped,   1 zombie
Cpu(s): 50.7% us,  0.2% sy,  0.0% ni, 49.2% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:   1031004k total,  1007708k used,    23296k free,     1672k buffers
Swap:  2096472k total,  2096468k used,        4k free,    52532k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4640 brad      16   0 2782m 758m 2832 S  100 75.3 890:50.78 beagled-helper
 2499 root      15   0 65556  27m 4428 S    1  2.7  20:23.25 Xorg
13124 brad      15   0 38460 9508 5492 S    0  0.9   0:02.49 gnome-terminal
13333 brad      16   0  2128 1048  796 R    0  0.1   0:00.21 top

The problem is still afoot.

Beagle was updated to version 2.5 via yum, and the problem still persisted.  I
removed .beagle/ (rm -rf .beagle/) and restarted beagle, and now the system is
behaving apropriately.

I'm using 0.2.6 from updates, and the option "--debug" is still hardwired in all
of the beagle scripts.
If I remove it, cpu and memory usage drop down to somewhat acceptable levels,
for me.

I'm removing the --debug flags in rawhide.

>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  4640 brad      16   0 2782m 758m 2832 S  100 75.3 890:50.78 beagled-helper

This implies that either (a) some file is causing problems with a filter and it
is eating all memory or (b) that some catastrophic exception is happening and
(probably) looping infinitely.

In general, beagled-helper will restart itself after it passes some memory
threshold.  Check (or attach) the IndexHelper log from ~/.beagle/Log to see
which it is.

At least in my case, I get empty logs and beagle eating 100% CPU.
(Though lucky for me, I've got enough CPU cores to go around ;)).

Restarting beagle doesn't seem to help. I need to delete the .beagle directory
and start from scratch.

Oh, I'm running under kde, if it means anything.

You may need to run beagled with the --debug option.  Unfortunately a patch
currently in FC5 (although being fixed in an errata) causes the Beagle log
itself to always be empty, regardless of --debug.

Brief periods of CPU usage are to be expected -- it does have to read and
process all of your data after all :) -- but extended periods are bugs.  It
doesn't matter that you're running KDE (although that might help track things
down), but it is important to note which process (beagled or beagled-helper) is
actually eating the CPU.

At least in my case, beagled is the CPU eater. When I say CPU eater, I mean:
100% CPU for more then 10 minutes.
I'm not if it's relevant, just before all hell breaks lose, beagled-helper gets
marked as "defunct" (or zombie).

Oh... here's the weird part.
I've got two identical FC5/x86_64 workstations, one at home and other at work.
In both cases I index the same data (my home directory is synced daily between
both machines)
While I beagle spikes on my work machines, it doesn't rarely fails on my home
workstation.

Cheers,

(In reply to comment #16)
> At least in my case, beagled is the CPU eater. When I say CPU eater, I mean:
> 100% CPU for more then 10 minutes.

Yeah, this looks like a bug.  Once the errata is out, I'd like to see what your
beagled log looks like.

> I'm not if it's relevant, just before all hell breaks lose, beagled-helper gets
> marked as "defunct" (or zombie).

This was recently fixed in the upstream 0.2.7 release.

> Oh... here's the weird part.
> While I beagle spikes on my work machines, it doesn't rarely fails on my home
> workstation.

Most likely something happened to get your index into a funky state, and beagle
isn't liking something.  Half of the potential for bugs is the data itself.  The
other half is circumstance. :)

> Most likely something happened to get your index into a funky state, and beagle
> isn't liking something.  Half of the potential for bugs is the data itself.
> The other half is circumstance. :)

A couple of weeks ago I decided to delete the .beagle directory on both machines
just for the hell of it; while my private workstation still works just fine, my
work machine gone nuts after a week. Since then, it goes bonkers ~4-6 days after
each .beagle delete.

Did I mention that both machines are 99.5% identical. (The private machien has 2
x Opteron 270s, while my work machine has 2 x Opteron 275... if it means
anything...)
Should I call for a shaman?

(In reply to comment #18)

> A couple of weeks ago I decided to delete the .beagle directory on both machines
> just for the hell of it; while my private workstation still works just fine, my
> work machine gone nuts after a week. Since then, it goes bonkers ~4-6 days after
> each .beagle delete.
> 
> Did I mention that both machines are 99.5% identical. (The private machien has 2
> x Opteron 270s, while my work machine has 2 x Opteron 275... if it means
> anything...)
> Should I call for a shaman?
> 

For what I've seen so far, the file system does a lot of difference. A local
file system with xattr enabled is doing good, my home on an nfs file system
works well if I use static indexes, my laptop with encfs is a tragedy. I didn't
try static indexes on that.

Both machines index has SCSI based software RAID5, with /home sitting on
separate LVM partition with xattr activated...

It's pure magic, I tell you! pure magic! ;)

argh... I mean, both machine index my home, which is sitting on a SCSI... well,
you get the rest.

No go.
Latest -testing beagle doesn't solve the problem.
I'm now running beagled with --debug (which works just fine, thanks). After the
last line in current-Beagle log, beagled continues to eats 100%, but no new
lines are added to the log.

Created attachment 131659 [details]
CPU hogging beagled-generated log file

... After looking at the logs, it seems that beagle goes bonkers in line 151:
149: 060628 1453196245 32498 Beagle DEBUG: Overall percent is 36
150: 060628 1453196245 32498 Beagle DEBUG: Inbox/Mailing/Linux-IL: indexed 0
messages
151: ---> 060628 1453201310 32498 Beagle DEBUG: Opening mbox Logs
152: 060628 1459475671 32498 Beagle DEBUG: Saw event in '/home/gilboa'
153: 060628 1459475673 32498 Beagle DEBUG: Saw event in '/home/gilboa'

Looks probably like a bug in the KMail backend.  Do you have a mbox named "Logs"
on your system?  It would probably be in your ~/Mail directory.  How big is that
file?  Is it really a valid mbox?

Ummm... I use evolution, but I once tried to import my evolution mboxes to
kmail... Seems that this is the source of the problem.
I'll delete the kmail configuration/mboxes and start over.

Thanks!

This bug seems to be overused for a number of bugreports. Hopefully things is
faster now with the latest rawhide version. If you have something more specific
to report, open a new bug.