Bug 216724 - Subversion 1.4.2 freeze on FC6
Subversion 1.4.2 freeze on FC6
Status: CLOSED WORKSFORME
Product: Fedora
Classification: Fedora
Component: subversion (Show other bugs)
6
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Joe Orton
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-11-21 13:13 EST by Stéphane Becker
Modified: 2007-11-30 17:11 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-04-26 11:30:10 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Stéphane Becker 2006-11-21 13:13:58 EST
Description of problem: I have a long history of running SVN on a fedora server
since FC3. I use the last stable version directly downloaded from SVN site that
I hand compile on my server. I am running the svnserve daemon to access the
subversion repository, this repository weight around 50 GB.

Until FC6 everything was working just fine but since I migrated on FC6 I've got
a problem which occurs once per day quite randomly. From a client perspective it
seems that the server froze, on the server I can see the svnserve processus
eating a lot of cpu cycle. At first I let the process run just to see if it was
only taking some times to complete, but this is not the case. When this happens
I must kill the process, do a recover on the db (quite fast ; a couple of
seconds) and then relaunch the process. After that I just do the update on the
client machine and everything is ok until the next freeze.

It seems that the problem occurs more frequently every morning as several
employee try to get the last version from the svn server.

Version-Release number of selected component (if applicable): SVN v1.4.2 ; every
other component are coming from the fedora install.


How reproducible:


Steps to Reproduce:
1. Try to do an update
2. SVN process eat a lot of ram and does not accomplish its task.
  
Actual results:

svnserve does not return the data and must be killed.

Expected results:

svnserve returns my data.

Additional info:

The server is an Athlon 64 3000+ on a socket 754 motherboard with 1gb of ram
running the i386 version of FC6.
Comment 1 Joe Orton 2006-11-21 13:36:17 EST
Please try using the 1.4.2 packages which were issued to the FC6 updates.  If
you're not using packaged software there's little we can do to help you here.

If this is still reproducible we need a exact reproduction recipe.   Exactly
what command caused svn to consume memory?  What was consuming memory, the "svn"
client or "svnserve"?  What repository backend is being used?  What state was
the working copy in? etc
Comment 2 Stéphane Becker 2006-11-21 15:34:00 EST
I just installed the 1.4.2 package from the FC6 Update, I will keep you informed
if it solved the problem.

The process who consume the memory is svnserve, the backend used is berkeleydb.
For the working copy state it seems that it does not matter (tried with
different states).
Comment 3 Stéphane Becker 2006-11-24 05:16:04 EST
Even with the package I've got the same problem. I can see several svnserve
process, one very high on cpu usage and from a user perspective the server froze.

- Every user is using tortoiseSVN.
- Problems happens when people update their local copy.
- Seems to be tied to concurrent access (as it happens most often in the morning
when everybody do update).
- When I kill the process, tortoise tell me that "the connection was closed
unexpectedly".

I just did a "yum update" this morning to see if there was update and as it says
that around 150 packages needed update I will do it now and checked if the
problem is still there.

Normally they are not any custom compiled package anymore on this server
(subversion was the only one).
Comment 4 Joe Orton 2006-11-24 07:48:50 EST
OK.  Are you running svnserve in --daemon mode, via xinetd, or via ssh?  Can you
give the exact command used to invoke svnserv.

Can you try to generate a core dump from the svnserve process when it is
consuming lots of RAM?  Process would be as follows:

1. set "ulimit -c unlimited" from the shell before starting svnserve
2. when you see a "svnserve" consumes RAM, send a SEGV to its pid:

      kill -SEGV <pid-of-svnserve>

There should be a core dump in / - if you don't mind uploading this (it may
reveal contents of your repository), bzip2 it and attach it to this bug report -
check the "private" box to keep it non-public.

Otherwise, generate a backtrace as follows:

  yum --enablerepo=updates-debuginfo install subversion-debuginfo \
     apr-debuginfo apr-util-debuginfo httpd-debuginfo glibc-debuginfo \
     db4-debuginfo
  gdb /usr/bin/svnserve /core.<pid>
  ...
  (gdb) bt

and attach the output of that command.
Comment 5 Stéphane Becker 2006-12-04 05:02:28 EST
The exact command to launch svnserve is :
sudo -u apache svnserve -d -r /path_to_repository

(well we sudo as apache user as we are running trac on the same computer with
access to this repository, but my problem existed before the installation of trac).

I just tried the "kill -SEGV" this morning, but my first interrogation was that
it seems that svnserve spawned several processes, so I sent the kill message to
the one consumming all my cpu time. But this did not produce a core dump,
certainly because I did not have the ulimit command launched prior to svnserve.
This is the case now, so I will keep you informed when I collect more data.
Comment 6 Joe Orton 2007-04-26 10:48:13 EDT
Is this reproducible in the 1.4.3 packages now in the updates?
Comment 7 Stéphane Becker 2007-04-26 11:28:37 EDT
I would be glad to answer but since then I moved the database on another disk on
the same server and it seems that this solved the problem.
Comment 8 Joe Orton 2007-04-26 11:30:10 EDT
No problem, let's presume this was some lower-level issue then.
Comment 9 Stéphane Becker 2007-04-26 12:27:15 EDT
From what I've read when I was searching the net it seems to be a "low level
issue with BerkeleyDB which explain why you should prefer the filesystem version".

Note You need to log in before you can comment on or make changes to this bug.