Bug 70846 - mm complied with semaphores is a DOS waiting to happen
mm complied with semaphores is a DOS waiting to happen
Status: CLOSED WONTFIX
Product: Red Hat Linux
Classification: Retired
Component: php (Show other bugs)
7.2
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Joe Orton
Brian Brock
:
Depends On: 61030
Blocks:
  Show dependency treegraph
 
Reported: 2002-08-05 21:42 EDT by dharris
Modified: 2007-04-18 12:45 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-01-25 11:47:50 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description dharris 2002-08-05 21:42:02 EDT
Description of Problem:

With the latest libmm package mm-1.1.3-8 and the php-4.0.6-15 package, php 
leaks semaphores. Over time the number of semaphores in the system grows until 
it hits the limit (in /proc/sys/kernel/sem). Once all semaphores are used up, 
Apache will not start as it can not allocate semaphores.

I do not know if the problem is with php or with mm. Let me talk about this:

This release of the mm package is now compiled to use semaphores (previous mm's 
used shared files), so if a leak already existed in php, it is now a serious 
problem. Leaking semaphores is a big deal because the default configuration 
only allows 128 system-wide.

When I system trace php I see that the leaked semaphores are created (through 
the semget system call) immediately after the loading of shared libraries and 
before even the /etc/php.ini file is opened. And semget is called even when I 
run "php /dev/null", so I do not think the leaked semaphores are caused by an 
application using php's semaphore extension incorrectly. My application does 
not even directly use phps semaphore extension.

I think the problem is in either php or libmm.

Although, my application (vBulletin Version 2.2.4) is very heavy and may flex 
php in such a way that a semaphore leak is exposed. The application uses a lot 
of php features including gzipping the outgoing HTML. (This is a wonderful 
bandwidth saver!!)

How does the problem show itself:

Immediately after I upgraded to the latest eratta (including mm-1.1.3-8), 
semaphores started leaking. I didnt notice this until a few days later when I 
couldn't start Apache.

I used this command to see the semaphores and sort them by creation date:

cat /proc/sysvipc/sem | perl -ne ' chomp; print $_ . " " . ( /(\d+)$/ ? 
localtime($1) : "") . "\n"; ' | sort -k10 -n

I traced the semaphore identifiers back to php (which was a pain) until I had 
specific system trace's of PHP running for actual user requests that showed the 
semget system call that created the particular leaked semaphore ID.

New semaphore creation/leakage was really out of control with multiple 
semaphores leaking each minute:

./show_sem.sh | tail
         0  619151384   600          1    99    99    99    99 1028595216 
1028595216 Mon Aug  5 20:53:36 2002
         0  619184153   600          1    99    99    99    99          0 
1028595216 Mon Aug  5 20:53:36 2002
         0  619216922   600          1    99    99    99    99 1028595217 
1028595217 Mon Aug  5 20:53:37 2002
         0  619249691   600          1    99    99    99    99          0 
1028595217 Mon Aug  5 20:53:37 2002
         0  619282460   600          1    99    99    99    99 1028595218 
1028595218 Mon Aug  5 20:53:38 2002
         0  619315229   600          1    99    99    99    99          0 
1028595218 Mon Aug  5 20:53:38 2002
         0  619347998   600          1    99    99    99    99 1028595277 
1028595276 Mon Aug  5 20:54:36 2002
       0  619380767   600          1    99    99    99    99          0 
1028595276 Mon Aug  5 20:54:36 2002
         0  619413536   600          1    99    99    99    99 1028595279 
1028595278 Mon Aug  5 20:54:38 2002
         0  619446305   600          1    99    99    99    99          0 
1028595278 Mon Aug  5 20:54:38 2002

When I stopped PHP, semaphores stopped leaking. When I started PHP, they 
started leaking again, so I was sure it was php.

So, I went back to the mm-1.1.3-1.i386.rpm mm-devel-1.1.3-1.i386.rpm packages 
(via: rpm -Uh --oldpackage) and immediately the leakage of semaphores stopped.

A quick fix could be rebuilding mm to use shared files instead of SysVIPC 
semaphores for now.

Note: I do not know if other applications were leaking semaphores. I didn't 
trace all of the leaked semaphore identifiers. I did do one test: I started and 
stopped Apache 10 times using the new semaphore mm and no semaphores were 
leaked.

Version-Release number of selected component (if applicable):

mm-1.1.3-8
php-4.0.6-15
apache-1.3.22-6

My PHP application: vBulletin Version 2.2.4

If you need, I can provide more detailed configuration or any other information 
that may be helpful.
Comment 1 dharris 2002-08-05 21:44:27 EDT
One additional note worth pointing out that may clarify things: Both PHP and 
Apache dynamically link mm and use it for semaphore operations.
Comment 2 Need Real Name 2002-08-27 10:24:48 EDT
I have seen this problem as well. I have Apache alert me when it dies, and it 
seems to be dying hourly at its most frequent. This corresponds to the 
semaphore list I'm seeing:

       key      semid perms      nsems   uid   gid  cuid  cgid      otime      
ctime
         0      32769   600          1   500   500   500   500          0 
1030418965 Mon Aug 26 23:29:25 2002
         0      65538   600          1    99     0     0     0          0 
1030419002 Mon Aug 26 23:30:02 2002
         0      98307   600          1    99     0     0     0          0 
1030419002 Mon Aug 26 23:30:02 2002
         0     196612   600          1    99     0     0     0 1030421359 
1030419004 Mon Aug 26 23:30:04 2002
         0     229381   600          1    99     0     0     0 1030421359 
1030419004 Mon Aug 26 23:30:04 2002
         0     262150   600          1     0     0     0     0 1030421359 
1030419004 Mon Aug 26 23:30:04 2002
         0     294919   600          1     0     0     0     0          0 
1030419004 Mon Aug 26 23:30:04 2002
         0     327688   600          1    99    99     0     0 1030421714 
1030419005 Mon Aug 26 23:30:05 2002
         0     360457   600          1    99     0     0     0          0 
1030421361 Tue Aug 27 00:09:21 2002
         0     393226   600          1    99     0     0     0          0 
1030421361 Tue Aug 27 00:09:21 2002
         0     491531   600          1    99     0     0     0          0 
1030421714 Tue Aug 27 00:15:14 2002
         0     524300   600          1    99     0     0     0          0 
1030421714 Tue Aug 27 00:15:14 2002
         0     786445   600          1    99     0     0     0 1030449614 
1030435334 Tue Aug 27 04:02:14 2002
         0     819214   600          1    99     0     0     0 1030449614 
1030435334 Tue Aug 27 04:02:14 2002
         0     851983   600          1     0     0     0     0 1030449614 
1030435334 Tue Aug 27 04:02:14 2002
         0     884752   600          1     0     0     0     0          0 
1030435334 Tue Aug 27 04:02:14 2002
         0     917521   600          1    99    99     0     0 1030449618 
1030435334 Tue Aug 27 04:02:14 2002
         0     950290   600          1    99     0     0     0          0 
1030449619 Tue Aug 27 08:00:19 2002
         0     983059   600          1    99     0     0     0          0 
1030449619 Tue Aug 27 08:00:19 2002
         0    1015828   600          1     0     0     0     0 1030449620 
1030449620 Tue Aug 27 08:00:20 2002
         0    1048597   600          1     0     0     0     0          0 
1030449620 Tue Aug 27 08:00:20 2002
         0    1081366   600          1    99     0     0     0 1030454113 
1030449620 Tue Aug 27 08:00:20 2002
         0    1114135   600          1    99     0     0     0 1030454113 
1030449620 Tue Aug 27 08:00:20 2002
         0    1146904   600          1     0     0     0     0 1030454113 
1030449620 Tue Aug 27 08:00:20 2002
         0    1179673   600          1     0     0     0     0          0 
1030449620 Tue Aug 27 08:00:20 2002
         0    1212442   600          1    99    99     0     0 1030454118 
1030449620 Tue Aug 27 08:00:20 2002
         0    1245211   600          1    99     0     0     0          0 
1030454118 Tue Aug 27 09:15:18 2002
         0    1277980   600          1    99     0     0     0          0 
1030454118 Tue Aug 27 09:15:18 2002
         0    1310749   600          1     0     0     0     0 1030454119 
1030454119 Tue Aug 27 09:15:19 2002
         0    1343518   600          1     0     0     0     0          0 
1030454119 Tue Aug 27 09:15:19 2002
         0    1376287   600          1    99     0     0     0 1030456813 
1030454122 Tue Aug 27 09:15:22 2002
         0    1409056   600          1    99     0     0     0 1030456813 
1030454122 Tue Aug 27 09:15:22 2002
         0    1441825   600          1     0     0     0     0 1030456813 
1030454122 Tue Aug 27 09:15:22 2002
         0    1474594   600          1     0     0     0     0          0 
1030454122 Tue Aug 27 09:15:22 2002
         0    1507363   600          1    99    99     0     0 1030456818 
1030454122 Tue Aug 27 09:15:22 2002
         0    1540132   600          1    99     0     0     0          0 
1030456818 Tue Aug 27 10:00:18 2002
         0    1572901   600          1    99     0     0     0          0 
1030456818 Tue Aug 27 10:00:18 2002
         0          0   600          1    99   500   500   500          0 
1030456819 Tue Aug 27 10:00:19 2002
         0    1605670   600          1     0     0     0     0 1030456819 
1030456819 Tue Aug 27 10:00:19 2002
         0    1638439   600          1     0     0     0     0          0 
1030456819 Tue Aug 27 10:00:19 2002
         0    1671208   600          1    99     0     0     0 1030456819 
1030456819 Tue Aug 27 10:00:19 2002
         0    1703977   600          1    99     0     0     0 1030456819 
1030456819 Tue Aug 27 10:00:19 2002
         0    1736746   600          1     0     0     0     0 1030456819 
1030456819 Tue Aug 27 10:00:19 2002
         0    1769515   600          1     0     0     0     0          0 
1030456819 Tue Aug 27 10:00:19 2002
         0    1802284   600          1    99    99     0     0 1030457784 
1030456819 Tue Aug 27 10:00:19 2002

Btw, I am using all the same versions of php, apache, and libmm as above. 
However, I am running 7.3 as opposed to 7.2.
Comment 3 John Morrissey 2002-08-28 12:00:22 EDT
I'm seeing similar behavior under 7.1 and 7.3 with:

mm-1.1.3-8
apache-1.3.23-14
php-4.2.2-0 (that I snagged from Rawhide before it was backed out)

I'm not seeing any problems with normal operations, but PHP (and therefore
Apache itself) won't restart after running for several days. I'm also seeing
this with command-line PHP scripts I run - after a few days, they won't run
because they can't initialize the mm library (strace shows that it can't obtain
a semaphore).
Comment 4 dharris 2002-08-28 14:10:46 EDT
Now that this has been confirmed, I'm updating it to a higher priority.
Comment 5 John Morrissey 2002-08-29 13:25:01 EDT
Slight correction: I'm seeing this on *7.1* with:

mm-1.1.3-8
apache-1.3.22-5.7.1
php-4.0.6-14

and *7.3* with:

mm-1.1.3-8
apache-1.3.23-14
php-4.2.2-0
Comment 6 Michiel Toneman 2002-08-30 13:54:00 EDT
This has bitten me bigtime :(

A fix to avoid having to reboot is:

for i in `ipcs -s -c | perl -pe "s/([0-9]+).+/\1/"`; do ipcrm sem $i; done

This works for me, although this could wreak havoc on the system :-S

Hope this is resolved soon, I'm in trouble over this (all our hosting went down,
we had major outage :( )
Comment 7 dharris 2002-08-30 14:24:59 EDT
I've e-mailed nalin@redhat.com to get some attention for this bug, since it has 
been over three weeks without a reply from QA and people are getting bitten. 
(Nalin Dahyabhai is the most recent changelog author on the apache, mm, and php 
packages I'm using.)
Comment 8 Phil Copeland 2002-08-30 15:26:27 EDT
[root@alpha root]# ipcs > l
[root@alpha root]# rpm -q php
php-4.1.2-7.2.4
[root@alpha root]# php -v
4.1.2
[root@alpha root]# ipcs > ll
[root@alpha root]# diff l ll
[root@alpha root]#

*** This bug has been marked as a duplicate of 71097 ***
Comment 9 Michiel Toneman 2002-08-30 16:03:40 EDT
I'm sorry, but this is not closed by any means! 

The use of kernel semaphores by mm has just made a bad problem worse. The fact
that PHP leaks semaphores just makes the problem surface faster, but it is
definitely not the cause.  

As stated above, the system only has 128 semaphore arrays total, and each apache
running uses up 4 of those. Any user can now start an apache instance, making a
DOS attack trivial. The old version of apache+mm used files instead, which were
written to /var/run, making it impossible for a non-privileged users to run an
apache process, which is a bug in itself (see
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=61030 which was totally
ignored, b.t.w.).

As we run separate httpd's for each customer on the same box, this makes hosting
 more than 32 sites on a box impossible, which is WAY below the capabilities of
the hardware. 

Please fix this properly, it is a constant source of headaches. A proposed
simple fix is included in the bugreport I referred to above. This has worked in
our hosting setup for the past year.

Comment 10 Peter Bowen 2002-08-30 17:55:31 EDT
Reopening this bug and changing topic to refer to potential denial-of-service
issue.  Also upgrading to 'security' severity, because, as you point out, any
user and DOS the system and cause all kinds of fun problems.
Comment 11 Phil Copeland 2002-08-31 01:26:54 EDT
I can only speak for php
mm support was *removed* because mm is no longer actively maintained and is no
longer in our current product.

in the older php sets mm support was only enabled in AS, 7.2 and 7.3 and has
since been removed in errata queued up in QA that specifically addresses this issue

php-4.1.2-2.1.5 - Advanced Server
php-4.1.2-7.0.5 - 7.0
php-4.1.2-7.1.5 - 7.1
php-4.1.2-7.2.5 - 7.2
php-4.1.2-7.3.5 - 7.3

The current rawhide version is
php-4.2.2-8.0.1 - 8.0

I have no idea when QA will get around to testing these packages. it may be some
weeks but you are welcome to ping me privately for copies

Phil
=--=
Comment 12 Michiel Toneman 2002-08-31 05:56:32 EDT
As far as I can tell, the problem with mm used to be due to mod_ssl.

There is another (better) patch for this floating around the net by
Christian Jaeger <christian.jaeger@sl.ethz.ch> at:

http://pflanze.mine.nu/~chris/debian/apache/checkcreatemmdir.patch

This patch creates a safe mm.user directory for each user needing to run apache.
May be worth looking into.
Comment 13 Joe Orton 2002-09-26 05:27:26 EDT
I'm presuming this is a PHP bug until diagnosed otherwise.
Comment 14 Michiel Toneman 2002-09-26 06:23:34 EDT
Nope, a bug with PHP not releasing the semaphore array just worsens/triggers it.

The problem is using kernel semaphores in the first place, since they are a
severely limited resource that can't be tuned runtime (requires a kernel
recompile). 

At the moment one of our webservers is using 116 of the 128 semaphore arrays,
just because it is running a few instances of apache. Restarting an apache leaks
me another two arrays. In a day or two, I'll have to take down all the apache
processes, manually clean out the arrays with ipcrm and restart all the
apache's. This machine is budgetted to run at least 5 times as many apache
processes, which clearly isn't possible. 

So my options are:

1. Compile custom kernel
2. Compile custom apache+php
3. Wait for Red Hat to fix the problem

Options 1 and 2 I'd like to avoid, as the whole point of running a packaged
distribution like Red Hat is to avoid the hassle of manually compiling custom
packages.

I don't care if "null" fixes the problem. Red Hat 8 isn't even out yet, and
won't be running on our servers until 8.1 at least.  Please fix it, I'm getting
tired of living in fear.... :(

Comment 15 Joe Orton 2002-09-26 06:35:09 EDT
Can you file a separate bug against mm for "mm should not use semaphores by
default" then, and leave this one to cover "PHP leaks semaphores"?
Comment 16 Peter Bowen 2002-09-26 09:14:39 EDT
Between this bug, and Bug #61030 and Bug #71097, I think the apache issue is
well spelled out.  If you need a single new bug to track the requests that a)
apache/mm use shared files rather than semaphores and b) these shared files be
placed in a location such that a non-privledged use can exec apache, I will be
happy to open one.
Comment 17 Kevin M. Myer 2002-10-23 11:04:35 EDT
Compiling php with "--without-mm" seems to fix the problem (tested php 4.1.2
errata release on 7.3).  Does this have any negative implications for php, if
its not using mm?  Also tested php 4.3.0dev snapshot - it appears to be exhibit
the same problem there so I'd say its either a mm bug that php exposes or its a
bug in php with its mm routines.

Is the only thing I lose in PHP the ability to use mm sessions management in
place of files?  Since I think php.ini is defaulted to files to begin with, if
you don't need session.save_handler=mm, then this could be a potential solution
(short of upgrading to Red Hat 8.0).

The notes state that errata for PHP were in QA that don't have mm enabled (for
7.3 it would be php 4.1.2-7.3.5).  Whats the hold up on these?

As I see it, and as I think is spelled out here by others, there are really two
problems:  a bug in php that causes it to leak semaphores, which exposes a very
real DOS consequence caused by configuring mm to use semaphores.  The first bug
can be fixed by compiling php without the MM library.  The second can only be
mitigated by not using semaphores with mm.

Kevin (who's off to recompile PHP RPMs without mm and see what breaks in
production web-servers since its taking Red Hat 2 months to do this...)
Comment 18 Jon Benson 2002-10-24 21:06:55 EDT
It's pretty obvious there is an issue with apache and mm using semaphores 
unrelated to the php problem.  If you want a separate bug report try:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=74543

But PLEASE can someone at RedHat get this actioned.  Even the php errata 
mentioned in this bug report has not made it out.
Comment 19 Lars Povlsen 2002-11-11 08:22:54 EST
Hear-hear!

Is anybody giving this any attention. PHP under 7.2 is inherently unable to be
used in production for as long as this issue last!

Lars Povlsen
Comment 20 James R Roe 2002-12-02 18:46:44 EST
This is a huge problem that has bitten me on multiple production servers doing 
virtual hosting.

The silence from Red Hat on this is deafening
Comment 21 Michiel Toneman 2002-12-02 19:46:19 EST
This has become a MAJOR disappointment for me from a company I used to hold in
high regard.

The problem first bit me at the end of August 2002. This means I've been trying
to keep our servers from breaking for the last 3 months awaiting a fix.

I've been asked to file this as a separate bug by jorton@redhat.com on September
26 , which I did the same day as
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=74543, notifying him by
email of the new bugreport. This report has been ignored (status is still
"NEW"). Many others have made it clear that they have been bitten by the same
problem.

When are you going to do something about this? Is the complexity beyond you, or
are you so understaffed that you can't get around fixing high priority bugs?
Either way, you might consider hiring me, I fail to see how I could possibly do
a worse job...

Comment 22 Joe Orton 2002-12-03 09:00:37 EST
An errata to fix bug 74543 is in progress and should be available soon -
apologies for the delay.
Comment 23 Kevin M. Myer 2002-12-03 09:05:24 EST
Maybe a post to BugTraq or some other high profile list might get some of
RedHat's attention.  I too am baffled by Red Hat's silence and inattention to a
serious problem.  Makes me wonder how much longer they're going to be around as
a company...  Anyone up to writing a security advisory or know someone who works
at a "security" firm that issues such advisories?  I hate to force Red Hat's
hand but three months is more than enough time to fix this bugger.  All the
arguments about open-source software bugs getting fixed faster than
closed-source software bugs are bunk on this bug...
Comment 24 Lars Povlsen 2002-12-09 08:53:47 EST
Incidentally, bug 74543 is still state "NEW", priority "normal"...
Comment 25 Joe Orton 2002-12-09 09:23:43 EST
http://people.redhat.com/jorton/7.x-mm/ for unsupported RPMs; errata currently
being prepared.
Comment 26 Louis Aslett 2002-12-10 14:11:55 EST
I am not sure if I should open a new bug for this or not:

Create the following simple PHP script and access it through a web browser:

<?php
  $crash = array( 0 => 2,
                 'test' => 2,
                  1 => 'hello',
                 'say' => 'hello',
                  2 => 42,
                 'life' => 42,
                  3 => 'this should help \'crash\' the machine',
                 'hoho' => 'this should help \'crash\' the machine');

  print_r($crash);

  for( $i=0; $i<count($crash); $i++ )
    $crash[$i] = stripslashes($crash[$i]);

  print_r($crash);
?>

It should die with an error similar to this:

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 
35 bytes) in /home/eid/crash.php on line 14

Reload this page a good 5-10 times.  If you run 'ipcs -s' and then restart 
apache and run 'ipcs -s' again you will find that the number of semaphore 
arrays has increased and the first few semid's are unchanged (not having been 
freed when apache shutdown?).  If you rinse and repeat this with a crude shell 
script like:

while [ true ]; do
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 wget -O - http://localhost/crash.php
 ipcs -s|grep apache|wc
 /etc/rc.d/init.d/httpd restart
 sleep 1
 ipcs -s|grep apache|wc
done

then you'll find the semaphore array numbers increasing a couple every restart 
and apache taking longer and longer to do each restart until eventually (once 
all 128 semaphore arrays are used) it refuses to start at all with the message 
reported earlier in this bug (70846):

Starting httpd: Ouch! ap_mm_create(1048576, "/var/run/httpd.mm.5619") failed
Error: MM: mm:core: failed to acquire semaphore (No space left on device): OS: 
Invalid argument
                                                           [FAILED]

Just restarting apache in a loop without loading crash.php on a freshly booted 
system does not cause the number of semaphores to spiral - it stays constant at 
5.

This is verifyable on multiple RH7.1 and a RH8.0 machine, all fully updated 
through RHN (except for kernels).

Louis
Comment 27 Joe Orton 2002-12-12 06:23:21 EST
http://rhn.redhat.com/errata/RHBA-2002-273.html

has been issued to change mm back to using fcntl locks, so this leak should go
away - again, apologies it took so long for this to happen.

I'll leave this bug open as there's clearly a PHP bug here too in how it uses mm
properly.  (although it is now less serious now)
Comment 28 Sergeyssels Wouter 2003-01-07 08:18:13 EST
Have installed this errata http://rhn.redhat.com/errata/RHBA-2002-273.html.

Problem with apache is fixed now. When I run some PHP scripts at the 
commandline I still get:  PHP Fatal error:  Unable to start session mm module 
in Unknown on line 0

Got this problem already for a few months.

php version: 4.1.2
redhat: 7.2
mm-1.1.3-11
mm-devel-1.1.3-11

Comment 29 Ourbrisbane Technical Support 2003-03-31 21:36:46 EST
We are running:

php version: 4.1.2
redhat: 7.3
mm-1.1.3-11

and have the same problem: 

When I run some PHP scripts at the commandline as non-root user:  PHP Fatal 
error:  Unable to start session mm module 
in Unknown on line 0.  Scripts run as root user however.

This error does not occur with mm-1.1.3-8.  Please advise why this error is 
occuring with mm-1.1.3-11?

Regards,

Jonne Hannon. 
Comment 30 Joe Orton 2003-04-01 03:54:37 EST
Can you attach a minimal script which reproduces that error? Also 'rpm -q php'
output.
Comment 31 Joe Orton 2005-01-25 11:47:50 EST
Thanks for the report.  This is a mass bug update; since this release
of Red Hat Linux is no longer supported, please either:

a) try and reproduce the bug with a supported version of Red Hat
Enterprise Linux or Fedora Core, and re-open this bug as appropriate
after changing the Product field, or,

b) if relevant, try and reproduce this bug using the current version
of the upstream package, and report the bug upstream.

c) report the bug to the Fedora Legacy project who may wish to
continue maintenance of this package.

Note You need to log in before you can comment on or make changes to this bug.