Bug 520618

Created attachment 359384 [details]
proposed patch, which fixes this issue

Description of problem:

Due to a patch, which introduced a mechanism, which hides breakpoint instructions and returns a 'shadow' content during every access to inferrior's memory by a function target_read_memory(), gdb, when thousands of breakpoints are inserted, is very slow when calling a continue command.


Version-Release number of selected component (if applicable):

gdb on RHEL5 (U3, U4), also on Fedora 10 and 11

How reproducible:

always


Steps to Reproduce:

1. Compile attached source file with debuging symbols turned on:

gcc -g -o testC5000 testC5000.c

2. Run gdb with attached command file:

$ gdb testC5000 -x cmd2000.txt

or

$ gdb testC5000 -x cmd5000.txt

3. Wait for gdb to insert all breakpoints and see that continuing is very slow
even at a first glance.


Actual results:

time gdb testC5000 -x cmd2000.txt

real    1m14.356s
user    0m59.857s
sys     0m13.444s


Expected results:

time gdb testC5000 -x cmd2000.txt

real    0m25.320s
user    0m11.769s
sys     0m13.033s


Additional info:

This issue in a case of RHEL5 is introduced by a patch:

gdb-6.8-bz467502-breakpoint_restore_shadows.patch

This patch joins the code responsible for restoring original content on place of breakpoint address from saved shadow_contents buffer:

(struct bp_location *)->(struct bp_target_info *)->shadow_contents)

from separate function read_memory_nobpt() into breakpoint_restore_shadows(), which is called by target_read_memory() - the function used by every single access (reading) to debugged application's memory.

According to posts on mailing list it is an intended behaviour:

http://sourceware.org/ml/gdb-patches/2008-02/msg00365.html
http://sourceware.org/ml/gdb-patches/2008-03/msg00106.html
http://sourceware.org/ml/gdb-patches/2008-02/msg00375.html

I can see from the discussion, that this is an intention to leave an user a chance to change this behaviour by calling "set breakpoint always-inserted on/off". But this option doesn't exist in RHEL5 even it is mentioned in testing scripts untroduced by a patch:

gdb-6.8-bz467502-ia64-breakpoint-restoration.patch

The original patch from Vladimir Prus contained an option "show-memory-breakpoints", but changes (which afterwards showed up in upstream sources) made by Daniel Jacobowitz removed it due to proposed always-inserted option. So no one of the mentioned options are presented in RHEL5 gdb sources (is this intended or is this ("always-inserted") option missing?).

So according to the fact that this is intended behaviour and by default turned on - global variable show_memory_breakpoints is zero, which enables automatic restoration of memory on address of breakpoint in every acces to debugged application's memory, I started to search for functions that doesn't need this restoration and are responsible for this slowdown. I have found a function default_memory_insert_breakpoint().

This function is called by:

insert_breakpoints()->insert_bp_location()->memory_insert_breakpoint()->gdbarch_memory_insert_breakpoint()->default_memory_insert_breakpoint()

A function memory_insert_breakpoint() does checking for existing and inserted breakpoints:

1032      if (!bpt->enabled || bpt->shlib_disabled || bpt->inserted || bpt->duplicate)
1033        return 0;

so trying to restore shadow content in default_memory_insert_breakpoint() is useless as there cannot be any inserted breakpoint by the time entering the function.

So I have created a patch which turns this restoration before reading memory content on address of future breakpoint off (see attached patch) and it solves the problem.

Someone, who is more familiar with the sources, should also check other target_read_memory() calls whether there is really needed to call breakpoint_restore_shadows() internally.

You can find an existing entry describing this bug in gdb bugzilla:

http://sourceware.org/bugzilla/show_bug.cgi?id=9335

Also one should look at sources in fedora 11, as the process of inserting breakpoints becomes slower and slower with increasing number of inserted breakpoints. This doesn't happen with gdb sources in RHEL5.