Bug 1723564 - gdb crash when using PYTHONMALLOC=debug on Python
Summary: gdb crash when using PYTHONMALLOC=debug on Python
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: gdb
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Sergio Durigan Junior
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-24 20:06 UTC by Victor Stinner
Modified: 2019-07-08 10:02 UTC (History)
6 users (show)

Fixed In Version: gdb-8.3-6.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-05 01:32:27 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Sourceware 24742 None None None 2019-07-30 16:13:17 UTC

Description Victor Stinner 2019-06-24 20:06:36 UTC
Install dependencies:

   sudo dnf debuginfo-install python3

Run Python:

$ PYTHONMALLOC=debug gdb -args python3 -c pass
GNU gdb (GDB) Fedora 8.3-3.fc30

Reading symbols from python3...
Reading symbols from /usr/lib/debug/usr/bin/python3.7m-3.7.3-3.fc30.x86_64.debug...

(gdb) run
Starting program: /usr/bin/python3 -c pass
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.29-9.fc30.x86_64
Debug memory block at address p=0x5603977bf330: API ''
    8098648152243306496 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfb):
        at p-7: 0x03 *** OUCH
        at p-6: 0x00 *** OUCH
        at p-5: 0x00 *** OUCH
        at p-4: 0x00 *** OUCH
        at p-3: 0x00 *** OUCH
        at p-2: 0x00 *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0x706483999ad1f330 are Segmentation fault (core dumped)


Version-Release number of selected component (if applicable):

* Up to date Fedora 30
* gdb-8.3-3.fc30.x86_64
* python3-3.7.3-3.fc30.x86_64


Maybe gdb uses the Python memory allocator, maybe indirectly by using the Python C API, before Python is fully initialized (before PYTHONMALLOC env var is handled). 

Maybe gdb should ignore PYTHONMALLOC and PYTHONDEVMODE (try PYTHONDEVMODE=1) environment variables when embedding Python.

For Python 3.8, I wrote a new C API to initialize Python which provides an "isolated configuration" which prevents this issue ;-)

* https://docs.python.org/dev/c-api/init_config.html#isolated-configuration
* http://docs.python.org/dev/c-api/init_config.html
* https://www.python.org/dev/peps/pep-0587/

Comment 1 Sergio Durigan Junior 2019-06-24 23:56:33 UTC
Thanks for the report, Victor.

I was able to reproduce this with upstream GDB as well, so I'll forward this bug to upstream.  However, something interesting happened: on my F30, I was only able to reproduce this bug once.  When I tried again, GDB would start normally.  I wonder why this is happening here...

I was also able to reproduce this on a Rawhide VM, using upstream GDB.

This seems like a very specific use case.  Would you recommend unsetting these two environment variables before we start initializing Python?

Thanks.

Comment 2 Victor Stinner 2019-06-27 15:37:42 UTC
> I was able to reproduce this with upstream GDB as well, so I'll forward this bug to upstream.  However, something interesting happened: on my F30, I was only able to reproduce this bug once.  When I tried again, GDB would start normally.  I wonder why this is happening here...

Memory allocations are not really deterministic. ASLR might enter into the play. File modification time as well. Many things can change the exact memory allocation order. free() does not always crash when misused :-)

> This seems like a very specific use case.  Would you recommend unsetting these two environment variables before we start initializing Python?

Well, there are multiple options:

* investigate how Python memory allocator functions are used
* Use PEP 587 on Python 3.8 and newer (that doesn't fix the issue for Python 3.7 and older)
* Unset PYTHONMALLOC and PYTHONDEVMODE env var, or unset all PYTHON* vars (what about PYTHONPATH for example? it's up to you)

I'm using PYTHONMALLOC and PYTHONDEVMODE to debug Python applications. I didn't *expect* that gdb would use them :-)

Comment 3 Sergio Durigan Junior 2019-06-27 15:48:53 UTC
(In reply to Victor Stinner from comment #2)
> > I was able to reproduce this with upstream GDB as well, so I'll forward this bug to upstream.  However, something interesting happened: on my F30, I was only able to reproduce this bug once.  When I tried again, GDB would start normally.  I wonder why this is happening here...
> 
> Memory allocations are not really deterministic. ASLR might enter into the
> play. File modification time as well. Many things can change the exact
> memory allocation order. free() does not always crash when misused :-)

I understand that, but what surprised me was that I was able to reproduce the bug like 10 times in a row, and then wasn't able to reproduce it anymore after hundreds of times.

> > This seems like a very specific use case.  Would you recommend unsetting these two environment variables before we start initializing Python?
> 
> Well, there are multiple options:
> 
> * investigate how Python memory allocator functions are used
> * Use PEP 587 on Python 3.8 and newer (that doesn't fix the issue for Python
> 3.7 and older)
> * Unset PYTHONMALLOC and PYTHONDEVMODE env var, or unset all PYTHON* vars
> (what about PYTHONPATH for example? it's up to you)

I think there may be a simpler solution.  I have a patch here, but as I said, I can't reproduce the bug anymore.

> I'm using PYTHONMALLOC and PYTHONDEVMODE to debug Python applications. I
> didn't *expect* that gdb would use them :-)

GDB doesn't use them, as far as I know and have researched.

Comment 4 Sergio Durigan Junior 2019-06-27 15:55:15 UTC
(In reply to Sergio Durigan Junior from comment #3)
> I think there may be a simpler solution.  I have a patch here, but as I
> said, I can't reproduce the bug anymore.

I managed to reproduce the bug here (by removing everything from the build directory and rebuilding GDB), and managed to test the patch.  It seems to work.  I will propose it upstream, let's see what they say.

Comment 5 Sergio Durigan Junior 2019-06-28 20:37:55 UTC
Patch has been pushed upstream:

https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5af5392a3d1525fb825747b203a6159ddcba0aa4

I'll backport it to F30.

Comment 6 Fedora Update System 2019-06-28 21:21:16 UTC
FEDORA-2019-0497420860 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-0497420860

Comment 7 Fedora Update System 2019-06-29 00:48:00 UTC
gdb-8.3-5.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-0497420860

Comment 8 Fedora Update System 2019-07-03 18:50:27 UTC
FEDORA-2019-4a8ba02caf has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-4a8ba02caf

Comment 9 Fedora Update System 2019-07-04 00:58:30 UTC
gdb-8.3-6.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-4a8ba02caf

Comment 10 Fedora Update System 2019-07-05 01:32:27 UTC
gdb-8.3-6.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 11 Victor Stinner 2019-07-08 10:02:11 UTC
I tested manually gdb-8.3-6.fc30 and I confirm that gdb no longer crash for my use case, thanks!


Note You need to log in before you can comment on or make changes to this bug.