Bug 1655310 - PyQt5 crash on KDE + plasma-integration: malloc_consolidate(): invalid chunk size when running tests
Summary: PyQt5 crash on KDE + plasma-integration: malloc_consolidate(): invalid chunk ...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: 29
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-02 15:49 UTC by TR Bentley
Modified: 2019-10-29 17:28 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-04 13:15:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
pytest log (207.73 KB, text/plain)
2018-12-03 22:43 UTC, Miro Hrončok
no flags Details
list of packages (14.31 KB, text/plain)
2018-12-03 22:44 UTC, Miro Hrončok
no flags Details
Crash report generated by kcrash (72.74 KB, text/plain)
2019-01-08 23:05 UTC, Raoul Snyman
no flags Details

Description TR Bentley 2018-12-02 15:49:46 UTC
Description of problem: When running tests under NoseTests or PyTests it fails with a malloc_consolidate(): invalid chunk size.


Version-Release number of selected component (if applicable): Up to date Fedora 29.


How reproducible: Every time on two different Fedora machines


Steps to Reproduce:
1. Run Nosetest-3 on a source tree with test files and it fails
2.
3.

Actual results:
⏚ 1z [tim:~/Projects/OpenLP/openlp/media] media(2863) ⚯ nosetests-3
Warning: QT_DEVICE_PIXEL_RATIO is deprecated. Instead use:
   QT_AUTO_SCREEN_SCALE_FACTOR to enable platform plugin controlled per-screen factors.
   QT_SCREEN_SCALE_FACTORS to set per-screen factors.
   QT_SCALE_FACTOR to set the application global scale factor.
malloc_consolidate(): invalid chunk size
KCrash: Application 'python3.7' crashing...
Alarm clock


⏚ 1z [tim:~/Projects/OpenLP/openlp/media] media(2863) ⚯ pytest-3
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.7.1, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /home/tim/Projects/OpenLP/openlp/media, inifile:
collecting 0 items                                                                                                                                                        [1]   Exit 253                pytest-3

[2]+  Stopped                 pytest-3
⏚ 1z [tim:~/Projects/OpenLP/openlp/media] media(2863) ⚯


Expected results: Some tests to be executed


Additional info:  Fails on NoseTests and pytests.

Comment 1 Miro Hrončok 2018-12-02 23:57:48 UTC
Is the OpenLP project open source? Can I download the tests you are running to be able to reproduce the problem?

Honestly, this smells like a problem in the code that's being tested or the tests themselves, but I'm willing to give it a try.

Comment 2 TR Bentley 2018-12-03 17:47:06 UTC
The code base I am running is here 

https://code.launchpad.net/~trb143/openlp/media_state

I am not the only person who is having a problem.  The project lead is a Red Hat employee and he checked as well.  He has the problem on a different branch.  This code base worked fine on F28!

The problem is seen running the nosetests-3 or pytests command in the project root directory.  It would normally run the tests directory.

Comment 3 Miro Hrončok 2018-12-03 18:23:12 UTC
E   ImportError: No module named 'alembic' (or 'alembic.migration.MigrationContext' is unknown)

E   ImportError: No module named 'qtawesome' (or 'openlp.core.ui.icons.UiIcons' is unknown)

This is what I get when I run the tests.

Before I proceed, how do you install the packages?

Comment 4 TR Bentley 2018-12-03 19:24:44 UTC
Ok all packages are available in fedora.  I am the package maintainer but the spec file needs to be updated for this version.

Here are the two files you seem to be missing.

python3-alembic
python3-QtAwesome.noarch

python3-pymediainfo.noarch may come up as well.

Comment 5 TR Bentley 2018-12-03 19:35:41 UTC
Just patched my machine from fedora stable with no code changes and got this.

[tim:~/Projects/OpenLP/openlp/media] media(2863) ⚯ nosetests-3
Warning: QT_DEVICE_PIXEL_RATIO is deprecated. Instead use:
   QT_AUTO_SCREEN_SCALE_FACTOR to enable platform plugin controlled per-screen factors.
   QT_SCREEN_SCALE_FACTORS to set per-screen factors.
   QT_SCALE_FACTOR to set the application global scale factor.
free(): invalid pointer
KCrash: Application 'python3.7' crashing...
KCrash: Attempting to start /usr/libexec/drkonqi from kdeinit
sock_file=/run/user/1000/kdeinit5__0

[1]+  Stopped                 nosetests-3

A different error. 

python3-gobject amd pyxdg were updated.

Comment 6 Miro Hrončok 2018-12-03 22:43:11 UTC
Created attachment 1511136 [details]
pytest log

Sorry I can reproduce this.

Comment 7 Miro Hrončok 2018-12-03 22:44:28 UTC
Created attachment 1511137 [details]
list of packages

Comment 8 Miro Hrončok 2018-12-03 22:46:04 UTC
Can you reproduce this with:

/usr/bin/python3.7 -s /usr/bin/pytest-3

?

Comment 9 Raoul Snyman 2018-12-04 04:25:48 UTC
Hey Tim, I just checked on my Fedora 29 machine (up-to-date) with nosetests-3.7, nose2-3.7, and pytest-3.7, and I cannot reproduce this, either against your branch or against trunk.

Perhaps there is something else? Are you running pytest or nose in a virtual environment?

Comment 10 TR Bentley 2018-12-08 08:40:56 UTC
Fixed this but it is strange.

Rebuilt laptop and tested worked OK.

Installed the OpenLP 2.4.6 version from fedora.
Fails
.
Uninstall OpenLP 2.4.6 
Works.

Scratch head.

I am trying to develop. OpenLP 3.0

This can be closed as I try to make sense of this!

Comment 11 TR Bentley 2018-12-08 09:01:27 UTC
I have now made sense of this and found the problem package.

python3-qt5

Installed each file in the spec file and tested before and after.

This has caused the problem.

Comment 12 Victor Stinner 2018-12-10 13:29:34 UTC
You may try to set PYTHONMALLOC=debug environment variable to enable Python builtin memory debugger, to check for buffer underflow and buffer overflow.

Or you can try PYTHONMALLOC=malloc but run your tests in Valgrind.

Comment 13 TR Bentley 2018-12-10 16:45:36 UTC
Tried this and also malloc_debug but still have no lock.
Using pytest-3 as the test runner all I get is. 

platform linux -- Python 3.7.1, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /home/tim/Projects/OpenLP/openlp/media, inifile:
collecting 0 items                                                                                                                                                        Alarm clock
[tim:~/Projects/OpenLP/openlp/media] media(2863) ⚯

Comment 14 Victor Stinner 2018-12-10 16:47:11 UTC
> Alarm clock

That looks a bug in your code, or maybe in pytest/nosetest.

Anyway, this issue looks a bug in your code, not in Python itself. Usually, it's a buffer overflow or buffer underflow somewhere in your code.

Comment 15 Petr Viktorin (pviktori) 2019-01-07 13:35:19 UTC
I was not able to reproduce this. Could you provide a complete reproducer? Preferably with the exact commands, so we know we're talking about the same steps.

Comment 16 Victor Stinner 2019-01-07 14:05:46 UTC
I downloaded OpenLP-2.4.6, installed it in a virtual environment, but I failed to reproduce the issue: 2 tests fail with "nosetests", but it doesn't crash.

I installed these packages for the test:

  sudo dnf install -y python3-qt5 python3-qt5-webkit bzr

Versions:

* Fedora 29 (up to date)
* OpenLP-2.4.6
* python3-3.7.2-1.fc29.x86_64
* python3-qt5-webkit-5.11.2-4.fc29.x86_64
* python3-qt5-5.11.2-4.fc29.x86_64
* nose 1.3.7

I also tried "PYTHONMALLOC=debug nosetests", "PYTHONMALLOC=malloc nosetests" and "python -X dev $(which nosetests)": no crash neither.

Maybe the bug was fixed in the meanwhile. I suggest to close this bug.

Failing tests:

* FAIL: Test that loading a SongShow Plus file works correctly on various files: AssertionError: add_verse('Did we in our own strength confide...) call not found
* FAIL: Test for proper bzr tags: AssertionError: 32 != 0 : List of tags should match

Comment 17 TR Bentley 2019-01-07 17:47:41 UTC
This is still happening but I have noticed one thing which is confusing.
Installing python3-pytest  brings in a dependance of python3-pluggy 0.6  

When you run pytest-3 it is running pluggy 0.7,1 which has never been released.

Below is a test run with Pytest removed, run fails,  Installed from dnf (see the versions) and then run (different versions).

Interesting why does ALARM CLOCK get printed!



1z [tim:~/Projects/OpenLP/openlp/media] media(2914) ⚯ pytest-3
bash: /usr/bin/pytest-3: No such file or directory
[1]+  Exit 253                pytest-3
[tim:~/Projects/OpenLP/openlp/media] media(2914) ⚯ sudo dnf install python3-pytest
Last metadata expiration check: 0:26:36 ago on Mon 07 Jan 2019 17:18:45 GMT.
Dependencies resolved.
===========================================================================================================================================================================
 Package                                            Arch                               Version                                    Repository                          Size
===========================================================================================================================================================================
Installing:
 python3-pytest                                     noarch                             3.6.4-1.fc29                               fedora                             1.4 M
Installing dependencies:
 python3-atomicwrites                               noarch                             1.1.5-12.fc29                              fedora                              21 k
 python3-attrs                                      noarch                             17.4.0-7.fc29                              fedora                              52 k
 python3-more-itertools                             noarch                             4.1.0-4.fc29                               fedora                              48 k
 python3-pluggy                                     noarch                             0.6.0-5.fc29                               fedora                              31 k
 python3-py                                         noarch                             1.5.4-3.fc29                               fedora                             463 k

Transaction Summary
===========================================================================================================================================================================
Install  6 Packages

Total download size: 2.0 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): python3-atomicwrites-1.1.5-12.fc29.noarch.rpm                                                                                        20 kB/s |  21 kB     00:01    
(2/6): python3-more-itertools-4.1.0-4.fc29.noarch.rpm                                                                                       44 kB/s |  48 kB     00:01    
(3/6): python3-attrs-17.4.0-7.fc29.noarch.rpm                                                                                               48 kB/s |  52 kB     00:01    
(4/6): python3-pluggy-0.6.0-5.fc29.noarch.rpm                                                                                              765 kB/s |  31 kB     00:00    
(5/6): python3-py-1.5.4-3.fc29.noarch.rpm                                                                                                  1.9 MB/s | 463 kB     00:00    
(6/6): python3-pytest-3.6.4-1.fc29.noarch.rpm                                                                                              3.1 MB/s | 1.4 MB     00:00    
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                      671 kB/s | 2.0 MB     00:03     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                   1/1 
  Installing       : python3-py-1.5.4-3.fc29.noarch                                                                                                                    1/6 
  Installing       : python3-pluggy-0.6.0-5.fc29.noarch                                                                                                                2/6 
  Installing       : python3-more-itertools-4.1.0-4.fc29.noarch                                                                                                        3/6 
  Installing       : python3-attrs-17.4.0-7.fc29.noarch                                                                                                                4/6 
  Installing       : python3-atomicwrites-1.1.5-12.fc29.noarch                                                                                                         5/6 
  Installing       : python3-pytest-3.6.4-1.fc29.noarch                                                                                                                6/6 
  Running scriptlet: python3-pytest-3.6.4-1.fc29.noarch                                                                                                                6/6 
  Verifying        : python3-atomicwrites-1.1.5-12.fc29.noarch                                                                                                         1/6 
  Verifying        : python3-attrs-17.4.0-7.fc29.noarch                                                                                                                2/6 
  Verifying        : python3-more-itertools-4.1.0-4.fc29.noarch                                                                                                        3/6 
  Verifying        : python3-pluggy-0.6.0-5.fc29.noarch                                                                                                                4/6 
  Verifying        : python3-py-1.5.4-3.fc29.noarch                                                                                                                    5/6 
  Verifying        : python3-pytest-3.6.4-1.fc29.noarch                                                                                                                6/6 

Installed:
  python3-pytest-3.6.4-1.fc29.noarch    python3-atomicwrites-1.1.5-12.fc29.noarch    python3-attrs-17.4.0-7.fc29.noarch    python3-more-itertools-4.1.0-4.fc29.noarch   
  python3-pluggy-0.6.0-5.fc29.noarch    python3-py-1.5.4-3.fc29.noarch              

Complete!
[tim:~/Projects/OpenLP/openlp/media] media(2914) ⚯ pytest-3
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.7.1, pytest-3.6.4, py-1.7.0, pluggy-0.7.1
rootdir: /home/tim/Projects/OpenLP/openlp/media, inifile:
collecting 0 items                                                                                                                                                        Alarm clock
[tim:~/Projects/OpenLP/openlp/media] media(2914) ⚯

Comment 18 Miro Hrončok 2019-01-07 18:25:04 UTC
What's the output here?


$ python3 -c 'import pluggy; print(pluggy.__file__)'

Comment 19 TR Bentley 2019-01-07 19:07:04 UTC
/usr/lib/python3.7/site-packages/pluggy/__init__.py

Comment 20 Miro Hrončok 2019-01-07 22:27:16 UTC
$ python3 -m pip show pluggy
$ python3 -m pip show pytest
$ rpm -V $(rpm -qa | grep python3)

Comment 21 TR Bentley 2019-01-08 19:11:24 UTC
[tim:~] $ python3 -m pip show pluggy
Name: pluggy
Version: 0.6.0
Summary: plugin and hook calling mechanisms for python
Home-page: https://github.com/pytest-dev/pluggy
Author: Holger Krekel
Author-email: holger
License: MIT license
Location: /usr/lib/python3.7/site-packages
Requires: 
Required-by: pytest
[tim:~] $ ^C
[tim:~] $ [tim:~] $ python3 -m pip show pytest
Name: pytest
Version: 3.6.4
Summary: pytest: simple powerful testing with Python
Home-page: http://pytest.org
Author: Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others
Author-email: None
License: MIT license
Location: /usr/lib/python3.7/site-packages
Requires: py, six, setuptools, attrs, more-itertools, atomicwrites, pluggy
Required-by:

[tim:~] $ rpm -V $(rpm -qa | grep python3)
.M.......  g /var/cache/dnf/packages.db
..?......    /usr/lib/python3.7/site-packages/_pytest/__pycache__/pytester.cpython-37-PYTEST.pyc
[tim:~] $

Comment 22 Miro Hrončok 2019-01-08 22:24:31 UTC
OK, I have no idea, why would you get pluggy-0.7.1.

Also, nobody was able to reproduce this crash.

Could you please provide a complete set of commands to reproduce it?

Can you reproduce in a fresh Virtual Machine?

Comment 23 Raoul Snyman 2019-01-08 23:05:03 UTC
Created attachment 1519299 [details]
Crash report generated by kcrash

I can get pytest to crash when running Tim's branch, as well as current trunk for OpenLP on my Fedora 29 laptop (up-to-date). This started crashing only after I upgraded on Monday, 7 January. My last update prior to this was beginning-to-mid December.

I've attached the crash report generated by kcrash.

Comment 24 Miro Hrončok 2019-01-09 07:53:57 UTC
Raoul, please provide a concrete set of commands leading to the crash.

Comment 25 Raoul Snyman 2019-01-09 15:06:19 UTC
Oh, sure. I had Tim's branch checked out, and just run "pytest-3", like this:


$ pytest-3
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.7.2, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /.../media_state, inifile:
collecting 0 items                                                                                                                                                                                                                          [1]    2900 alarm      pytest-3
$

Comment 26 Raoul Snyman 2019-01-09 15:20:51 UTC
I get a similar (if not the same) crash on my Debian PC. I'll have to take a deeper look at the crash report and see if I can track down where it's happening.

Comment 27 Miro Hrončok 2019-01-09 16:11:01 UTC
please provide a concrete set of commands.

"I had Tim's branch checked out" is not a concrete set of commands.

But I'll play along:

[empty]$ pwd
/home/churchyard/tmp/empty
[empty]$ bzr branch lp:~trb143/openlp/media_state
bash: bzr: command not found
[empty]$ dnf -qy install bzr
[sudo] password for churchyard: 
Failed loading plugin: py3query
[empty]$ bzr branch lp:~trb143/openlp/media_state
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Branched 2915 revisions.                                                                                                                                                                                                        
[empty]$ ls
media_state
[empty]$ cd media_state/
[media_state]$ pytest-3
============================= test session starts ==============================
platform linux -- Python 3.7.2, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /home/churchyard/tmp/empty/media_state, inifile:
plugins: virtualenv-1.2.11, shutil-1.2.6
collected 457 items / 106 errors

==================================== ERRORS ====================================
__________ ERROR collecting tests/functional/openlp_core/test_app.py ___________
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/test_app.py:28: in <module>
    from openlp.core.app import OpenLP, parse_options
openlp/core/app.py:41: in <module>
    from openlp.core.loader import loader
openlp/core/loader.py:27: in <module>
    from openlp.core.ui.media.mediacontroller import MediaController
openlp/core/ui/media/mediacontroller.py:39: in <module>
    from openlp.core.api.http import register_endpoint
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
________ ERROR collecting tests/functional/openlp_core/api/test_tab.py _________
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/test_tab.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/test_tab.py:30: in <module>
    from openlp.core.api.tab import ApiTab
openlp/core/api/tab.py:32: in <module>
    from openlp.core.ui.icons import UiIcons
openlp/core/ui/icons.py:26: in <module>
    import qtawesome as qta
E   ModuleNotFoundError: No module named 'qtawesome'
_____ ERROR collecting tests/functional/openlp_core/api/test_websockets.py _____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/test_websockets.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/test_websockets.py:29: in <module>
    from openlp.core.api.websockets import WebSocketServer
openlp/core/api/websockets.py:31: in <module>
    from websockets import serve
E   ModuleNotFoundError: No module named 'websockets'
 ERROR collecting tests/functional/openlp_core/api/endpoint/test_controller.py _
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/endpoint/test_controller.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/endpoint/test_controller.py:29: in <module>
    from openlp.core.api.endpoint.controller import controller_text, controller_direction
openlp/core/api/endpoint/controller.py:28: in <module>
    from openlp.core.api.http import requires_auth
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
__ ERROR collecting tests/functional/openlp_core/api/endpoint/test_remote.py ___
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/endpoint/test_remote.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/endpoint/test_remote.py:27: in <module>
    from openlp.core.api.endpoint.remote import index
openlp/core/api/endpoint/remote.py:24: in <module>
    from openlp.core.api.endpoint.core import TRANSLATED_STRINGS
openlp/core/api/endpoint/core.py:28: in <module>
    from openlp.core.api.http import requires_auth
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
_____ ERROR collecting tests/functional/openlp_core/api/http/test_error.py _____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/http/test_error.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/http/test_error.py:25: in <module>
    from openlp.core.api.http.errors import HttpError, NotFound, ServerError
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
_____ ERROR collecting tests/functional/openlp_core/api/http/test_http.py ______
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/http/test_http.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/http/test_http.py:28: in <module>
    from openlp.core.api.http.server import HttpServer
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
_____ ERROR collecting tests/functional/openlp_core/api/http/test_init.py ______
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/http/test_init.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/http/test_init.py:28: in <module>
    from openlp.core.api.http import check_auth, requires_auth, authenticate
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
____ ERROR collecting tests/functional/openlp_core/api/http/test_wsgiapp.py ____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/http/test_wsgiapp.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/http/test_wsgiapp.py:29: in <module>
    from openlp.core.api.http import register_endpoint, application
openlp/core/api/http/__init__.py:26: in <module>
    from webob import Response
E   ModuleNotFoundError: No module named 'webob'
_______ ERROR collecting tests/functional/openlp_core/common/test_db.py ________
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/common/test_db.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/common/test_db.py:35: in <module>
    from openlp.core.lib.db import init_db, get_upgrade_op
openlp/core/lib/db.py:32: in <module>
    from alembic.migration import MigrationContext
E   ModuleNotFoundError: No module named 'alembic'
____ ERROR collecting tests/functional/openlp_core/display/test_renderer.py ____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/display/test_renderer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/display/test_renderer.py:31: in <module>
    from openlp.core.display.renderer import Renderer, words_split, get_start_tags
openlp/core/display/renderer.py:26: in <module>
    from PyQt5 import QtGui, QtCore, QtWebKitWidgets
E   ImportError: cannot import name 'QtWebKitWidgets' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
_________ ERROR collecting tests/functional/openlp_core/lib/test_db.py _________
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/lib/test_db.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/lib/test_db.py:35: in <module>
    from openlp.core.lib.db import init_db, get_upgrade_op, delete_database, upgrade_db
openlp/core/lib/db.py:32: in <module>
    from alembic.migration import MigrationContext
E   ModuleNotFoundError: No module named 'alembic'
____ ERROR collecting tests/functional/openlp_core/lib/test_htmlbuilder.py _____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/lib/test_htmlbuilder.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/lib/test_htmlbuilder.py:7: in <module>
    from PyQt5 import QtCore, QtWebKit
E   ImportError: cannot import name 'QtWebKit' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
__ ERROR collecting tests/functional/openlp_core/lib/test_mediamanageritem.py __
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/lib/test_mediamanageritem.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/lib/test_mediamanageritem.py:28: in <module>
    from openlp.core.lib.mediamanageritem import MediaManagerItem
openlp/core/lib/mediamanageritem.py:31: in <module>
    from openlp.core.ui.icons import UiIcons
openlp/core/ui/icons.py:26: in <module>
    import qtawesome as qta
E   ModuleNotFoundError: No module named 'qtawesome'
____ ERROR collecting tests/functional/openlp_core/lib/test_serviceitem.py _____
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/lib/test_serviceitem.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/lib/test_serviceitem.py:35: in <module>
    from openlp.core.lib.serviceitem import ItemCapabilities, ServiceItem, ServiceItemType
openlp/core/lib/serviceitem.py:39: in <module>
    from openlp.core.ui.icons import UiIcons
openlp/core/ui/icons.py:26: in <module>
    import qtawesome as qta
E   ModuleNotFoundError: No module named 'qtawesome'
...snip...



Now how do I get the missing deps? pip? dnf? pipenv? easy_install?

Comment 28 TR Bentley 2019-01-09 17:45:09 UTC
python3-appdirs
python3-webob
python3-QtAwesome.noarch
python3-websockets
python3-waitress
python3-pymediainfo

are the new fedora packages which will be added to the spec file when the next version is released.

Comment 29 Miro Hrončok 2019-01-09 19:50:06 UTC
$ pytest-3
============================= test session starts ==============================
platform linux -- Python 3.7.2, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /home/churchyard/tmp/empty/media_state, inifile:
plugins: virtualenv-1.2.11, shutil-1.2.6
collected 716 items / 64 errors

==================================== ERRORS ====================================
__________ ERROR collecting tests/functional/openlp_core/test_app.py ___________
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/test_app.py:28: in <module>
    from openlp.core.app import OpenLP, parse_options
openlp/core/app.py:41: in <module>
    from openlp.core.loader import loader
openlp/core/loader.py:29: in <module>
    from openlp.core.display.renderer import Renderer
openlp/core/display/renderer.py:26: in <module>
    from PyQt5 import QtGui, QtCore, QtWebKitWidgets
E   ImportError: cannot import name 'QtWebKitWidgets' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
 ERROR collecting tests/functional/openlp_core/api/endpoint/test_controller.py _
ImportError while importing test module '/home/churchyard/tmp/empty/media_state/tests/functional/openlp_core/api/endpoint/test_controller.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/functional/openlp_core/api/endpoint/test_controller.py:30: in <module>
    from openlp.core.display.renderer import Renderer
openlp/core/display/renderer.py:26: in <module>
    from PyQt5 import QtGui, QtCore, QtWebKitWidgets
E   ImportError: cannot import name 'QtWebKitWidgets' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
...
E   ImportError: No module named 'alembic' (or 'alembic.migration.MigrationContext' is unknown)
E   ImportError: cannot import name 'QtWebKit' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
E   ImportError: cannot import name 'QtWebKitWidgets' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py)
E   ImportError: cannot import name 'QtWebKitWidgets' from 'PyQt5' (/usr/lib64/python3.7/site-packages/PyQt5/__init__.py) (or 'openlp.core.ui.media.webkitplayer.VIDEO_EXT' is unknown)
...

Comment 30 Thomas Capricelli 2019-01-13 22:48:24 UTC
Hello. I'm not a redhat user, but i have a similar problem on gentoo, since very recently. Seeing the comment i think it's probably the same.

My problem only appears on kde/plasma, and is probably related to pyqt5 + plasma-integration. It crashes here (gdb stacktrace):

#28 0x00007ffff7b8e3a1 in dlopen () from /lib64/libdl.so.2
#29 0x00007ffff5992080 in QLibraryPrivate::load_sys (this=0x555556b12a30) at plugin/qlibrary_unix.cpp:197
#30 0x00007ffff598c145 in QLibraryPrivate::load (this=this@entry=0x555556b12a30) at plugin/qlibrary.cpp:549
#31 0x00007ffff598c541 in QLibraryPrivate::loadPlugin (this=this@entry=0x555556b12a30) at plugin/qlibrary.cpp:604
#32 0x00007ffff597dea8 in QFactoryLoader::instance (this=this@entry=0x7ffff23ba060 <_ZZN12_GLOBAL__N_112Q_QGS_loader13innerFunctionEvE6holder>, index=<optimized out>)
    at plugin/qfactoryloader.cpp:317
#33 0x00007ffff1f45e21 in qLoadPlugin<QPlatformTheme, QPlatformThemePlugin, QStringList&> (
    loader=0x7ffff23ba060 <_ZZN12_GLOBAL__N_112Q_QGS_loader13innerFunctionEvE6holder>, key=..., args#0=...)
    at /usr/include/qt5/QtCore/5.11.3/QtCore/private/qfactoryloader_p.h:105
#34 0x00007ffff1f43558 in QPlatformThemeFactory::create (key=..., platformPluginPath=...) at kernel/qplatformthemefactory.cpp:73
#35 0x00007ffff1f4edc8 in init_platform (argv=<optimized out>, argc=@0x7ffff2fe53c0: 0, platformThemeName=..., platformPluginPath=..., pluginNamesWithArguments=...)
    at kernel/qguiapplication.cpp:1228
--Type <RET> for more, q to quit, c to continue without paging--
#36 QGuiApplicationPrivate::createPlatformIntegration (this=0x555556376130) at kernel/qguiapplication.cpp:1373
#37 0x00007ffff1f4ff95 in QGuiApplicationPrivate::createEventDispatcher (this=<optimized out>) at kernel/qguiapplication.cpp:1390
#38 0x00007ffff599ddd9 in QCoreApplicationPrivate::init (this=this@entry=0x555556376130) at kernel/qcoreapplication.cpp:844

(pyqt appears later on)

I know a 'turn around'. On my system, the following two environement variables are set : KDE_FULL_SESSION  XDG_CURRENT_DESKTOP.

Unsetting BOTH of them will allow my pyqt5 apps to start without crashing.

Comparing strace, the difference starts at this line:
    openat(AT_FDCWD, "/usr/lib64/qt5/plugins/platformthemes/KDEPlasmaPlatformTheme.so", O_RDONLY|O_CLOEXEC) = 6
Which belongs to the package plasma-integration.

While the one not-crashing have references to
openat(AT_FDCWD, "/usr/lib64/qt5/plugins/styles", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 8
That do NOT appear on the crashing strace.


That could explain why some of you can reproduce it, and others not. The original reporter clearly uses kde/plasma, as it mentions krash (kde crash handler).

Comment 31 Miro Hrončok 2019-01-21 09:26:16 UTC
I still don't have a full reproducer.

Comment 32 Thomas Capricelli 2019-01-21 12:02:55 UTC
@Miro : i can reproduce the problem any time here, what do you need ?

Comment 33 Miro Hrončok 2019-01-21 12:15:30 UTC
A complete set of commands to reproduce the issue.

So far I have missing dependencies, so the reproducer is not complete.

https://bugzilla.redhat.com/show_bug.cgi?id=1655310#c29

Note that I can get those missing dependencies **somehow**, but I want to get them in the **exact same way** you did.

Comment 34 Thomas Capricelli 2019-01-21 12:45:24 UTC
Ok, you can probably reproduce by using the same soft as I do:

I think the bug is related to pyqt/qt/kde integration, and has nothing to do with mercurial, but this mercurial extension is my easiest way of reproducing the bug:

* install mercurial / pyqt5
* clone the extension somewhere : https://sources.freehackers.org/Hgactivity/
* configure mercurial to use it. In ~/.hgrc, set in [extensions]: activity=<path to previous checkout>
* go in some mercurial repository, for example the Hgactivity clone you just did

There, the following should happen

* "hg activity" -> works, as it doesn't use the pyqt based interface
* "hg activity --mode=gui" -> crash if you are in kde/plasma
* "hg activity --mode=gui" -> doesn't crash, display a pyqt window, if you are outside kde/plasma
* "hg activity --mode=gui" -> doesn't crash if you are in kde/plasma AND you unset/unsetenv BOTH KDE_FULL_SESSION and XDG_CURRENT_DESKTOP

When it crashes, it displays something along:

% hg activity --mode=gui
There are 205 changesets
malloc_consolidate(): invalid chunk size
KCrash: Application 'python2.7' crashing...
Alarm clock


Sometimes "Alarm clock" is replaced by "Suspended" or any other 'signal' related messag. Note that i did nothing more but launching 'hg activity', but it acts exactly as if i had done ^Z or whatever. My guess is that's KDE crash handler 'blocking' the crashed process to try to get a stacktrace.

You can also contact me on irc ('orzel' on freenode) if you want.

Comment 35 Miro Hrončok 2019-01-21 12:50:50 UTC
Plese provide a complete set of commands I can copy paste to a terminal. "install mercurial / pyqt5" is ambiguous - there is infinite number of ways how to do that.

Comment 36 Thomas Capricelli 2019-01-21 13:18:43 UTC
Oh ? Really ? 

I don't know, i'm not a redhat user, but i thought it would be obvious for someone on this thread how to do it with redhat. Isn't it a matter of "rpm -i mercurial pyqt5" ??

Comment 37 Miro Hrončok 2019-01-21 14:36:30 UTC
It's obvious to me how would I install this if I needed it. It's not obvious to me how you did it.

There is no pyqt5 RPM package. There is python3-qt5 etc.

You can also use the Fedora's RPM package, use pip, compile from sources or anything else.

Comment 38 Thomas Capricelli 2019-01-21 14:48:05 UTC
Well, as said...  i'm no redhat user.

I use the standard gentoo package (emerge mercurial pyqt5). But i'm pretty sure it's independant of the package.

Comment 39 Miro Hrončok 2019-01-21 14:58:29 UTC
In that case it should be reported to upstream pyqt5, not to a distribution Fedora maintainers.

I really try to help but I'm afraid this is beyond my ability.

Comment 40 Miro Hrončok 2019-02-03 19:47:36 UTC
Does anyone have a complete reproducer?

Comment 41 Thomas Capricelli 2019-02-03 22:13:26 UTC
I do, it's explained there.
You can probably reproduce in few minutes following my instructions.

Comment 42 Miro Hrončok 2019-02-03 23:25:18 UTC
Do you have a complete set of commands I need to run on a Fedora system to reproduce the bug?

If you have a Gentoo reproducer, please report it to Gentoo bug tracker.

Comment 43 Petr Viktorin (pviktori) 2019-02-04 13:15:49 UTC
This bug is not moving forward.
Please re-open if you have a Fedora reproducer – that is, a full list of packages to install and commands to run.
We haven't been able to reproduce the bug without that.


Note You need to log in before you can comment on or make changes to this bug.