Bug 1128293

Summary: System clucene09 should be built with LUCENE_ENABLE_REFCOUNT (crash in clucene when QtCreator indexes the help)
Product: [Fedora] Fedora Reporter: Loïc Yhuel <loic.yhuel>
Component: clucene09Assignee: Kevin Kofler <kevin>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: urgent    
Version: 21CC: jgrulich, jreznik, kevin, ltinkl, rdieter, redhat-bugzilla, rnovacek, than
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qt-4.8.6-18.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-12-13 09:41:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Loïc Yhuel 2014-08-08 19:55:20 UTC
Description of problem:
QtCreator crashes when indexing the help files

Version-Release number of selected component (if applicable):
qt-creator-3.2.0-0.3.rc1.fc21.x86_64
qt5-qttools-5.3.1-1.fc21.x86_64
clucene09-core-0.9.21b-9.fc21.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Launch QtCreator
2. Switch to Help mode
3. In the sidebar, choose "Search" instead of "Index" or "Contents"

Actual results:
QtCreator starts to index the help files and crashes

Additional info:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff71bfb700 (LWP 6845)]
lucene::index::IndexReader::~IndexReader (this=0x7fff541cbca0, __in_chrg=<optimized out>) at ../src/CLucene/index/IndexReader.cpp:56
56                writeLock->release();
(gdb) bt
#0  0x00007fffece7e8d4 in lucene::index::IndexReader::~IndexReader() (this=0x7fff541cbca0, __in_chrg=<optimized out>) at ../src/CLucene/index/IndexReader.cpp:56
#1  0x00007fffece8b2b9 in lucene::index::SegmentReader::~SegmentReader() (this=0x7fff541cbca0, __in_chrg=<optimized out>) at ../src/CLucene/index/SegmentReader.cpp:213
#2  0x00007fffed1105e4 in QCLuceneIndexReader::~QCLuceneIndexReader() (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/qt5/QtCore/qshareddata.h:87
#3  0x00007fffed1105e4 in QCLuceneIndexReader::~QCLuceneIndexReader() (this=0xffffffff, __in_chrg=<optimized out>) at qindexreader.cpp:53
#4  0x00007fffed5ca95d in fulltextsearch::clucene::QHelpSearchIndexWriter::removeDocuments(QString const&, QString const&) (this=this@entry=0x3531750, indexPath=..., namespaceName=...)
    at qhelpsearchindexwriter_clucene.cpp:841
#5  0x00007fffed5cbf01 in fulltextsearch::clucene::QHelpSearchIndexWriter::run() (this=0x3531750) at qhelpsearchindexwriter_clucene.cpp:701
#6  0x0000003c7bc9cb0e in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
#7  0x0000003be9a075fa in start_thread (arg=0x7fff71bfb700) at pthread_create.c:310
#8  0x0000003be970507d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) p writeLock
$1 = (lucene::store::LuceneLock *) 0xffffffff

Where writeLock is changed :
Old value = 0
New value = -1
QCLuceneIndexReaderPrivate::~QCLuceneIndexReaderPrivate (this=<optimized out>, __in_chrg=<optimized out>) at qindexreader.cpp:43
43              _CLDECDELETE(reader);
(gdb) bt
#0  0x00007fffed11059a in QCLuceneIndexReaderPrivate::~QCLuceneIndexReaderPrivate() (this=<optimized out>, __in_chrg=<optimized out>) at qindexreader.cpp:43
#1  0x00007fffed1105e4 in QCLuceneIndexReader::~QCLuceneIndexReader() (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/qt5/QtCore/qshareddata.h:87
#2  0x00007fffed1105e4 in QCLuceneIndexReader::~QCLuceneIndexReader() (this=0x7fff4c1cbca0, __in_chrg=<optimized out>) at qindexreader.cpp:53
#3  0x00007fffed5ca95d in fulltextsearch::clucene::QHelpSearchIndexWriter::removeDocuments(QString const&, QString const&) (this=this@entry=0x35af0f0, indexPath=..., namespaceName=...)
    at qhelpsearchindexwriter_clucene.cpp:841
#4  0x00007fffed5cbf01 in fulltextsearch::clucene::QHelpSearchIndexWriter::run() (this=0x35af0f0) at qhelpsearchindexwriter_clucene.cpp:701
#5  0x0000003c7bc9cb0e in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
#6  0x0000003be9a075fa in start_thread (arg=0x7fff69bfb700) at pthread_create.c:310
#7  0x0000003be970507d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109


This is due to qttools-opensource-src-5.3.0-system-clucene.patch :
The patch defines LUCENE_ENABLE_REFCOUNT, which allows to build since Qt needs a refcounted clucene.
But the system clucene is built without this flag (see /usr/include/clucene09/CLucene/CLConfig.h), and it changes the clucene ABI (classes like IndexReader inherit from LuceneBase or LuceneVoidBase depending on this flag).

So the patch should either be dropped, or should modify Qt to work without LUCENE_ENABLE_REFCOUNT (It would need to do the refcounting in Qt, but I don't know if it would be enough since the feature might be used to extend clucene classes lifetime, or to forget them earlier on the Qt side).

Comment 1 Rex Dieter 2014-10-16 19:12:43 UTC
Thanks for your bug report and analysis, we're looking into it.

Comment 2 Kevin Kofler 2014-10-16 20:24:30 UTC
Reassigning, see the original post for details.

This issue would also affect the Qt 4 Assistant, the only reason it currently doesn't is that Qt 4 is being built against a bundled CLucene. Thus, this issue also blocks fixing that bundling issue.

Users of CLucene that don't need the reference counting shouldn't be negatively affected by it, except maybe a minimal performance decrease (one addition added to some memory allocations, one subtraction to some memory unallocations, probably hardly measurable).

The only other packages using clucene09 in Fedora are zarafa-indexer and zarafa-search, so it should also be possible to push a grouped update with this change and rebuilds of qt5-qttools, zarafa-indexer and zarafa-search to F20 and F19.

Comment 3 Kevin Kofler 2014-10-25 16:13:17 UTC
Ping? We really need this change for the Qt Assistant.

Comment 4 Robert Scheck 2014-10-25 16:30:56 UTC
Kevin, feel free to perform the necessary changes yourself.

Comment 5 Kevin Kofler 2014-10-25 16:53:53 UTC
OK, looking into it.

Comment 6 Kevin Kofler 2014-10-25 20:14:45 UTC
So, I finally have something in Rawhide (it took a bit longer than expected because I had to modify the gcc48 patch and sort out the soname), but it failed to build because of a testsuite failure on x86_64 that's not even 100% reproducible (in scratch builds, I had 1 failure and 1 success, the official build failed again).

Comment 7 Kevin Kofler 2014-10-25 20:47:27 UTC
1) testEscaped: FAILED Query /\\\[brackets/ yielded /\[baackets/, expecting /\[brackets/

Comment 8 Robert Scheck 2014-10-25 21:05:16 UTC
From my tests, builds on ix86 in VMs succeed, builds on HW fail?!

Comment 9 Kevin Kofler 2014-10-25 22:13:57 UTC
The failure seems to be finally fixed now. I found a total of 3 (!) similar defects only in the area of the failing test: 2 buffers whose 0-termination wasn't guaranteed and one strcpy with overlapping destination and source. That strcpy (_tcscpy actually) was apparently the source of the failure, I changed it to a memmove.

Comment 10 Robert Scheck 2014-11-07 21:28:12 UTC
Kevin, I meanwhile run some tests with the new zarafa-search that is built
against the changed clucene09 - and I did not notice any regressions (just
to follow up our IRC conversation).

Comment 11 Fedora Update System 2014-12-04 23:41:34 UTC
zarafa-7.1.11-2.fc21, qt-4.8.6-18.fc21, clucene09-0.9.21b-13.fc21, qt5-qttools-5.3.2-4.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/qt5-qttools-5.3.2-4.fc21,zarafa-7.1.11-2.fc21,qt-4.8.6-18.fc21,clucene09-0.9.21b-13.fc21

Comment 12 Fedora Update System 2014-12-04 23:47:06 UTC
qt-4.8.6-18.fc20, zarafa-7.1.11-2.fc20, clucene09-0.9.21b-13.fc20, qt5-qttools-5.3.2-4.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/qt5-qttools-5.3.2-4.fc20,qt-4.8.6-18.fc20,zarafa-7.1.11-2.fc20,clucene09-0.9.21b-13.fc20

Comment 13 Fedora Update System 2014-12-05 00:11:57 UTC
qt5-qttools-5.3.2-4.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/qt5-qttools-5.3.2-4.fc19

Comment 14 Kevin Kofler 2014-12-05 00:19:30 UTC
As explained in the update notes, we decided to just go with the bundled clucene09 on Fedora 19, to avoid breaking a library ABI that late in the life cycle.

Comment 15 Fedora Update System 2014-12-06 02:28:11 UTC
Package qt-4.8.6-18.fc20, zarafa-7.1.11-2.fc20, clucene09-0.9.21b-13.fc20, qt5-qttools-5.3.2-4.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing qt-4.8.6-18.fc20 zarafa-7.1.11-2.fc20 clucene09-0.9.21b-13.fc20 qt5-qttools-5.3.2-4.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-16339/qt5-qttools-5.3.2-4.fc20,qt-4.8.6-18.fc20,zarafa-7.1.11-2.fc20,clucene09-0.9.21b-13.fc20
then log in and leave karma (feedback).

Comment 16 Fedora Update System 2014-12-13 09:41:09 UTC
qt-4.8.6-18.fc20, zarafa-7.1.11-2.fc20, clucene09-0.9.21b-13.fc20, qt5-qttools-5.3.2-4.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2014-12-13 09:52:00 UTC
zarafa-7.1.11-2.fc21, qt-4.8.6-18.fc21, clucene09-0.9.21b-13.fc21, qt5-qttools-5.3.2-4.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2014-12-15 04:31:38 UTC
qt5-qttools-5.3.2-4.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2014-12-15 13:03:52 UTC
clucene09-0.9.21b-13.el7, zarafa-7.1.11-2.el7, qt5-qtbase-5.4.0-2.el7, qt5-qtdeclarative-5.4.0-1.el7, qt5-qtgraphicaleffects-5.4.0-1.el7, qt5-qtlocation-5.4.0-1.el7, qt5-qtquick1-5.4.0-1.el7, qt5-qtscript-5.4.0-1.el7, qt5-qtserialport-5.4.0-1.el7, qt5-qttools-5.4.0-1.el7, qt5-qtwebkit-5.4.0-1.el7, qt5-qtxmlpatterns-5.4.0-1.el7, qt5-qtconnectivity-5.4.0-1.el7, qt5-qtdoc-5.4.0-1.el7, qt5-qtimageformats-5.4.0-1.el7, qt5-qtmultimedia-5.4.0-1.el7, qt5-qtquickcontrols-5.4.0-1.el7, qt5-qtsensors-5.4.0-1.el7, qt5-qtsvg-5.4.0-1.el7, qt5-qttranslations-5.4.0-1.el7, qt5-qtx11extras-5.4.0-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-4632/zarafa-7.1.11-2.el7,clucene09-0.9.21b-13.el7,qt5-qtbase-5.4.0-2.el7,qt5-qtconnectivity-5.4.0-1.el7,qt5-qtdeclarative-5.4.0-1.el7,qt5-qtdoc-5.4.0-1.el7,qt5-qtgraphicaleffects-5.4.0-1.el7,qt5-qtimageformats-5.4.0-1.el7,qt5-qtlocation-5.4.0-1.el7,qt5-qtmultimedia-5.4.0-1.el7,qt5-qtquick1-5.4.0-1.el7,qt5-qtquickcontrols-5.4.0-1.el7,qt5-qtscript-5.4.0-1.el7,qt5-qtsensors-5.4.0-1.el7,qt5-qtserialport-5.4.0-1.el7,qt5-qtsvg-5.4.0-1.el7,qt5-qttools-5.4.0-1.el7,qt5-qttranslations-5.4.0-1.el7,qt5-qtwebkit-5.4.0-1.el7,qt5-qtx11extras-5.4.0-1.el7,qt5-qtxmlpatterns-5.4.0-1.el7

Comment 20 Fedora Update System 2014-12-31 23:07:48 UTC
clucene09-0.9.21b-13.el7, zarafa-7.1.11-2.el7, qt5-qtbase-5.4.0-2.el7, qt5-qtdeclarative-5.4.0-1.el7, qt5-qtgraphicaleffects-5.4.0-1.el7, qt5-qtlocation-5.4.0-1.el7, qt5-qtquick1-5.4.0-1.el7, qt5-qtscript-5.4.0-1.el7, qt5-qtserialport-5.4.0-1.el7, qt5-qttools-5.4.0-1.el7, qt5-qtwebkit-5.4.0-1.el7, qt5-qtxmlpatterns-5.4.0-1.el7, qt5-qtconnectivity-5.4.0-1.el7, qt5-qtdoc-5.4.0-1.el7, qt5-qtimageformats-5.4.0-1.el7, qt5-qtmultimedia-5.4.0-1.el7, qt5-qtquickcontrols-5.4.0-1.el7, qt5-qtsensors-5.4.0-1.el7, qt5-qtsvg-5.4.0-1.el7, qt5-qttranslations-5.4.0-1.el7, qt5-qtx11extras-5.4.0-1.el7 has been pushed to the Fedora EPEL 7 stable repository.  If problems still persist, please make note of it in this bug report.