Bug 906367 - FTBFS: 93% tests passed, 3 tests failed out of 45
Summary: FTBFS: 93% tests passed, 3 tests failed out of 45
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: mariadb
Version: 19
Hardware: powerpc
OS: Linux
high
high
Target Milestone: ---
Assignee: Honza Horak
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 907430
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-01-31 14:15 UTC by Karsten Hopp
Modified: 2014-01-08 08:51 UTC (History)
3 users (show)

Fixed In Version: mariadb-5.5.34-3.fc19
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-08 08:51:48 UTC
Type: Bug


Attachments (Terms of Use)
log of failing tests (53.52 KB, text/plain)
2013-02-07 14:24 UTC, Karsten Hopp
no flags Details

Description Karsten Hopp 2013-01-31 14:15:57 UTC
Description of problem:
      Start 10: my_atomic
10/45 Test #10: my_atomic ........................***Failed    0.02 sec
...
      Start 12: lf
12/45 Test #12: lf ...............................***Failed    0.08 sec
...
      Start 20: trnman
20/45 Test #20: trnman ...........................***Failed    0.24 sec
93% tests passed, 3 tests failed out of 45
Total Test time (real) =  27.84 sec
The following tests FAILED:
	 10 - my_atomic (Failed)
	 12 - lf (Failed)
	 20 - trnman (Failed)
Errors while running CTest
make: *** [test] Error 8

Version-Release number of selected component (if applicable):
mariadb-5.5.28a-6.fc19

How reproducible:
always

Steps to Reproduce:
1. ppc-koji build --scratch f19 mariadb-5.5.28a-6.fc19.src.rpm
2.
3.
  
Actual results:
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=886935

Expected results:


Additional info:

Comment 1 Honza Horak 2013-02-01 14:33:51 UTC
I tried to reproduce that on F18 because I don't have a working F19 machine and found that it worked. However, it started to fail after upgrading the following packages to F19 builds:

Updated: libgcc-4.8.0-0.4.fc19.ppc64
Updated: glibc-2.17-1.fc19.ppc64
Updated: glibc-common-2.17-1.fc19.ppc64
Installed: libmpc-0.9-3.fc18.2.ppc64
Updated: libstdc++-4.8.0-0.4.fc19.ppc64
Updated: libstdc++-devel-4.8.0-0.4.fc19.ppc64
Updated: cpp-4.8.0-0.4.fc19.ppc64
Updated: libgomp-4.8.0-0.4.fc19.ppc64
Updated: glibc-headers-2.17-1.fc19.ppc64
Updated: glibc-devel-2.17-1.fc19.ppc64
Updated: gcc-4.8.0-0.4.fc19.ppc64
Updated: libtool-2.4.2-12.fc19.ppc64
Updated: gcc-c++-4.8.0-0.4.fc19.ppc64

So I tried to run the test cases built with gcc-4.7.2-8 on system with upgraded packages above and it worked. From that observation I'd say it is probably a bug in gcc, but I haven't managed to create a minimal reproducer.

Comment 2 Honza Horak 2013-02-04 11:32:06 UTC
Reported against gcc: bug #907430

Comment 3 Jakub Jelinek 2013-02-04 13:22:08 UTC
If you build mariadb with gcc 4.8 and just rebuild the problematic tests with older gcc, does it still fail?  I.e. is the problem in mysql libraries/binaries, or just in the unit test binaries?

Comment 4 Honza Horak 2013-02-04 15:55:35 UTC
Issue consulted with Jakub in private channel, he already has a reproducer, confirmed a gcc bug.

Comment 5 Jakub Jelinek 2013-02-04 16:11:18 UTC
Tracking that upstream now: http://gcc.gnu.org/PR56205

Comment 6 Karsten Hopp 2013-02-07 14:23:41 UTC
With the new gcc the self checks now proceed where they previously failed, only to fail later on at other checks:
main.query_cache                         [ fail ]
.
.
.
main.gis-precise                         [ fail ]
.
.
.
main.myisampack                          [ fail ]
.
.
.
sys_vars.query_cache_size_basic          [ fail ]


I'll attach the relevant parts of the build.log from 
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=894885

Comment 7 Karsten Hopp 2013-02-07 14:24:16 UTC
Created attachment 694517 [details]
log of failing tests

Comment 8 Jakub Jelinek 2013-02-07 18:19:06 UTC
Does it reproduce even when built with -O0?  If not, can you bisect it which *.o file matters?

Comment 9 Honza Horak 2013-02-08 08:26:38 UTC
Some of these tests are also failing on s390, reported as bug #906746. I'm not sure if these are still caused by gcc.

Comment 10 Honza Horak 2013-02-08 15:02:37 UTC
While looking into that issues I've created a summary of failing tests on different architectures, compared with mysql:

                       mysql   mariadb mariadb mariadb mariadb mariadb
test case              *       ppc     ppc64   s390    s390x   x86_64
gis-precise            N/A     fail    fail    fail    fail    pass
query_cache_size_basic disable fail    pass    ?       pass    pass
query_cache            pass    fail    pass    fail    pass    pass
index_merge_myisam     pass    pass    pass    fail    pass    pass
myisampack             pass    fail    pass    ?       pass    pass

Comment 11 Honza Horak 2013-02-08 16:01:32 UTC
gis-precise test failure reported upstream: 
https://mariadb.atlassian.net/browse/MDEV-4153

I'll try to look more closely on the other test cases and will report them after that.

Comment 12 Honza Horak 2013-02-08 20:51:52 UTC
query_cache and query_cache_size_basic are probably caused by the same reason, which seems more like a bug in code than a bug in test case. Reported upstream:
https://mariadb.atlassian.net/browse/MDEV-4156

Comment 13 Karsten Hopp 2013-02-08 21:51:44 UTC
Would you mind skipping those tests on ppc and %{power64} until this gets sorted out upstream ?
That would allow us to keep up with the primary arches for the upcoming mass rebuild.

Comment 14 Honza Horak 2013-02-12 15:45:18 UTC
I think skipping them is the only thing we can do right now. I've added problematic ones into the set of skipped tests, but then another failure appeared -- some expected server warnings are not stripped from output and tests have false negative results in the end (reported upstream as https://mariadb.atlassian.net/browse/MDEV-4169).

We should be able to suppress this by specifying --nowarnings option of mysql-test-run -- I'm testing that currently and will try to commit that hopefully today.

Comment 15 Honza Horak 2013-02-12 18:23:58 UTC
Changes are in git and I'm waiting for the scratch build keeping fingers crossed:
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=904328

Comment 16 Honza Horak 2013-02-14 18:03:57 UTC
The previous one didn't succeeded, but this one did:
https://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=906812

Comment 17 Fedora End Of Life 2013-04-03 19:49:19 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Comment 18 Honza Horak 2014-01-08 08:51:48 UTC
It seems to be resolved by fixing gcc, since this F19 build is fine now:
https://ppc.koji.fedoraproject.org/koji/buildinfo?buildID=214139

Thus closing.


Note You need to log in before you can comment on or make changes to this bug.