Bug 2337089 - dlib: tests crash on aarch64
Summary: dlib: tests crash on aarch64
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dlib
Version: 42
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Onuralp SEZER
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2025-01-11 12:11 UTC by Ben Beasley
Modified: 2025-03-28 15:05 UTC (History)
4 users (show)

Fixed In Version: dlib-19.24.8-3.fc42 dlib-19.24.8-3.fc40 dlib-19.24.8-3.fc41
Clone Of:
Environment:
Last Closed: 2025-03-24 01:32:47 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ben Beasley 2025-01-11 12:11:32 UTC
Description of problem:

The dtest executable crashes on aarch64.

Version-Release number of selected component (if applicable): dlib-19.24.6-3.fc42


How reproducible:

Build the package on aarch64.


Actual results:

+ ./dtest --runall --no_test_ffmpeg
Running test_matrix3 \ double free or corruption (!prev)
qemu: uncaught target signal 6 (Aborted) - core dumped

(The above is in emulation; this happens on real hardware, too.)


Expected results:

All tests pass.


Additional info:

First failing Koschei build: https://koschei.fedoraproject.org/build/19123027

Comment 1 Ben Beasley 2025-01-11 12:13:02 UTC
(gdb) run
Starting program: /builddir/build/BUILD/dlib-19.24.6-build/dlib-19.24.6/dlib/test/redhat-linux-build/dtest --runall --no_test_ffmpeg
warning: Could not trace the inferior process.
warning: ptrace: Function not implemented
During startup program exited with code 127.

It looks like I can only debug this on real hardware.

Comment 2 Sandro 2025-01-11 12:28:12 UTC
No issues building `dlib` in Copr. The hardware according to `hw_info.log` differs slightly (stepping, flags).

We are also tweaking the parallelism in the spec based on memory available. On the Koji builders this results in `%global _smp_mflags "-j5"`, whereas in Copr it's `-j1`. I could try just setting it to `-j1` for `aarch64` regardless of memory available. But I'm not sure it's actually related.

Comment 3 Sandro 2025-01-11 21:48:18 UTC
The good news, if you want to call it that, is: only `test_matrix3` fails on aarch64 and it fails consistently. When I run only that test with full logging enabled, the log shows the following entries before the test explodes:

   31 TRACE [0] test.matrix3:  
big double
   33 TRACE [0] test.matrix3: 1.1
   41 TRACE [0] test.matrix3: 1.2
   45 TRACE [0] test.matrix3: 1.3
   49 TRACE [0] test.matrix3: 1.4
   52 TRACE [0] test.matrix3: 1.5
   58 TRACE [0] test.matrix3: 1.5.1
   67 TRACE [0] test.matrix3: 1.5.2
   73 TRACE [0] test.matrix3: 1.5.3
   83 TRACE [0] test.matrix3: 1.6
   88 TRACE [0] test.matrix3: 1.6.1
   97 TRACE [0] test.matrix3: 1.6.2
  102 TRACE [0] test.matrix3: 1.6.2.1
  107 TRACE [0] test.matrix3: 1.6.3
  118 TRACE [0] test.matrix3: 1.7

The bad news: the output is not very helpful. I looked at the code of the test. Right after "big double" is logged, it calls `test_blas<double>(120,131)`. That function mentions something with regards to matrices and references `matrix.h`[1] and after "1.7" is printed I see two function calls [2]. That's where my expertise ends. Maybe that helps. But a proper backtrace would of course be better.

[1] https://github.com/davisking/dlib/blob/master/dlib/matrix/matrix.h
[2] https://github.com/davisking/dlib/blob/daefd41b9fff274ca9e70e9b5f65d25a9799ca96/dlib/test/matrix3.cpp#L211C1-L212C118

Comment 4 Aoife Moloney 2025-02-26 13:22:32 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle.
Changing version to 42.

Comment 5 Fedora Update System 2025-03-19 09:38:40 UTC
FEDORA-2025-118d5710f8 (dlib-19.24.8-3.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-118d5710f8

Comment 6 Fedora Update System 2025-03-19 09:38:53 UTC
FEDORA-2025-3ac216fc54 (dlib-19.24.8-3.fc40) has been submitted as an update to Fedora 40.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-3ac216fc54

Comment 7 Fedora Update System 2025-03-19 09:39:05 UTC
FEDORA-2025-f045e7d0ee (dlib-19.24.8-3.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-f045e7d0ee

Comment 8 Fedora Update System 2025-03-20 06:12:36 UTC
FEDORA-2025-f045e7d0ee has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-f045e7d0ee`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-f045e7d0ee

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2025-03-20 06:31:41 UTC
FEDORA-2025-118d5710f8 has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-118d5710f8`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-118d5710f8

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2025-03-20 06:50:18 UTC
FEDORA-2025-3ac216fc54 has been pushed to the Fedora 40 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-3ac216fc54`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-3ac216fc54

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2025-03-24 01:32:47 UTC
FEDORA-2025-f045e7d0ee (dlib-19.24.8-3.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2025-03-28 14:48:29 UTC
FEDORA-2025-3ac216fc54 (dlib-19.24.8-3.fc40) has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2025-03-28 15:05:11 UTC
FEDORA-2025-118d5710f8 (dlib-19.24.8-3.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.