Bug 2244926

Summary:	mmlib: FTBFS in Fedora Rawhide
Product:	[Fedora] Fedora	Reporter:	Sandro <gui1ty>
Component:	mmlib	Assignee:	Ben Beasley <code>
Status:	CLOSED ERRATA	QA Contact:
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	code, neuro-sig
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Linux
URL:	https://koschei.fedoraproject.org/package/mmlib
Whiteboard:
Fixed In Version:	mmlib-1.4.2-10.fc40 mmlib-1.4.2-7.el9 mmlib-1.4.2-8.fc38 mmlib-1.4.2-8.fc37 mmlib-1.4.2-8.el8 mmlib-1.4.2-10.fc39 mmlib-1.4.2-13.el10_0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-10-19 17:19:12 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sandro 2023-10-18 20:52:16 UTC

Description of problem:
Package mmlib fails to build from source in Fedora Rawhide.

Version-Release number of selected component (if applicable):
1.4.2-5.fc39

Steps to Reproduce:
koji build --scratch f40 mmlib-1.4.2-5.fc39.src.rpm

Additional info:
This package is tracked by Koschei. See:
https://koschei.fedoraproject.org/package/mmlib

Reproducible: Always




This happens on all Fedora branches except F37. I haven't checked the EPEL builds.

The build errors with:

+ /usr/bin/meson test -C redhat-linux-build --num-processes 6 --print-errorlogs -- --num-processes 1
ERROR: *:--num-processes test name does not match any test

The relevant line in the spec file is:

%meson_test -- --num-processes 1

I tried changing that to:

a) %meson_test --num-processes 1
b) /usr/bin/meson test -C redhat-linux-build --num-processes 1 --print-errorlogs

In both cases one of the test fails:

――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
Listing only the last 100 lines from a long log.
18/10/23 22:41:08 ERROR mmlib            : write(8, ...) failed ; Bad file descriptor (mm_write() in ../src/file-posix.c:257)
failed write for fd=8
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "./process-data/file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "./process-data/file-unkfmt" ; Exec format error (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "./process-data/does-not-exists" ; No such file or directory (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-unkfmt" ; Exec format error (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/does-not-exists" ; No such file or directory (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "does-not-exists" ; No such file or directory (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-unkfmt" ; Exec format error (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            :  ; No such file or directory (spawn_daemon() in ../src/process-posix.c:404)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/process-data/file-unkfmt" ; Exec format error (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            :  ; No such file or directory (spawn_daemon() in ../src/process-posix.c:404)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "file-noexec" ; Permission denied (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : Cannot run "does-not-exists" ; No such file or directory (load_new_proc_img() in ../src/process-posix.c:242)
18/10/23 22:41:08 ERROR mmlib            : file must not be NULL (mm_spawn() in ../src/process-posix.c:527)
18/10/23 22:41:08 ERROR mmlib            : file must not be NULL (mm_spawn() in ../src/process-posix.c:527)
18/10/23 22:41:08 ERROR mmlib            : file must not be NULL (mm_spawn() in ../src/process-posix.c:527)
18/10/23 22:41:08 ERROR mmlib            : file must not be NULL (mm_spawn() in ../src/process-posix.c:527)
18/10/23 22:41:08 ERROR mmlib            : Invalid flags (00000008) (mm_spawn() in ../src/process-posix.c:530)
map_fd = [ 3:7 4:8 5:9 ]
/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/child-proc
check-open-files
3
num_fd = 3
18/10/23 22:41:08 ERROR mmlib            : write(7, ...) failed ; Bad file descriptor (mm_write() in ../src/file-posix.c:257)
failed write for fd=7
18/10/23 22:41:08 ERROR mmlib            : write(8, ...) failed ; Bad file descriptor (mm_write() in ../src/file-posix.c:257)
failed write for fd=8
18/10/23 22:41:08 ERROR mmlib            : waitpid(1591) failed ; No child processes (mm_wait_process() in ../src/process-posix.c:642)
map_fd = [ 3:7 4:8 5:9 ]
18/10/23 22:41:08 ERROR mmlib            : unable to fork ; Resource temporarily unavailable (spawn_child() in ../src/process-posix.c:326)
spawn_child() failed: unable to fork ; Resource temporarily unavailablemap_fd = [ 3:7 4:8 5:9 ]
18/10/23 22:41:08 ERROR mmlib            : unable to do first fork ; Resource temporarily unavailable (spawn_daemon() in ../src/process-posix.c:378)
spawn_child() failed: unable to do first fork ; Resource temporarily unavailable18/10/23 22:41:08 ERROR mmlib            : unable to get /builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/completion_testdir filetype ; No such file or directory (mm_remove() in ../src/file-posix.c:903)
18/10/23 22:41:08 ERROR mmlib            : open(/builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/tests/completion_testdir/emptydir/, 00080041) failed ; Is a directory (mm_open() in ../src/file-posix.c:98)
Unsupported option -k
Use -h or --help to display usage.
Option -i|--set-i needs value.
Use -h or --help to display usage.
Option -i|--set-i needs value.
Use -h or --help to display usage.
Unsupported option ---set-ll
Use -h or --help to display usage.
Unsupported option --unknown-opt
Use -h or --help to display usage.
Unsupported option --unknown-opt
Use -h or --help to display usage.
Unsupported option --unknown-opt
Use -h or --help to display usage.
Option -i|--set-i accepting int value type has received an invalid value "not_a_number" (Invalid argument)
Use -h or --help to display usage.
Option -i|--set-i accepting int value type has received an invalid value "21_noise" (Invalid argument)
Use -h or --help to display usage.
Option -i|--set-i accepting int value type has received an invalid value "2147483648" (Numerical result out of range)
Use -h or --help to display usage.
Option -i|--set-i accepting int value type has received an invalid value "-2147483649" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ui accepting unsigned int value type has received an invalid value "4294967296" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ui accepting unsigned int value type has received an invalid value "-1" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ll accepting long long value type has received an invalid value "9223372036854775808" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ll accepting long long value type has received an invalid value "-9223372036854775809" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ull accepting unsigned long long value type has received an invalid value "18446744073709551616" (Numerical result out of range)
Use -h or --help to display usage.
Option --set-ull accepting unsigned long long value type has received an invalid value "-1" (Numerical result out of range)
Use -h or --help to display usage.
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-5) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-4) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-3) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-2) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-1) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (5) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (6) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (7) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (8) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (9) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-5) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-4) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-3) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-2) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (-1) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : suffix cannot be null (mm_path_from_basedir() in ../src/utils.c:316)
18/10/23 22:41:08 ERROR mmlib            : suffix cannot be null (mm_path_from_basedir() in ../src/utils.c:316)
18/10/23 22:41:08 ERROR mmlib            : suffix cannot be null (mm_path_from_basedir() in ../src/utils.c:316)
18/10/23 22:41:08 ERROR mmlib            : suffix cannot be null (mm_path_from_basedir() in ../src/utils.c:316)
18/10/23 22:41:08 ERROR mmlib            : suffix cannot be null (mm_path_from_basedir() in ../src/utils.c:316)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (5) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (6) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (7) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (8) (mm_get_basedir() in ../src/utils.c:289)
18/10/23 22:41:08 ERROR mmlib            : Unknown dir type (9) (mm_get_basedir() in ../src/utils.c:289)
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――


Summary of Failures:

5/5 unit api tests FAIL            11.72s   418/421 subtests passed

Ok:                 4   
Expected Fail:      0   
Fail:               1   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /builddir/build/BUILD/mmlib-1.4.2/redhat-linux-build/meson-logs/testlog.txt

Prime suspect appears to be the meson update from 1.0.1 to 1.2.1: https://koschei.fedoraproject.org/affected-by/meson?epoch1=0&version1=1.0.1&release1=1.fc38&epoch2=0&version2=1.2.1&release2=1.fc38&collection=f38

Comment 1 Sandro 2023-10-18 20:55:31 UTC

> Prime suspect appears to be the meson update from 1.0.1 to 1.2.1: https://koschei.fedoraproject.org/affected-by/meson?epoch1=0&version1=1.0.1&release1=1.fc38&epoch2=0&version2=1.2.1&release2=1.fc38&collection=f38

Prime suspect for `%meson_test -- --num-processes 1` no longer working, that is. I haven't got a clue about the test failure.

Comment 2 Ben Beasley 2023-10-19 10:56:24 UTC

For the meson error, I found that

-%meson_test -- --num-processes 1
+%global _smp_build_ncpus 1
+%meson_test

works nicely.

-----

The failing test says:

▶ 5/5 - ../tests/socket-api-tests.c:socket:getaddrinfo_valid: Assertion 'rp->ai_socktype == exp_socktype' failed FAIL
▶ 5/5 - ../tests/socket-api-tests.c:socket:getaddrinfo_error: Assertion 'mm_get_lasterror_number() == 1010' failed: mm_get_lasterror_number() == 11, 1010 == 1010 FAIL
▶ 5/5 - ../tests/socket-api-tests.c:socket:create_invalid_sockclient: Assertion 'mm_create_sockclient("ssh://localhost:10") == -1' failed FAIL

A first guess was that that might be related to the offline build, but I see the same thing when I do a mock build with --enable-network.

Looking at F38 to winnow down the updated packages a bit, the first failing Koschei build is:

https://koschei.fedoraproject.org/build/16384749

It’s not immediately obvious what would be responsible.

Comment 3 Ben Beasley 2023-10-19 10:59:35 UTC

This reproduces using meson directly in a git checkout, so I’ll report it upstream.

Comment 4 Ben Beasley 2023-10-19 12:12:50 UTC

Reported upstream in https://github.com/mmlabs-mindmaze/mmlib/issues/7 with detailed analysis.

Partial fix offered in https://github.com/mmlabs-mindmaze/mmlib/pull/8; some discussion required for the third test failure, https://github.com/mmlabs-mindmaze/mmlib/issues/7#issuecomment-1770809321.

Preparing to build a package with the meson fix, the PR for the first two failing tests, and the workaround for the third failing test.

Comment 5 Ben Beasley 2023-10-19 16:18:07 UTC

Currently working through and reporting/skipping additional architecture-dependent test failures.

It’s amazing that this was building cleanly and then suddenly broke in multiple ways on all releases!

Comment 6 Ben Beasley 2023-10-19 17:10:39 UTC

I’m skipping the wait_signal test on ppc64le/aarch64 for https://github.com/mmlabs-mindmaze/mmlib/issues/9.

I find that I still have occasional flaky timing-related failures on s390x:

4/5 internal tests OK               0.00s   1 subtests passed
▶ 5/5 - ../tests/file-api-tests.c:file:file_times: Assertion 'buf.atime == ts3.tv_sec' failed: buf.atime == 2147483647, ts3.tv_sec == 2239999890 FAIL          
▶ 5/5 - ../tests/file-api-tests.c:file:file_times_now: Assertion 'buf.atime == ts3.tv_sec' failed: buf.atime == 2147483647, ts3.tv_sec == 2239999890 FAIL          
▶ 5/5 - ../tests/file-api-tests.c:file:file_fd_times: Assertion 'buf.atime == ts3.tv_sec' failed: buf.atime == 2147483647, ts3.tv_sec == 2239999890 FAIL          
▶ 5/5 - ../tests/file-api-tests.c:file:file_fd_times_now: Assertion 'buf.atime == ts3.tv_sec' failed: buf.atime == 2147483647, ts3.tv_sec == 2239999890 FAIL          

Plus I saw this once on ppc64le:

▶ 5/5 - ../tests/process-api-tests.c:process:spawn_error: Test timeout expired FAIL          

I’m not going to dig into any of those right now. I believe they’re all due to bad assumptions in the tests rather than real/serious problems.

Comment 7 Fedora Update System 2023-10-19 17:16:42 UTC

FEDORA-2023-77fb2cdaf1 has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2023-77fb2cdaf1

Comment 8 Fedora Update System 2023-10-19 17:19:12 UTC

FEDORA-2023-77fb2cdaf1 has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2023-10-19 21:06:19 UTC

FEDORA-2023-aa62881cba has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-aa62881cba

Comment 10 Fedora Update System 2023-10-20 02:13:38 UTC

FEDORA-2023-aa62881cba has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-aa62881cba`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-aa62881cba

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2023-10-20 11:44:48 UTC

FEDORA-2023-587b305c8a has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-587b305c8a

Comment 12 Fedora Update System 2023-10-20 14:01:49 UTC

FEDORA-2023-cae23d55ef has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-cae23d55ef

Comment 13 Fedora Update System 2023-10-20 14:38:08 UTC

FEDORA-EPEL-2023-c6137b5c66 has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-c6137b5c66

Comment 14 Fedora Update System 2023-10-20 17:28:50 UTC

FEDORA-EPEL-2023-e10308a01e has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-e10308a01e

Comment 15 Fedora Update System 2023-10-21 02:39:57 UTC

FEDORA-2023-587b305c8a has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-587b305c8a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-587b305c8a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2023-10-21 02:40:49 UTC

FEDORA-2023-cae23d55ef has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-cae23d55ef`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-cae23d55ef

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 17 Fedora Update System 2023-10-21 02:41:39 UTC

FEDORA-EPEL-2023-c6137b5c66 has been pushed to the Fedora EPEL 9 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-c6137b5c66

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2023-10-22 08:22:24 UTC

FEDORA-EPEL-2023-e10308a01e has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-e10308a01e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2023-10-29 00:41:46 UTC

FEDORA-EPEL-2023-c6137b5c66 has been pushed to the Fedora EPEL 9 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 20 Fedora Update System 2023-10-29 01:33:05 UTC

FEDORA-2023-587b305c8a has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 21 Fedora Update System 2023-10-29 01:47:10 UTC

FEDORA-2023-cae23d55ef has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 22 Fedora Update System 2023-10-30 00:35:56 UTC

FEDORA-EPEL-2023-e10308a01e has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 23 Fedora Update System 2023-11-03 18:42:23 UTC

FEDORA-2023-aa62881cba has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 24 Fedora Update System 2024-08-12 18:35:52 UTC

FEDORA-EPEL-2024-a8f27a60a0 (mmlib-1.4.2-13.el10_0) has been submitted as an update to Fedora EPEL 10.0.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-a8f27a60a0

Comment 25 Fedora Update System 2024-08-12 18:57:16 UTC

FEDORA-EPEL-2024-a8f27a60a0 (mmlib-1.4.2-13.el10_0) has been pushed to the Fedora EPEL 10.0 stable repository.
If problem still persists, please make note of it in this bug report.