Fedora Account System
Red Hat Associate
Red Hat Customer
The test suite of the "glib" crate started failing in Fedora with segmentation faults. It is possible to track this back to the update from Rust 1.75 to Rust 1.76, across all current branches of Fedora: https://koschei.fedoraproject.org/package/rust-glib The following tests now cause segmentation faults (list updated for glib v0.19.3): - variant_iter::tests::test_variant_iter_array (a --lib test) - variant_iter::tests::test_variant_str_iter_last (a --lib test) - variant_iter::tests::test_variant_str_iter_nth (a --lib test) - variant::Variant::array_iter_str (a --doc test) The error message looks something like this: """ error: test failed, to rerun pass `--lib` Caused by: process didn't exit successfully: `/builddir/build/BUILD/glib-0.18.5/target/rpm/deps/glib-8a813dbb6056e1aa --skip structured_log` (signal: 11, SIGSEGV: invalid memory reference) """ Since the test harness is killed by the SIGSEGV, the output of "cargo test" does not actually show the failing test. I isolated the failure cases by running the tests with "--test-threads 1" and added known bad cases with "--skip" until the test suite passed. The "structured_log" test always fails - it expects some paths to match the workspace layout in the upstream git repository. It can be skipped and is not relevant to this issue. ============================================================ Test code for the first three failing tests is in this file upstream (which has not been changed in the last two years): https://github.com/gtk-rs/gtk-rs-core/blob/master/glib/src/variant_iter.rs#L204 Test code for the failing doctest is in this file upstream: https://github.com/gtk-rs/gtk-rs-core/blob/master/glib/src/variant.rs#L843 ============================================================ I tried the following scenarios to reproduce the issue (always running "cargo clean" inbetween): 1. system rustc (1.77.0 on Fedora 39): a. "/usr/bin/cargo test --release -- --skip structured_log": crashes b. "cargo +system test --release -- --skip structured_log": passes (system = linked toolchain at /usr) (I have *no idea* how case 1.a and 1.b can behave differently.) 2. rustup toolchains: a. "cargo +stable test --release -- --skip structured_log": passes (stable = 1.77.1 (7cf61ebde 2024-03-27)) b. "cargo +beta test --release -- --skip structured_log": crashes (beta = 1.78.0-beta.3 (4147533e0 2024-03-27)) c. "cargo +nightly test --release -- --skip structured_log": crashes (nightly = 1.79.0-nightly (c9f8f3438 2024-03-27)) 3. different compiler flags for system rustc (1.77.0 on Fedora 39): a. "RUSTFLAGS="-Copt-level=1" /usr/bin/cargo test -- --skip structured_log": passes b. "RUSTFLAGS="-Copt-level=2" /usr/bin/cargo test -- --skip structured_log": crashes c. "RUSTFLAGS="-Copt-level=3" /usr/bin/cargo test -- --skip structured_log": crashes 4. different compiler flags for rustup toolchains: a. "RUSTFLAGS="-Copt-level=1" cargo +stable test -- --skip structured_log": passes b. "RUSTFLAGS="-Copt-level=2" cargo +stable test -- --skip structured_log": passes c. "RUSTFLAGS="-Copt-level=3" cargo +stable test -- --skip structured_log": passes d. "RUSTFLAGS="-Copt-level=1" cargo +beta test -- --skip structured_log": passes e. "RUSTFLAGS="-Copt-level=2" cargo +beta test -- --skip structured_log": crashes f. "RUSTFLAGS="-Copt-level=3" cargo +beta test -- --skip structured_log": crashes g. "RUSTFLAGS="-Copt-level=1" cargo +nightly test -- --skip structured_log": passes h. "RUSTFLAGS="-Copt-level=2" cargo +nightly test -- --skip structured_log": crashes i. "RUSTFLAGS="-Copt-level=3" cargo +nightly test -- --skip structured_log": crashes ============================================================ I cannot determine why the Fedora Rust toolchain seems to have introduced this issue with Rust 1.76, whereas upstream rust toolchains only start exhibiting it with Rust 1.78.0 beta. I don't think different LLVM versions should be the cause, since the crashes started happening with Rust 1.76 across all branches of Fedora, where - on Fedora Rawhide, rustc links LLVM 18, - on Fedora 39 and 40, rustc links LLVM 17, and - on Fedora 38, rustc links LLVM 16. Maybe the glib crate has some UB which is only triggered by optimizations in recent Rust releases? But that wouldn't necessarily explain why the Fedora and the upstream toolchains behave differently. Reproducible: Always Steps to Reproduce: 1. unpack sources for the "glib" crate 2. run "/usr/bin/cargo test --release" Actual Results: Tests pass. Expected Results: Test runner segfaults.
(In reply to Fabio Valentini from comment #0) > 1. system rustc (1.77.0 on Fedora 39): > > a. "/usr/bin/cargo test --release -- --skip structured_log": crashes > b. "cargo +system test --release -- --skip structured_log": passes (system = > linked toolchain at /usr) > > (I have *no idea* how case 1.a and 1.b can behave differently.) Is your `rustup default` also "system"? Because 1.a will still use "rustc" from your PATH, which will probably go through the rustup shim again. > I cannot determine why the Fedora Rust toolchain seems to have introduced > this issue with Rust 1.76, whereas upstream rust toolchains only start > exhibiting it with Rust 1.78.0 beta. I think there is UB, and I would not try to reason about the compiler beyond that. Any subtle difference in the toolchains could change optimization in a way that may or may not manifest in noticeably bad behavior. There's an unsafe block in variant_iter.rs: unsafe { let p: *mut libc::c_char = std::ptr::null_mut(); let s = b"&s\0"; ffi::g_variant_get_child( self.variant.to_glib_none().0, i, s as *const u8 as *const _, &p, std::ptr::null::<i8>(), ); let p = std::ffi::CStr::from_ptr(p); p.to_str().unwrap() } The first p is initialized to null_mut(), but it is not itself mutable. It is passed immutably as &p to the FFI call, and then passed to CStr::from_ptr. By Rust semantics, it must still be null! But clearly this code expects g_variant_get_child to have filled in a valid pointer. You can get a fun demonstration of UB here too. I added an assertion before the CStr::from_ptr call: assert_ne!(p, std::ptr::null_mut()); and it fails in a confusing way: thread 'variant_iter::tests::test_variant_iter_array' panicked at glib/src/variant_iter.rs:129:13: assertion `left != right` failed left: 0x7f1fa80011b0 right: 0x0 The fix is to start with "let mut p" and pass "&mut p" to g_variant_get_child. With that, all tests pass in release mode for me on F39 +system, +stable, +beta, and +nightly.
Thank you for the pointers! I submitted the suggested fix as a PR to upstream: https://github.com/gtk-rs/gtk-rs-core/pull/1343
FEDORA-2024-f8adb1bac8 (rust-glib0.15-0.15.12-5.fc40, rust-glib0.17-0.17.10-3.fc40, and 2 more) has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2024-f8adb1bac8
FEDORA-2024-3ee122ae90 (rust-glib0.15-0.15.12-5.fc38 and rust-glib0.17-0.17.10-3.fc38) has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2024-3ee122ae90
FEDORA-2024-ec54a6adea (rust-glib0.15-0.15.12-5.fc39, rust-glib0.17-0.17.10-3.fc39, and 2 more) has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2024-ec54a6adea
FEDORA-2024-3ee122ae90 has been pushed to the Fedora 38 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-3ee122ae90` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3ee122ae90 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-f8adb1bac8 has been pushed to the Fedora 40 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-f8adb1bac8` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-f8adb1bac8 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-ec54a6adea has been pushed to the Fedora 39 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-ec54a6adea` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-ec54a6adea See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-3ee122ae90 (rust-glib0.15-0.15.12-5.fc38 and rust-glib0.17-0.17.10-3.fc38) has been pushed to the Fedora 38 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2024-ec54a6adea (rust-glib0.15-0.15.12-5.fc39, rust-glib0.17-0.17.10-3.fc39, and 2 more) has been pushed to the Fedora 39 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2024-f8adb1bac8 (rust-glib0.15-0.15.12-5.fc40, rust-glib0.17-0.17.10-3.fc40, and 2 more) has been pushed to the Fedora 40 stable repository. If problem still persists, please make note of it in this bug report.