Bug 2168622

Summary: rustc 1.67.0 on LLVM 14 crashes when compiling certain crates with LTO
Product: [Fedora] Fedora Reporter: Fabio Valentini <decathorpe>
Component: rustAssignee: Josh Stone <jistone>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 36CC: amulhern, igor.raits, jistone, rust-sig, TicoTimo
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rust-1.67.1-1.fc36 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-02-19 02:11:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Fabio Valentini 2023-02-09 15:09:15 UTC 
It appears that Rust 1.67.0 in combination with LLVM 14 is crash-prone when compiling certain crates.

As far as I can tell, this is not architecture-specific. Currently affected are:

- actix 0.10.0: https://koschei.fedoraproject.org/package/rust-actix0.10?collection=f36
- afterburn 5.3.0: https://koschei.fedoraproject.org/package/rust-afterburn?collection=f36
- bodhi-cli 2.0.1: https://koschei.fedoraproject.org/package/rust-bodhi-cli?collection=f36
- fedora-update-feedback 2.0.2: https://koschei.fedoraproject.org/package/rust-fedora-update-feedback?collection=f36
- tealdeer 1.5.0: https://koschei.fedoraproject.org/package/rust-tealdeer?collection=f36

Example output from bodhi-cli test log below.

It looks like all affected crates have LTO enabled. In contrast to previous similar issues with LTO, this affects all architectures (or at least x86_64, aarch64, and ppcle64, as these are what's tested in koschei), and it is not triggered by building doctests, but either by building an actual "bin" target, or in the case of actix 0.10.0, a "test" target binary.

================================================================================

     Running `/usr/bin/rustc --crate-name bodhi_cli --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=3 -C lto -C codegen-units=1 -C metadata=5f210e025e47e46a -C extra-filename=-5f210e025e47e46a --out-dir /builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps -L dependency=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps --extern bodhi=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libbodhi-7b4628bdac7a8309.rlib --extern chrono=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libchrono-3b7049fe58e6c02f.rlib --extern dirs=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libdirs-f8d10ef55c901f4d.rlib --extern rpassword=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/librpassword-6a657e0c04a90866.rlib --extern secret_service=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libsecret_service-8a6e726ef94e517d.rlib --extern serde=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libserde-01837d1ac57fa4cb.rlib --extern serde_json=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libserde_json-20a4f6426eedf456.rlib --extern structopt=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libstructopt-eb3b87c9f89ff968.rlib --extern term_size=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libterm_size-ea88a6fd147296fb.rlib --extern tokio=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libtokio-05c8ed79f42a4ad0.rlib --extern toml=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libtoml-6fe1b3d12671d3df.rlib -Copt-level=3 -Cdebuginfo=1 -Ccodegen-units=1 -Clink-arg=-Wl,-z,relro -Clink-arg=-Wl,-z,now -Clink-arg=-Wl,-dT,/builddir/build/BUILD/bodhi-cli-2.0.1/.package_note-rust-bodhi-cli-2.0.1-1.fc36.x86_64.ld --cap-lints=warn`
/lib64/librustc_driver-124d21943690d304.so(+0x701633)[0x7f79dff01633]
/lib64/libc.so.6(+0x3ea00)[0x7f79df43ea00]
/lib64/libLLVM-14.so(_ZN4llvm11PointerType3getEPNS_4TypeEj+0x24)[0x7f79d9bcd494]
/lib64/libLLVM-14.so(+0xc724a0)[0x7f79d9a724a0]
/lib64/libLLVM-14.so(+0x16983de)[0x7f79da4983de]
/lib64/libLLVM-14.so(+0x168f61c)[0x7f79da48f61c]
/lib64/libLLVM-14.so(+0x168b0a9)[0x7f79da48b0a9]
/lib64/libLLVM-14.so(+0x1fbdb44)[0x7f79dadbdb44]
/lib64/libLLVM-14.so(+0x1fbd6f3)[0x7f79dadbd6f3]
/lib64/libLLVM-14.so(+0x18c7b93)[0x7f79da6c7b93]
/lib64/libLLVM-14.so(+0x18c8b9d)[0x7f79da6c8b9d]
/lib64/libLLVM-14.so(+0x18c9733)[0x7f79da6c9733]
/lib64/libLLVM-14.so(+0x18c747b)[0x7f79da6c747b]
/lib64/libLLVM-14.so(+0x1fb877f)[0x7f79dadb877f]
/lib64/libLLVM-14.so(_ZN4llvm7IRMover4moveESt10unique_ptrINS_6ModuleESt14default_deleteIS2_EENS_8ArrayRefIPNS_11GlobalValueEEESt8functionIFvRS7_SA_IFvSB_EEEEb+0xb84)[0x7f79dadb5bd4]
/lib64/libLLVM-14.so(_ZN4llvm6Linker12linkInModuleESt10unique_ptrINS_6ModuleESt14default_deleteIS2_EEjSt8functionIFvRS2_RKNS_9StringSetINS_15MallocAllocatorEEEEE+0x1b0b)[0x7f79dadc247b]
/lib64/librustc_driver-124d21943690d304.so(+0x72adbc)[0x7f79dff2adbc]
/lib64/librustc_driver-124d21943690d304.so(+0x87636c)[0x7f79e007636c]
/lib64/librustc_driver-124d21943690d304.so(+0x8d32b5)[0x7f79e00d32b5]
/lib64/librustc_driver-124d21943690d304.so(+0x9aeb89)[0x7f79e01aeb89]
/lib64/librustc_driver-124d21943690d304.so(+0x9ab1c6)[0x7f79e01ab1c6]
/lib64/librustc_driver-124d21943690d304.so(+0x92fe34)[0x7f79e012fe34]
/lib64/libstd-9c495a6b4cffc387.so(rust_metadata_std_d75ed06da712468a+0xb73e3)[0x7f79df71d3e3]
/lib64/libc.so.6(+0x8cdcd)[0x7f79df48cdcd]
/lib64/libc.so.6(+0x112630)[0x7f79df512630]
error: could not compile `bodhi-cli`
Caused by:
  process didn't exit successfully: `/usr/bin/rustc --crate-name bodhi_cli --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=3 -C lto -C codegen-units=1 -C metadata=5f210e025e47e46a -C extra-filename=-5f210e025e47e46a --out-dir /builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps -L dependency=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps --extern bodhi=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libbodhi-7b4628bdac7a8309.rlib --extern chrono=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libchrono-3b7049fe58e6c02f.rlib --extern dirs=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libdirs-f8d10ef55c901f4d.rlib --extern rpassword=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/librpassword-6a657e0c04a90866.rlib --extern secret_service=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libsecret_service-8a6e726ef94e517d.rlib --extern serde=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libserde-01837d1ac57fa4cb.rlib --extern serde_json=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libserde_json-20a4f6426eedf456.rlib --extern structopt=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libstructopt-eb3b87c9f89ff968.rlib --extern term_size=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libterm_size-ea88a6fd147296fb.rlib --extern tokio=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libtokio-05c8ed79f42a4ad0.rlib --extern toml=/builddir/build/BUILD/bodhi-cli-2.0.1/target/release/deps/libtoml-6fe1b3d12671d3df.rlib -Copt-level=3 -Cdebuginfo=1 -Ccodegen-units=1 -Clink-arg=-Wl,-z,relro -Clink-arg=-Wl,-z,now -Clink-arg=-Wl,-dT,/builddir/build/BUILD/bodhi-cli-2.0.1/.package_note-rust-bodhi-cli-2.0.1-1.fc36.x86_64.ld --cap-lints=warn` (signal: 11, SIGSEGV: invalid memory reference)
Comment 1 Josh Stone 2023-02-09 21:05:23 UTC 
I reproduced this with LLVM assertions enabled, getting this failure:

rustc: [...]/llvm/include/llvm/IR/Instructions.h:962: static llvm::GetElementPtrInst* llvm::GetElementPtrInst::Create(llvm::Type*, llvm::Value*, llvm::ArrayRef<llvm::Value*>, const llvm::Twine&, llvm::Instruction*): Assertion `cast<PointerType>(Ptr->getType()->getScalarType()) ->isOpaqueOrPointeeTypeMatches(PointeeType)' failed.
error: could not compile `reqwest`

Bisecting Rust 1.66 to 1.67 with this LLVM brought me to rust#104321, and comments there pointed me to a followup fix in rust#105082. Backporting that to 1.67 does fix it in my reproducer -- I'm testing a full rpm build in mock now.
Comment 2 Fedora Update System 2023-02-10 16:21:07 UTC 
FEDORA-2023-31013e77ca has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2023-31013e77ca
Comment 3 Fedora Update System 2023-02-11 02:08:23 UTC 
FEDORA-2023-31013e77ca has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-31013e77ca`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-31013e77ca

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 4 Fedora Update System 2023-02-19 02:11:19 UTC 
FEDORA-2023-31013e77ca has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.