Bug 1902663 - compiling doctests fails during %cargo_test on aarch64 only
Summary: compiling doctests fails during %cargo_test on aarch64 only
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: rust
Version: 34
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Rust SIG
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-11-30 10:31 UTC by Fabio Valentini
Modified: 2021-06-16 23:24 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-16 23:24:54 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
stack traces with missing annotations, gdb hates me (8.25 KB, text/plain)
2021-02-09 00:40 UTC, Fabio Valentini
no flags Details

Description Fabio Valentini 2020-11-30 10:31:24 UTC
I have now found two packages that fail to build on aarch64 only during compilation of doctests: clap and slog

clap: https://koji.fedoraproject.org/koji/taskinfo?taskID=56241039
slog: https://koji.fedoraproject.org/koji/taskinfo?taskID=56431109

There is no output for the failed tests other than:
"Couldn't compile the test."

Comment 1 Fabio Valentini 2021-01-25 16:24:12 UTC
Also affects nom 5:
https://koji.fedoraproject.org/koji/taskinfo?taskID=60471169

Comment 2 Josh Stone 2021-01-25 19:27:28 UTC
Is this a consistent failure in those packages, or only occasional?

Comment 3 Fabio Valentini 2021-01-25 20:20:46 UTC
The failures happens consistently every time on aarch64 for the affected packages.

Though nom seems to be the worst of the bunch, with *all* non-ignored doctests failing to compile.

Looks like rustdoc is crashing ...

Comment 4 Josh Stone 2021-01-25 21:04:44 UTC
aarch64-unknown-linux-gnu is now considered a tier-1 target, so if this is reproducible with the upstream toolchain, I expect it will get a lot of attention.

Comment 5 Robert-André Mauchin 🐧 2021-01-29 15:13:21 UTC
same with rust-slice-deque

Comment 6 Fabio Valentini 2021-01-29 18:22:11 UTC
I'll be getting aarch64 hardware on Monday, so I hope to be able to debug this better soon :)

Comment 7 Fabio Valentini 2021-02-04 21:21:10 UTC
Alright. I have reproduced this natively on aarch64 hardware.

Cloned the clap git repository, ran "cargo test --release".
With cargo + rust from Fedora 33 repos I have these crashes:

Process 20893 (rustc) crashed in getDefSrcRegIgnoringCopies(llvm::Register, llvm::MachineRegisterInfo const&)()

 rustc killed by SIGSEGV
 
 #1 [libLLVM-11.so] getDefSrcRegIgnoringCopies(llvm::Register, llvm::MachineRegisterInfo const&)
 #2 [libLLVM-11.so] llvm::getDefIgnoringCopies(llvm::Register, llvm::MachineRegisterInfo const&)
 #3 [libLLVM-11.so] llvm::getOpcodeDef(unsigned int, llvm::Register, llvm::MachineRegisterInfo const&)
 #4 [libLLVM-11.so] llvm::CombinerHelper::matchUndefStore(llvm::MachineInstr&)
 #5 [libLLVM-11.so] (anonymous namespace)::AArch64GenPreLegalizerCombinerHelper::tryCombineAll(llvm::GISelChangeObserver&, llvm::MachineInstr&, llvm::MachineIRBuilder&) const [clone .constprop.0]

I'll try to reproduce with stable rust installed via rustup next.

Comment 8 Fabio Valentini 2021-02-04 23:34:51 UTC
I cannot reproduce the crashes with "official" Rust binaries installed with rustup:

stable-aarch64-unknown-linux-gnu (default)
rustc 1.49.0 (e1884a8e3 2020-12-29)

So the crashes are specific to the rust / cargo packages installed from Fedora repos:

rust-1.49.0-1.fc33.aarch64
cargo-1.49.0-1.fc33.aarch64

(or rather, LLVM, looking at the stack trace in the last comment?)

Comment 9 Josh Stone 2021-02-08 19:27:06 UTC
Can you install the corresponding llvm-libs-debuginfo so we get inlines and file:line in the backtrace?

There are a few backports in upstream rust's llvm that *might* be relevant, but their bugs don't match your backtrace info AFAICS.

Comment 10 Fabio Valentini 2021-02-08 23:38:36 UTC
Interesting. gdb crashed while generating a backtrace from the coredump ...
This is the only thing I got from it:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  llvm::MachineInstr::getOperand (i=0, this=0x0) at ../include/llvm/CodeGen/MachineInstr.h:472
472	    return Operands[i];

Comment 11 Fabio Valentini 2021-02-09 00:40:04 UTC
Something very weird is going on.

debuginfo for all involved packages is already installed:
Going to install 0 debuginfo packages

But gdb can't find any of them:
Can't find packages for 16 debuginfo files

Missing debuginfo file: /usr/lib/debug/.build-id/c1/d93c73ef25ac17c8fc246842d95e835bfe6920.debug
Missing debuginfo file: /usr/lib/.build-id/6d/5c6c52f2523cfecd1f850120f3cda4c1fd7ff8.debug
Missing debuginfo file: /usr/lib/.build-id/bb/70237d48dc3f8c59e4a24f7957149334f3a476.debug
Missing debuginfo file: /usr/lib/.build-id/f8/b6f038f29adf3d706c50fd31ff5d1a1c9e2a0b.debug
Missing debuginfo file: /usr/lib/.build-id/d8/8efbcde5da4516f55939527314406d6acc5fa4.debug
Missing debuginfo file: /usr/lib/.build-id/c1/d93c73ef25ac17c8fc246842d95e835bfe6920.debug
Missing debuginfo file: /usr/lib/.build-id/e5/428466c972ce3253ee98c1bf4e0bd8a3e46450.debug
Missing debuginfo file: /usr/lib/.build-id/b0/409727b2ea2ace7d340b21f4c22a49c8586238.debug
Missing debuginfo file: /usr/lib/.build-id/d8/f5acb829dca2b1fc56f8c9370914f1dae7214a.debug
Missing debuginfo file: /usr/lib/.build-id/b9/d5a34ebc331bf181da2a96a3f061928afcfda3.debug
Missing debuginfo file: /usr/lib/.build-id/d1/db9e285cb7c68f98d2becbcf4a74cf4dc3505a.debug
Missing debuginfo file: /usr/lib/.build-id/e3/1ef7b1bb28ff8e35ca83ef10d5ff5b2c8b4e50.debug
Missing debuginfo file: /usr/lib/.build-id/de/6207efea2eed6cb4dbd7f72f20d2fe8f07811a.debug
Missing debuginfo file: /usr/lib/.build-id/19/1b98dee8807a42746c5ae1fc59f0919e297781.debug
Missing debuginfo file: /usr/lib/.build-id/3c/954829f1ba61a250a43048cbcdd43bd3ff767d.debug
Missing debuginfo file: /usr/lib/.build-id/74/4e3bac88fb0888aa37a5c322974ae8f672a829.debug

And indeed, those files do not exist, only their non-debug-suffix symlink counterparts.


I have attached the stack traces I could wrangle out from coredumpctl, since gdb apparently cannot generate any annotated backtraces for me without crashing.

Comment 12 Fabio Valentini 2021-02-09 00:40:49 UTC
Created attachment 1755819 [details]
stack traces with missing annotations, gdb hates me

Comment 13 Ben Cotton 2021-02-09 15:29:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 14 Fabio Valentini 2021-06-16 23:24:54 UTC
As far as I can tell, this has been fixed for rawhide and f34 since Rust switched to LLVM 12.


Note You need to log in before you can comment on or make changes to this bug.