Bug 2248097

Summary: [s390x] "cabal update" segfaults
Product: [Fedora] Fedora Reporter: Jens Petersen <petersen>
Component: cabal-installAssignee: Jens Petersen <petersen>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 40CC: petersen, stefansf
Target Milestone: ---Keywords: Regression, Reopened
Target Release: ---Flags: petersen: mirror+
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: cabal-install-3.8.1.0-7.fc40 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-11-13 10:22:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jens Petersen 2023-11-06 05:54:44 UTC
Not sure what (or why/since when) is happening, but "cabal update -w ghc-9.6"
for example is segfaulting in Fedora s390x copr chroots at least.

Even worse - not just F39+ but also F38 even.

See for example https://copr.fedorainfracloud.org/coprs/petersen/cabal-install/build/6594155/

Reproducible: Always

Steps to Reproduce:
1. run "cabal update -w ghc-9.6" in s390x fedora copr build (ie in spec file)
Actual Results:  
+ cabal update -v3 -w ghc-9.6.3
Config file path source is default config file.
Config file not found: /builddir/.config/cabal/config
Writing default configuration to /builddir/.config/cabal/config
Searching for curl in path.
Found curl at /usr/bin/curl
Searching for powershell in path.
Cannot find powershell on the path
Searching for wget in path.
Cannot find wget on the path
Selected http transport implementation: curl
File monitor 'config' changed: first run
Project settings changed, reconfiguring...
creating /builddir/build/BUILD/cabal-install-3.10.1.0/dist-newstyle
creating /builddir/build/BUILD/cabal-install-3.10.1.0/dist-newstyle/cache
/var/tmp/rpm-tmp.LV9I8Q: line 48:  7846 Segmentation fault      (core dumped) cabal update -v3 -w ghc-9.6.3

Expected Results:  
No crash

Previous builds in May were successful:
eg https://download.copr.fedorainfracloud.org/results/petersen/cabal-install/fedora-rawhide-s390x/05989927-cabal-install/

Comment 1 Jens Petersen 2023-11-06 06:00:36 UTC
Also just confirmed same segfault happening in Koji s390x rawhide (though of course cabal-install cannot normally be used there due to no network).

Segfault doesn't happen with F37 s390x though.

Comment 2 Jens Petersen 2023-11-06 07:25:56 UTC
(Just for completeness: same thing with latest cabal-3.10.2.0 just released:
https://copr.fedorainfracloud.org/coprs/petersen/cabal-install/build/6601582/
)

But I confirmed in Koji that "cabal update" also segfaults

Comment 3 Jens Petersen 2023-11-06 07:26:50 UTC
(epel9 also does not segfault of course)

Comment 4 Jens Petersen 2023-11-06 11:31:36 UTC
I managed to reproduce in a s390x fedora:39 container:

strace output

newfstatat(AT_FDCWD, "/usr/bin/ghc-pkg", {st_mode=S_IFREG|0755, st_size=352, ...}, 0) = 0
getpid()                                = 279
openat(AT_FDCWD, "/dist-newstyle/cache/compiler279-0.tmp", O_RDWR|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK, 0666) = 11
newfstatat(11, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(11, TCGETS, 0x3ffc88f5988)        = -1 ENOTTY (Inappropriate ioctl for device)
sched_getaffinity(0, 128, [0 1])        = 8
rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x3fbe4400000000} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

Comment 5 Jens Petersen 2023-11-06 12:12:43 UTC
cabal-install-3.10.1.0-1.fc39.s390x also dumps core inside fedora:39

(gdb) bt
#0  0x000000008000e008 in evacuate1 ()
#1  0x0000000083f3c9b6 in scavenge_loop1 ()
#2  0x0000000083f31cbc in scavenge_until_all_done ()
#3  0x0000000083f334fa in GarbageCollect ()
#4  0x0000000083f1dce2 in scheduleDoGC ()
#5  0x0000000083f1ee90 in schedule ()
#6  0x0000000083f1f6de in scheduleWaitThread ()
#7  0x0000000083f19610 in hs_main ()
#8  0x000000008003226e in main ()

Comment 6 Jens Petersen 2023-11-06 12:15:08 UTC
Reverting F39 to https://kojipkgs.fedoraproject.org//packages/cabal-install/3.6.2.0/3.fc39/s390x/cabal-install-3.6.2.0-3.fc39.s390x.rpm
stops `cabal update` from segfaulting and after that (~/.cabal/packages/ created) current cabal-install-3.8.1.0 no longer segfaults.

Comment 7 Jens Petersen 2023-11-06 14:21:51 UTC
Crash is also reproduced with fedora-toolbox:39 toolbox (provided there is no existing ~/.cabal/)

Comment 9 Jens Petersen 2023-11-25 02:01:53 UTC
See https://gitlab.haskell.org/ghc/ghc/-/issues/24163

Comment 10 Fedora Update System 2023-12-16 10:06:32 UTC
FEDORA-2023-d1dd535dda has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-d1dd535dda

Comment 11 Fedora Update System 2023-12-17 01:51:58 UTC
FEDORA-2023-d1dd535dda has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-d1dd535dda`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-d1dd535dda

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2023-12-25 03:36:49 UTC
FEDORA-2023-d1dd535dda has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 stefansf 2024-01-31 15:21:36 UTC
Could you also release new packages for F39 including a rebuild of cabal?

Comment 14 Jens Petersen 2024-02-01 06:09:14 UTC
Thank you

Seems I didn't push the f39 build into the updates system, ugh

Comment 15 Fedora Update System 2024-02-01 06:21:19 UTC
FEDORA-2024-88c617181a has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2024-88c617181a

Comment 16 Fedora Update System 2024-02-02 02:16:56 UTC
FEDORA-2024-88c617181a has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-88c617181a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-88c617181a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 17 Fedora Update System 2024-02-10 01:25:21 UTC
FEDORA-2024-88c617181a (cabal-install-3.8.1.0-5.fc39 and ghc-9.4.5-137.fc39) has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 stefansf 2024-02-10 08:03:17 UTC
I gave cabal a try and it still fails. Is it possible that cabal-install was still compiled with the broken GHC?

$ rpm -q cabal-install
cabal-install-3.8.1.0-5.fc39.s390x
$ cabal update
Config file path source is default config file.
Config file not found: /root/.cabal/config
Writing default configuration to /root/.cabal/config
Segmentation fault (core dumped)

Comment 19 Jens Petersen 2024-02-13 16:50:00 UTC
Hm, it was built with ghc-compiler-9.4.5-137.fc39.s390x,
which is supposed to have the fixed...

https://koji.fedoraproject.org/koji/taskinfo?taskID=112716128

Just double-checking: so "/usr/bin/cabal update" segfaults for you, right?
I need to check more carefully then I guess.

Comment 20 Jens Petersen 2024-02-13 16:52:15 UTC
Is it because I need to rebuild all the deps too? ;-(
I seem to remember something like that: need to check my notes in gitlab.

Comment 21 stefansf 2024-02-13 18:45:46 UTC
(In reply to Jens Petersen from comment #19)
> Hm, it was built with ghc-compiler-9.4.5-137.fc39.s390x,
> which is supposed to have the fixed...
> 
> https://koji.fedoraproject.org/koji/taskinfo?taskID=112716128
> 
> Just double-checking: so "/usr/bin/cabal update" segfaults for you, right?

Yes, I have tested it in a fresh container, i.e., no old/potentially broken $HOME/.cabal lingering around are whatever.

Comment 22 Jens Petersen 2024-02-14 17:46:33 UTC
As I feared: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/11662#note_536725

> (Just rebuilding the Fedora cabal-install package was insufficient alas.)

Sigh

Comment 23 Jens Petersen 2024-02-14 17:47:53 UTC
I wonder if the mass-rebuild was even enough to fix this for deps in F40.

Comment 24 Jens Petersen 2024-02-15 05:56:12 UTC
Okay I rebuilt cabal-install for Rawhide and next F40, which should fix the problem there.

Comment 25 stefansf 2024-02-27 16:15:58 UTC
I can confirm that cabal is working for me on rawhide and next (40).  However, it is still segfaulting on 39 for me.

Comment 26 Jens Petersen 2024-02-29 13:18:34 UTC
Okay, thank you.

Yeah, I didn't have time yet sorry - there has been quite a lot of fall out for the F40, and also some pending pandoc CVEs.
So I need more time - but let me know if you consider this is a priority, and I can try to bump it up my to-do list.

Comment 27 stefansf 2024-02-29 15:16:15 UTC
Fine for me.  Just wanted to confirm that F40/Rawhide packages are good. Once F39 packages are rebuild I will give them a try, too.

Comment 28 Aoife Moloney 2024-11-13 09:59:26 UTC
This message is a reminder that Fedora Linux 39 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 39 on 2024-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '39'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 39 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 29 Jens Petersen 2024-11-13 10:22:30 UTC
Sorry, I was not able to find time to fix this for F39.

Comment 30 Jens Petersen 2024-12-24 06:09:55 UTC
I was a little surprised to see some segfault happening still in F40, though it could be unrelated perhaps

See https://copr.fedorainfracloud.org/coprs/petersen/cabal-install/build/8428383/
https://download.copr.fedorainfracloud.org/results/petersen/cabal-install/fedora-40-s390x/08428383-cabal-install/

+ cd cabal-install-3.14.1.0
+ cabal update -w ghc-9.6
Config file path source is default config file.
Config file not found: /builddir/.config/cabal/config
Writing default configuration to /builddir/.config/cabal/config
/var/tmp/rpm-tmp.vvK7K9: line 48: 19178 Segmentation fault      (core dumped) cabal update -w ghc-9.6

(F41 built fine)