Bug 2236516 - segfault when using coroutine due to miscompilation.
Summary: segfault when using coroutine due to miscompilation.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: llvm
Version: 38
Hardware: aarch64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Tom Stellard
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/llvm/llvm-project/...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-08-31 14:38 UTC by Kefu Chai
Modified: 2023-09-13 02:36 UTC (History)
13 users (show)

Fixed In Version: llvm-16.0.6-3.fc38
Clone Of:
Environment:
Last Closed: 2023-09-13 02:36:05 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Kefu Chai 2023-08-31 14:38:26 UTC
if compiled and run with ASan, it would report stack-use-after-return.

please see https://github.com/llvm/llvm-project/issues/59723 for more details

Reproducible: Always

Comment 1 Kefu Chai 2023-08-31 14:40:19 UTC
as the upstream won't backport the fix to llvm 16, so we need to backport the fix to f38 package.

filed following pull request to address this issue

- https://src.fedoraproject.org/rpms/clang/pull-request/211
- https://src.fedoraproject.org/rpms/llvm/pull-request/182

Comment 2 Fedora Update System 2023-09-05 14:37:15 UTC
FEDORA-2023-7cdaddb308 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-7cdaddb308

Comment 3 Fedora Update System 2023-09-06 01:24:32 UTC
FEDORA-2023-7cdaddb308 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-7cdaddb308`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-7cdaddb308

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 4 Warren Togami 2023-09-06 23:14:04 UTC
updates-testing llvm-16.0.6-3.fc38 is somehow breaking Fedora's chromium and Google's chrome browsers.

100% reproducible.

Simply upgrade llvm-libs and restart either browser to see extreme rendering glitches and slow video.

https://twitter.com/wtogami/status/1699560023947399239
Screenshot of rendering glitches. This is supposed to be the Ctrl-T new window search screen. It is failing to render the images including "Google" and the icons below it.
If you view any Youtube video it plays with extreme frame drop and 100% CPU.

Comment 5 Tulio Magno Quites Machado Filho 2023-09-07 00:17:23 UTC
@tchaikov Could you investigate this, please?

Comment 6 Warren Togami 2023-09-07 02:59:01 UTC
kernel-6.4.14-200.fc38.x86_64
AMD Ryzen 7 PRO 5850U with Radeon Graphics

If I were to guess llvm-libs broke something to do with GPU hardware acceleration?

Comment 7 Kefu Chai 2023-09-07 12:01:50 UTC
(In reply to Tulio Magno Quites Machado Filho from comment #5)
> @tchaikov Could you investigate this, please?

sure. will take a look this weekend. meanwhile, will keep needinfo flag.

Comment 8 Kefu Chai 2023-09-11 10:23:40 UTC
(In reply to Warren Togami from comment #4)
> updates-testing llvm-16.0.6-3.fc38 is somehow breaking Fedora's chromium and
> Google's chrome browsers.
> 
> 100% reproducible.
> 
> Simply upgrade llvm-libs and restart either browser to see extreme rendering
> glitches and slow video.
> 
> https://twitter.com/wtogami/status/1699560023947399239
> Screenshot of rendering glitches. This is supposed to be the Ctrl-T new
> window search screen. It is failing to render the images including "Google"
> and the icons below it.
> If you view any Youtube video it plays with extreme frame drop and 100% CPU.

hi Warren, thank you for testing the testing build of llvm-libs and for the directions. but i don't really think that the updated llvm-libs is the root cause of the extreme rendering glitches and slow video on chromium  and the google chrome browsers.

please let me explain what's new in llvm-16.0.6-3 in comparision to the previous release (-2). the gist is the patch at https://src.fedoraproject.org/fork/tchaikov/rpms/llvm/c/2f5dbfd99349d76d24d242a7ad053c09e3af0630?branch=f38 . the change touches one of coroutine passes. these coroutine passes performs the transformations and optimizations to lower the coroutine intrinitcs to, well, lower level representations of the LLVM IR. in thie case, the CoroElide pass checks if we can avoid heap allocation by inlining the coroutine for better performance. this happens after the source code is parsed by Clang frontend, and then compiled into LLVM IR. C++20 coroutine, as its name suggests, is a C++20 features, but chomium does not use C++20 yet. see https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++-features.md . so, the pass should not take effect when building chromium.

llvm-libs provides the shared bits of the LLVM compiler. so that, for instance, clang and llvm-ar can shared the code in a single place. in other words, llvm-libs are part of the clang's toolchain. in general, it is not even necessary from a regular user's perspective.

> Simply upgrade llvm-libs and restart either browser to see extreme rendering glitches and slow video.

yeah, the chromium packages are compiled using clang by default (https://src.fedoraproject.org/rpms/chromium/blob/f38/f/chromium.spec#_123). but the llvm-libs only kicks in when compiling and linking the application. once the application is built and packaged, i don't think llvm-libs is used anymore. IIUC, one can even install and use chromium without llvm-libs, as llvm-libs is not one of its (indirect) runtime dependencies.

if possible, could you please try to remove llvm-libs, and test chromium and/or google's chrome browsers again?

Comment 9 Tulio Magno Quites Machado Filho 2023-09-11 13:11:42 UTC
(In reply to Kefu Chai from comment #8)
> yeah, the chromium packages are compiled using clang by default
> (https://src.fedoraproject.org/rpms/chromium/blob/f38/f/chromium.spec#_123).
> but the llvm-libs only kicks in when compiling and linking the application.

Or at runtime, when code is generated using LLVM, e.g. via Mesa's LLVMpipe.

> once the application is built and packaged, i don't think llvm-libs is used
> anymore. IIUC, one can even install and use chromium without llvm-libs, as
> llvm-libs is not one of its (indirect) runtime dependencies.

Won't this drop support for HW acceleration?

Comment 10 Josh Stone 2023-09-11 15:58:42 UTC
Warren, are your issues here separate from your mesa bug #2238198?
i.e. are there problems with this LLVM build *without* that problematic mesa build?

Either way, I wonder if it could be similar caching problems as mentioned here:
https://discussion.fedoraproject.org/t/chromium-based-browsers-display-garbled-web-pages-after-mesa-is-updated/83438

Comment 11 Warren Togami 2023-09-11 17:14:47 UTC
> Warren, are your issues here separate from your mesa bug #2238198?
> i.e. are there problems with this LLVM build *without* that problematic mesa build?

The new mesa build was pushed to updates-testing after this llvm was unpushed. I had removed this llvm update before testing that mesa update.

> Either way, I wonder if it could be similar caching problems as mentioned here:
> https://discussion.fedoraproject.org/t/chromium-based-browsers-display-garbled-web-pages-after-mesa-is-updated/83438

That said both of these issues seem to be duplicates of Bug #2193335. Deleting chrome/chromium's generated code caches fixes both chrome and chromium with llvm-16.0.6-3.fc38.

So this isn't llvm's fault. Chromium needs to learn when to invalidate its caches which is Bug #2193335.

A lot of people will have an unexpected surprise from their browser. But this isn't llvm's fault.

Comment 12 Fedora Update System 2023-09-12 01:23:48 UTC
FEDORA-2023-7cdaddb308 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-7cdaddb308`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-7cdaddb308

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Kefu Chai 2023-09-12 05:55:07 UTC
(In reply to Tulio Magno Quites Machado Filho from comment #9)
> (In reply to Kefu Chai from comment #8)
> > yeah, the chromium packages are compiled using clang by default
> > (https://src.fedoraproject.org/rpms/chromium/blob/f38/f/chromium.spec#_123).
> > but the llvm-libs only kicks in when compiling and linking the application.
> 
> Or at runtime, when code is generated using LLVM, e.g. via Mesa's LLVMpipe.
> 
> > once the application is built and packaged, i don't think llvm-libs is used
> > anymore. IIUC, one can even install and use chromium without llvm-libs, as
> > llvm-libs is not one of its (indirect) runtime dependencies.
> 
> Won't this drop support for HW acceleration?

ahh, right. i missed the runtime code generation case. but the argument on C++20 coroutine still holds.

Comment 14 Fedora Update System 2023-09-13 02:36:05 UTC
FEDORA-2023-7cdaddb308 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.