Bug 2305342 - Review Request: fex-emu - Fast usermode x86 and x86-64 emulator for ARM64
Summary: Review Request: fex-emu - Fast usermode x86 and x86-64 emulator for ARM64
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Neal Gompa
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/FEX-Emu/FEX
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-08-16 15:46 UTC by Davide Cavalca
Modified: 2024-09-03 07:32 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2024-08-28 01:59:02 UTC
Type: ---
Embargoed:
ngompa13: fedora-review+


Attachments (Terms of Use)

Description Davide Cavalca 2024-08-16 15:46:36 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Description:
FEX allows you to run x86 and x86-64 binaries on an AArch64 host, similar to
qemu-user and box86. It has native support for a rootfs overlay, so you don't
need to chroot, as well as some thunklibs so it can forward things like GL to
the host. FEX presents a Linux 5.0+ interface to the guest, and supports only
AArch64 as a host. FEX is very much work in progress, so expect things to
change.

Fedora Account System Username: dcavalca

Reproducible: Always

Comment 1 Davide Cavalca 2024-08-16 15:48:05 UTC
This is a very early WIP based on https://copr.fedorainfracloud.org/coprs/teohhanhui/fex-emu/ and https://copr.fedorainfracloud.org/coprs/dcavalca/fex-emu/ that I'm putting up early to make iteration and feedback easier. We're tracking outstanding work items to make this packageable properly in https://hackmd.io/@dcavalca/rkzes1Tq0 (open document, just need to be signed in).

Comment 2 Davide Cavalca 2024-08-16 16:00:27 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- fix typo for binfmts bcond
- add python3-setuptools for BRs for aarch64

Comment 3 Davide Cavalca 2024-08-16 16:01:34 UTC
Scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=122045902

We're building for x86_64 to ease iteration at this stage, but that'll be disabled in the final package, as this really only makes sense on aarch64.

Comment 4 Davide Cavalca 2024-08-16 16:38:40 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- add a bcond for integration tests
- start gating bundled libs on bconds

Comment 5 Neal Gompa 2024-08-16 18:58:40 UTC
> # FEX upstream only supports these architectures
> %if %{with x86_debug}
> ExclusiveArch:  aarch64 x86_64
> %else
> ExclusiveArch:  aarch64
> %endif

This should use "%{arm64} %{x86_64}" instead.

Comment 6 Davide Cavalca 2024-08-16 20:03:35 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- fix the arch macros usage
- audit thunks BRs
- add support for snapshot builds

Comment 7 Neal Gompa 2024-08-16 20:18:19 UTC
Review notes:

[x]: Package follows Fedora packaging guidelines
[!]: Bundled dependencies are documented
>  { name="fex-gcc-target-tests-bins", ref="442678a", owner="FEX-Emu", license="GPL-2.0-or-later", bcond="integration" },
>  { name="fex-gvisor-tests-bins", ref="71349ae", owner="FEX-Emu", license="Apache-2.0", bcond="integration" },
>  { name="fex-posixtest-bins", ref="9ae2963", owner="FEX-Emu", package="posixtest", version="1.5.2", license="GPL-2.0-or-later", bcond="integration" },

These are not used and part of the packaging, instead we should strip it at prep and just drop these lines.
[x]: Latest sources are packaged

> #global date 20240101
> #global commit 4a0e9289bf443fec9fc56af25f90e2a100ba7a41

This should be updated to recent values because the hash doesn't seem to exist in the FEX GitHub repo, so I got confused when I looked over it.

[x]: Package builds and installs

Once the following things are fixed, we should be good to go.

Comment 8 Davide Cavalca 2024-08-16 20:45:44 UTC
> These are not used and part of the packaging, instead we should strip it at prep and just drop these lines.

So right now these are gated by the integration bcond, as afaik they are needed for the integration tests, but not for the actual build. I can drop them, but I suspect the integration tests just won't work (we disable them by default, so not an issue for the package itself in practice). I also just realized now instead of doing conditional sources I should do conditional unpacking to ensure the src.rpm is reproducible. Will fix that.

> This should be updated to recent values because the hash doesn't seem to exist in the FEX GitHub repo, so I got confused when I looked over it.

Oh yeah that was just copypasta from another package :)

Comment 9 Davide Cavalca 2024-08-16 22:57:18 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- drop prebuilt externals
- rework externals logic to ensure reproducibility
- use a real commit
- fix license tag

Comment 10 Teoh Han Hui 2024-08-17 17:28:12 UTC
If I understand correctly...

```
%ifarch aarch64
```

should be

```
%ifarch %{arm64}
```

as well?

Comment 11 Neal Gompa 2024-08-17 19:30:39 UTC
Yes.

Comment 12 Davide Cavalca 2024-08-18 19:28:13 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- fix one more arch macro usage
- backport upstream PRs for binfmts, gdb, static library
- drop useless binary that's being removed upstream

Comment 13 Neal Gompa 2024-08-18 21:26:05 UTC
The binfmts are not packaged correctly with that patch applied, the systemd style files need to be installed, and "%binfmt_apply" macro is no longer needed.

Comment 14 Davide Cavalca 2024-08-19 01:00:18 UTC
They're installed by cmake in /usr/lib/binfmt.d, which matches %_binfmtdir so it should be fine. https://docs.fedoraproject.org/en-US/packaging-guidelines/#_binfmt_d_sysctl_d_and_tmpfiles_d  says we still need the macro, though it doesn't say anything about restarting the service in %postun. Looking over sourcegraph is inconclusive -- e.g. ksh does the try-restart too, but it does it in %post, so I'm not sure.

Comment 16 Davide Cavalca 2024-08-19 13:51:41 UTC
Thanks! Put up https://pagure.io/packaging-committee/pull-request/1385 to get the guidelines updated.

Comment 17 Davide Cavalca 2024-08-19 14:03:35 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- drop unneeded %post for binfmts
- tidy up the spec a bit

Comment 18 Teoh Han Hui 2024-08-20 20:49:09 UTC
Is it intentional that this package does not have something like:

```
Recommends:     squashfs-tools
Recommends:     squashfuse

Suggests:       erofs-fuse
Suggests:       erofs-utils
```

Comment 19 Davide Cavalca 2024-08-21 15:31:11 UTC
Yes, I left that out because we haven't decided yet what to use for the rootfs (I know Neal was looking at erofs but it might require some enablement work in Kiwi first) and didn't want to be prescriptive at this stage. I suppose we could add Suggests for all the formats fex supports?

Comment 20 Davide Cavalca 2024-08-21 16:24:07 UTC
Reading the sources again, it looks like FEX has built-in logic to handle squashfs and erofs, so it probably makes sense to have all these as Recommends.

Comment 21 Davide Cavalca 2024-08-21 16:33:08 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- disable x86_debug by default
- add Recommends for rootfs support packages

Comment 22 Teoh Han Hui 2024-08-21 16:35:09 UTC
I think Recommends for erofs is not a good idea for now, as it'd just increase the download size for the FEX RootFS by a lot. Perhaps Suggests for now, and could always be switched to Recommends later when the size reduction has been figured out?

Comment 23 Davide Cavalca 2024-08-21 17:04:28 UTC
(In reply to Teoh Han Hui from comment #22)
> I think Recommends for erofs is not a good idea for now, as it'd just
> increase the download size for the FEX RootFS by a lot. Perhaps Suggests for
> now, and could always be switched to Recommends later when the size
> reduction has been figured out?

I don't think this will matter in practice. For Fedora we're going to package the rootfs in the distribution itself, so users will get it via the package manager.

Comment 24 Davide Cavalca 2024-08-21 17:05:27 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- add README.fedora to clarify this only works on 4k page-size systems
- number externals starting from 100 to avoid confusion

Comment 25 Neal Gompa 2024-08-24 10:43:05 UTC
> # Removes GDBSymbol integration
> Patch:      %{forgeurl}/pull/3969.patch

This patch should be dropped, as upstream has rejected it. The GDBSymbol integration should be split into its own subpackage.

Comment 26 Davide Cavalca 2024-08-27 21:57:08 UTC
Spec URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu.spec
SRPM URL: https://dcavalca.fedorapeople.org/review/fex-emu/fex-emu-2408-1.fc42.src.rpm

Changelog:
- add back GDBSymbol integration and move it to a subpackage

Comment 27 Neal Gompa 2024-08-27 22:20:26 UTC
I think this is finally at the point where it's good to land, so...

PACKAGE APPROVED.

Comment 28 Fedora Admin user for bugzilla script actions 2024-08-27 22:22:24 UTC
The Pagure repository was created at https://src.fedoraproject.org/rpms/fex-emu

Comment 29 Fedora Update System 2024-08-27 22:37:54 UTC
FEDORA-2024-a25fc9b40c (fex-emu-2408-1.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-a25fc9b40c

Comment 30 Fedora Update System 2024-08-27 23:03:40 UTC
FEDORA-2024-3ba3f0dc6e (fex-emu-2408-1.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-3ba3f0dc6e

Comment 31 Fedora Update System 2024-08-27 23:28:54 UTC
FEDORA-2024-4e0edf75bb (fex-emu-2408-1.fc40) has been submitted as an update to Fedora 40.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-4e0edf75bb

Comment 32 Fedora Update System 2024-08-27 23:50:52 UTC
FEDORA-2024-95bb281344 (fex-emu-2408-1.fc39) has been submitted as an update to Fedora 39.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-95bb281344

Comment 33 Fedora Update System 2024-08-28 01:59:02 UTC
FEDORA-2024-a25fc9b40c (fex-emu-2408-1.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 34 Fedora Update System 2024-08-29 01:34:52 UTC
FEDORA-2024-4e0edf75bb (fex-emu-2408-1.fc40) has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 35 Fedora Update System 2024-08-29 01:58:39 UTC
FEDORA-2024-3ba3f0dc6e has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf install --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-3ba3f0dc6e \*`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3ba3f0dc6e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 36 Fedora Update System 2024-08-29 02:38:15 UTC
FEDORA-2024-95bb281344 has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf install --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-95bb281344 \*`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-95bb281344

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 37 Fedora Update System 2024-08-30 03:00:29 UTC
FEDORA-2024-95bb281344 (fex-emu-2408-1.fc39) has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 38 Fedora Update System 2024-09-03 07:32:22 UTC
FEDORA-2024-3ba3f0dc6e (fex-emu-2408-1.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.