Bug 2267683 - Review Request: wasi-sdk - The WebAssembly System Interface SDK
Summary: Review Request: wasi-sdk - The WebAssembly System Interface SDK
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Nobody's working on this, feel free to take it
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/WebAssembly/wasi-sdk
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-03-04 12:15 UTC by Jan Horak
Modified: 2024-03-08 11:08 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Jan Horak 2024-03-04 12:15:03 UTC
Spec URL: https://xhorak.fedorapeople.org/wasi-sdk.spec
SRPM URL: https://xhorak.fedorapeople.org/wasi-sdk-20-1.fc39.src.rpm
Description: The WebAssembly System Interface SDK
Fedora Account System Username: xhorak

Note: this is build dependency for the Firefox rlbox implementation - ie. sandboxing of the C/C++ libraries by using webassembler.

Comment 1 Fedora Review Service 2024-03-04 12:15:06 UTC
There seems to be some problem with the following file.
SRPM URL: https://xhorak.fedorapeople.org/wasi-sdk-20-1.fc39.src.rpm
Fetching it results in a 404 Not Found error.
Please make sure the URL is correct and publicly available.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 2 Jan Horak 2024-03-04 13:19:04 UTC
fedora-review-service-build

Comment 3 Jan Horak 2024-03-04 13:43:58 UTC
[fedora-review-service-build]

Comment 4 Fedora Review Service 2024-03-04 14:52:28 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/7103368
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2267683-wasi-sdk/fedora-rawhide-x86_64/07103368-wasi-sdk/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 5 Elliott Sales de Andrade 2024-03-06 10:28:33 UTC
A submodule-recursive checkout include LLVM; I don't think you should be including that.

Comment 6 Jan Horak 2024-03-06 13:04:55 UTC
I understand, but the wasi-sdk is basically using the sub modules to build the package. Do you have any other suggestions?
What worries me is the DLLVM_TARGETS_TO_BUILD define and some more:
cmake -G Ninja \
		-DCMAKE_BUILD_TYPE=MinSizeRel \
		-DLLVM_ENABLE_TERMINFO=OFF \
		-DLLVM_ENABLE_ZLIB=OFF \
		-DLLVM_ENABLE_ZSTD=OFF \
		-DLLVM_STATIC_LINK_CXX_STDLIB=ON \
		-DLLVM_HAVE_LIBXAR=OFF \
		-DCMAKE_INSTALL_PREFIX=$(PREFIX) \
		-DLLVM_INCLUDE_TESTS=OFF \
		-DLLVM_INCLUDE_UTILS=OFF \
		-DLLVM_INCLUDE_BENCHMARKS=OFF \
		-DLLVM_INCLUDE_EXAMPLES=OFF \
		-DLLVM_TARGETS_TO_BUILD=WebAssembly \
		-DLLVM_DEFAULT_TARGET_TRIPLE=wasm32-wasi \
		-DLLVM_ENABLE_PROJECTS="lld;clang;clang-tools-extra" \
		$(if $(patsubst 9,,$(CLANG_VERSION)), \
	             $(if $(patsubst 10,,$(CLANG_VERSION)), \
		          -DDEFAULT_SYSROOT=../share/wasi-sysroot, \
			  -DDEFAULT_SYSROOT=$(PREFIX)/share/wasi-sysroot), \
		     -DDEFAULT_SYSROOT=$(PREFIX)/share/wasi-sysroot) \
		-DLLVM_INSTALL_BINUTILS_SYMLINKS=TRUE \
		-DLLVM_ENABLE_LIBXML2=OFF \
		$(LLVM_CMAKE_FLAGS) \
		$(LLVM_PROJ_DIR)/llvm

Comment 7 Josh Stone 2024-03-06 17:25:33 UTC
The system LLVM_TARGETS_TO_BUILD already includes WebAssembly, and I think most of the other settings are fine as well.

The main tricky thing I see is the *default* target and sysroot, because the system tools default to native compilation. If the sdk wants to use clang without specifying the target, you may need something like a wrapper script to set the target and sysroot in that context.

The other one I'm not sure about is -DLLVM_STATIC_LINK_CXX_STDLIB=ON, as I haven't dealt with that.

cc Tom for additional LLVM feedback.

Comment 8 Jan Horak 2024-03-07 07:24:53 UTC
Thanks for the feedback guys. I'd rather put the whole Makefile there:
https://github.com/WebAssembly/wasi-sdk/blob/main/Makefile

I forget that we also need compiler-rt lib. Without it I get some missing symbols (we've tried really hard without it together with jstanek):
wasm-ld: error: /home/jhorak/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc.a(intscan.o): undefined symbol: __multi3
wasm-ld: error: /home/jhorak/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc++.a(algorithm.cpp.o): undefined symbol: __lttf2
wasm-ld: error: /home/jhorak/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc.a(intscan.o): undefined symbol: __multi3
wasm-ld: error: /home/jhorak/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc++.a(int128_builtins.cpp.o): undefined symbol: _
_divti3
wasm-ld: error: /home/jhorak/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc++.a(int128_builtins.cpp.o): undefined symbol: _
_divti3

( https://github.com/mono/mono/issues/18560#issuecomment-578902458 )

And we also need libcxx which is not in the rpms.

Comment 9 Elliott Sales de Andrade 2024-03-07 11:27:51 UTC
compiler-rt and libcxx are separate packages from llvm:
https://src.fedoraproject.org/rpms/compiler-rt
https://src.fedoraproject.org/rpms/libcxx

Comment 10 Jan Horak 2024-03-07 19:47:46 UTC
(In reply to Elliott Sales de Andrade from comment #9)
> compiler-rt and libcxx are separate packages from llvm:
> https://src.fedoraproject.org/rpms/compiler-rt
> https://src.fedoraproject.org/rpms/libcxx

I don't think any of these has been built for the wasi, ie. using toolchain file -DCMAKE_TOOLCHAIN_FILE=$(ROOT_DIR)/wasi-sdk.cmake. 

I'm not sure if the worth the trouble to trying recreate what already exists. Also I doubt the package will be used by anyone but Firefox package only as a BuildRequires.

Comment 11 Josh Stone 2024-03-07 22:03:30 UTC
It looks like build/config.BUILT is the only thing that's really unique to the wasi-sdk. The rest can potentially use existing packages.

(In reply to Jan Horak from comment #10)
> I don't think any of these has been built for the wasi, ie. using toolchain
> file -DCMAKE_TOOLCHAIN_FILE=$(ROOT_DIR)/wasi-sdk.cmake. 

Just because they haven't been, doesn't mean they can't. They could probably produce noarch wasm packages, just like rust builds a rust-std-static-wasm32-wasi subpackage, for example.

> Also I doubt the package will be used by anyone but Firefox package
> only as a BuildRequires.

Maybe that's true at first, but we should still design for general use.

Comment 12 Jan Horak 2024-03-08 11:08:00 UTC
I've spent few hours trying to build the compile-rt without any luck [1] - mostly ending by unable to find the llvm compiler. Unless someone steps in and do the necessary work - ie. build the compiler-rt and libc++ for WASI in the end we'll have to bundle the wasi-sdk in the Firefox package and rebuild it every time.


[1] https://pastebin.com/684d9jru


Note You need to log in before you can comment on or make changes to this bug.