Currently, libarrow is built without several optional components. In particular, I am looking for S3 support. Is it possible to build it with more of the optional features enabled? ``` r arrow_info() #> Arrow package version: 11.0.0.3 #> #> Capabilities: #> #> dataset TRUE #> substrait FALSE #> parquet TRUE #> json TRUE #> s3 FALSE #> gcs FALSE #> utf8proc TRUE #> re2 TRUE #> snappy TRUE #> gzip TRUE #> brotli TRUE #> zstd TRUE #> lz4 TRUE #> lz4_frame TRUE #> lzo FALSE #> bz2 TRUE #> jemalloc FALSE #> mimalloc FALSE #> #> To reinstall with more optional capabilities enabled, see #> https://arrow.apache.org/docs/r/articles/install.html #> #> Memory: #> #> Allocator system #> Current 0 bytes #> Max 0 bytes #> #> Runtime: #> #> SIMD Level avx2 #> Detected SIMD Level avx2 #> #> Build: #> #> C++ Library Version 11.0.0 #> C++ Compiler GNU #> C++ Compiler Version 13.1.1 ``` Reproducible: Always
enabling s3 requires Amazon aws-lc, and maybe more build-time dependencies. aws-lc doesn't exist in Fedora, at least not that I can find. (Amazon likes to download sources at build time, which doesn't work in koji/mock builds which disable network access.) When someone adds aws-lc to fedora (or when I find time to add it) then S3 can be enabled. setting CLOSED/WONTFIX. When aws-lc gets added you can reopen this bz or open a new bz.
It does build https://copr.fedorainfracloud.org/coprs/fed500/aws-lc/build/6182074 though will conflict with OpenSSL. Some licensing issues also need to be addressed.
T(In reply to Benson Muite from comment #2) > It does build > https://copr.fedorainfracloud.org/coprs/fed500/aws-lc/build/6182074 though > will conflict > with OpenSSL. Some licensing issues also need to be addressed. That's good to know. But it doesn't help until the aws-lc is actually added to fedora. Do you want to elaborate on the licensing issues? aws-lc appears to be licensed the same as OpenSSL and BoringSSL (which is not surprising, since it's derived from them.) All things considered, it would be better, and easier, if Arrow's S3 just used OpenSSL.
I am somewhat confused by this requirement. The arrow documentation (https://arrow.apache.org/docs/r/articles/install.html#libraries) states that the only additional dependencies for S3 and GCS would be CURL and OpenSSL.
Simply flipping use_s3 or -DARROW_S3=ON results in ... -- stderr output is: CMake Error at aws_lc_ep-stamp/download-aws_lc_ep.cmake:170 (message): Each download failed! error: downloading 'https://github.com/awslabs/aws-lc/archive/v1.3.0.tar.gz' failed ... (full log at https://kojipkgs.fedoraproject.org//work/tasks/924/103480924/build.log) Setting aside the fact that Fedora koji builds literally can not download third party sources. And then there's the work to patch up the cmake to use the system's SSL lib, which looks, to me, like it's really aws-lc and not, as the documentation claims, OpenSSL. (There are lies, damned lies, and documentation. :-)) If you know something I don't know then send a PR to https://src.fedoraproject.org/rpms/libarrow
s/use the system's SSL lib/use a theoretical unbundled aws-lc SSL lib/