Bug 2278011 - jupyterlab: endian issues in bundled libzip interface
Summary: jupyterlab: endian issues in bundled libzip interface
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: jupyterlab
Version: rawhide
Hardware: s390x
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Lumír Balhar
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-04-30 16:32 UTC by Jerry James
Modified: 2024-05-25 08:59 UTC (History)
2 users (show)

Fixed In Version: jupyterlab-4.2.0-1.fc41
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-05-25 08:59:25 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jerry James 2024-04-30 16:32:46 UTC
Description of problem:
I am still trying to create a python-jupytext package.  The build fails on s390x, a big endian architecture, during the "jlpm install" step:

  ➤ YN0001: │ Libzip Error: typescript@patch:typescript@npm%3A5.0.4#~builtin<compat/typescript>::version=5.0.4&hash=85af82: Failed to open the cache entry for typescript@npm:5.0.4: Unknown error 469762048
      at Wr.makeLibzipError (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:12847)
      at new Wr (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:12288)
      at L (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3109)
      at /usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3173
      at Jx (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:389:11143)
      at Bg.factory (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3164)
      at get baseFs [as baseFs] (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:41188)
      at Bg.lstatPromise (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:35831)
      at bo.lstatPromise (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:35838)
      at iK (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:308:7267)
  ➤ YN0001: │ Libzip Error: typescript@patch:typescript@npm%3A4.9.5#~builtin<compat/typescript>::version=4.9.5&hash=23ec76: Failed to open the cache entry for typescript@npm:4.9.5: Unknown error 469762048
      at Wr.makeLibzipError (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:12847)
      at new Wr (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:12288)
      at L (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3109)
      at /usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3173
      at Jx (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:389:11143)
      at Bg.factory (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:428:3164)
      at get baseFs [as baseFs] (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:41188)
      at Bg.lstatPromise (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:35831)
      at bo.lstatPromise (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:313:35838)
      at iK (/usr/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:308:7267)
  ➤ YN0000: └ Completed in 0s 356ms
  ➤ YN0000: Failed with errors in 0s 689ms

That unknown error, 469762048, is 0x1c000000 in hex.  That is clearly a byte-reversed number: 0x1c (28) is ZIP_ER_OPNOTSUPP, operation not supported.  In %{python3_sitelib}/jupyterlab/staging/yarn.lock, I see mention of @yarnpkg/libzip, version 2.3.0.  It appears to be included in %{python3_sitelib}/jupyterlab/staging/yarn.js as a base64-encoded web assembly.

The endianness issues with version 2.3.0 appear to have been corrected by upstream already.  I see changes like this between 2.3.0 and the current version, 3.0.1:

diff -durpN berry--yarnpkg-libzip-2.3.0/packages/yarnpkg-libzip/sources/libzipAsync.js berry--yarnpkg-libzip-3.0.1/packages/yarnpkg-libzip/sources/libzipAsync.js
--- berry--yarnpkg-libzip-2.3.0/packages/yarnpkg-libzip/sources/libzipAsync.js  2023-03-16 13:45:30.000000000 -0600
+++ berry--yarnpkg-libzip-3.0.1/packages/yarnpkg-libzip/sources/libzipAsync.js  2024-01-30 08:45:25.000000000 -0700
@@ -111,15 +107,15 @@ var createModule = (function() {
         case "i8":
           return HEAP8[ptr >> 0];
         case "i16":
-          return HEAP16[ptr >> 1];
+          return LE_HEAP_LOAD_I16((ptr >> 1) * 2);
         case "i32":
-          return HEAP32[ptr >> 2];
+          return LE_HEAP_LOAD_I32((ptr >> 2) * 4);
         case "i64":
-          return HEAP32[ptr >> 2];
+          return LE_HEAP_LOAD_I32((ptr >> 2) * 4);
         case "float":
-          return HEAPF32[ptr >> 2];
+          return LE_HEAP_LOAD_F32((ptr >> 2) * 4);
         case "double":
-          return HEAPF64[ptr >> 3];
+          return LE_HEAP_LOAD_F64((ptr >> 3) * 8);
         default:
           abort("invalid type for getValue: " + type);
       }

The issue is that the zip format stores integers in little endian order (see https://libzip.org/specifications/extrafld.txt), so they have to be byte-reversed as they are read from the zip file on a big endian architecture.

Could you do something to force @yarnpkg/libzip to version 3.0.1 in that base64-encoded web assembly?

Version-Release number of selected component (if applicable):
jupyterlab-4.1.6-1.fc41

How reproducible:
Always

Steps to Reproduce:
1. Visit https://copr.fedorainfracloud.org/coprs/jjames/Jupyter/builds/
2. Find the python-jupytext build
3. Click through to any of the build directories and download the source rpm, python-jupytext-1.16.1-1.fc41.src.rpm
4. Attempt to build the source RPM on an s390x machine; e.g., do a scratch build

Actual results:
The build fails with libzip errors.

Expected results:
The build should succeed like it does on x86_64, i386, and ppc64le.  The aarch64 build failure is due to a nodejs bug.

Additional info:

Comment 1 Jerry James 2024-04-30 17:55:59 UTC
(In reply to Jerry James from comment #0)
> Steps to Reproduce:
> 1. Visit https://copr.fedorainfracloud.org/coprs/jjames/Jupyter/builds/
> 2. Find the python-jupytext build
> 3. Click through to any of the build directories and download the source
> rpm, python-jupytext-1.16.1-1.fc41.src.rpm
> 4. Attempt to build the source RPM on an s390x machine; e.g., do a scratch
> build

Well, except I added "ExcludeArch: s390x" so I can move forward on reviews for python-jupytext and some consuming packages.  You'll have to unpack the source RPM and remove that first.

Comment 2 Lumír Balhar 2024-05-02 10:31:39 UTC
Thanks for the report. I'm afraid I cannot easily fix it in Fedora so an upstream issue report might be better. Should I create one? I mean, I can try to change the version in the file you've mentioned but I have no idea what impact it might have on jupyterlab itself or other packages so better would be to fix this upstream for everyone.

Comment 3 Jerry James 2024-05-02 21:19:39 UTC
That's the reply I expected, although one can always hope for a miracle. :-)  Actually, it looks like upstream has already bumped the @yarnpkg/libzip version up to 3.0.1, in this commit:

https://github.com/jupyterlab/jupyterlab/commit/cd8a0291f9cc46b7645a270848e5ca0ba0ece847

That change is included in version 4.2.0rc0, so I guess I have to wait for version 4.2.0 to be released.

Thanks for the reply.

Comment 4 Lumír Balhar 2024-05-03 09:42:06 UTC
Let me check how the rc0 would work in rawhide.

Comment 5 Lumír Balhar 2024-05-03 09:55:19 UTC
Hmm, it builds fine but unfortunately, python-notebook requires jupyterlab < 4.2 so I'd have to patch that one as well.

Comment 6 Jerry James 2024-05-03 16:55:59 UTC
Thanks for trying.  Let's leave things alone for now.  I'll simply ExcludeArch: s390x for now, and we can revisit the issue once 4.2.0 final is released.

Comment 7 Fedora Update System 2024-05-25 08:55:44 UTC
FEDORA-2024-77d6c4fc2f (jupyterlab-4.2.0-1.fc41 and python-notebook-7.2.0-1.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-77d6c4fc2f

Comment 8 Fedora Update System 2024-05-25 08:59:25 UTC
FEDORA-2024-77d6c4fc2f (jupyterlab-4.2.0-1.fc41 and python-notebook-7.2.0-1.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.