Bug 1808128 - enjarify fails to build with Python 3.9: enjarify.jvm.error.ClassfileLimitExceeded
Summary: enjarify fails to build with Python 3.9: enjarify.jvm.error.ClassfileLimitExc...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: enjarify
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedFreezeException
: 1842085 1871481 (view as bug list)
Depends On:
Blocks: F33BetaFreezeException PYTHON39 F33FTBFS F33FailsToInstall 1841628
TreeView+ depends on / blocked
 
Reported: 2020-02-27 21:29 UTC by Miro Hrončok
Modified: 2020-09-18 10:07 UTC (History)
7 users (show)

Fixed In Version: enjarify-1.0.3-14.fc34 enjarify-1.0.3-14.fc33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-11 23:01:16 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2020-02-27 21:29:10 UTC
enjarify fails to build with Python 3.9.0a4.

+ /usr/bin/python3 -m enjarify.runtests
Traceback (most recent call last):
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 116, in toClassFile
    pool, rest_stream = classFileAfterPool(cls, opts=opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 100, in classFileAfterPool
    writeMethods(pool, stream, cls.data.methods, opts=opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 65, in writeMethods
    code_attrs = writebytecode.finishCodeAttrs(pool, code_irs, opts=opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writebytecode.py", line 75, in finishCodeAttrs
    return {irdata.method: writeCodeAttributeTail(pool, irdata, opts=opts) for irdata in code_irs}
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writebytecode.py", line 75, in <dictcomp>
    return {irdata.method: writeCodeAttributeTail(pool, irdata, opts=opts) for irdata in code_irs}
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writebytecode.py", line 99, in writeCodeAttributeTail
    raise error.ClassfileLimitExceeded()
enjarify.jvm.error.ClassfileLimitExceeded
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/runtests.py", line 45, in <module>
    executeTest('test{}'.format(i), opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/runtests.py", line 30, in executeTest
    classes, errors = translate(rawdex, opts=opts, allowErrors=False)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/main.py", line 38, in translate
    class_data = writeclass.toClassFile(cls, opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 119, in toClassFile
    pool, rest_stream = classFileAfterPool(cls, opts=options.ALL)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 100, in classFileAfterPool
    writeMethods(pool, stream, cls.data.methods, opts=opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeclass.py", line 64, in writeMethods
    code_irs.append(writebytecode.getCodeIR(pool, method, opts=opts))
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writebytecode.py", line 21, in getCodeIR
    irdata = writeir.writeBytecode(pool, method, opts)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeir.py", line 597, in writeBytecode
    VISIT_FUNCS[instr.type](method, dex, instr_d, type_data, block, instr)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeir.py", line 380, in visitFillArrayData
    block.fillarraydata(_arrStoreOps.get(elet, AASTORE), st, arrdata)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeir.py", line 146, in fillarraydata
    self.fillarraysub(op, [partial(self.const, val, stype) for val in vals])
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeir.py", line 129, in fillarraysub
    cb()
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/writeir.py", line 118, in const
    self.add(ir.PrimConstant(stype, val, pool=pool))
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/ir.py", line 82, in __init__
    self.bytecode = calc.calc(st, val)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 179, in calc
    return calcDouble(val)
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 162, in calcDouble
    def calcDouble(x): return _calcDouble(normalizeDouble(x))
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 157, in _calcDouble
    return _calcLong(mantissa) + bytes([L2D]) + exponent_parts
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 66, in _calcLong
    result += _calcInt(low) + bytes([I2L, LXOR])
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 50, in _calcInt
    return _calcInt(high) + _calcInt(16) + bytes([ISHL]) + _calcInt(low) + bytes([IXOR])
  File "/builddir/build/BUILD/enjarify-1.0.3/enjarify/jvm/constants/calc.py", line 47, in _calcInt
running test test1
running test test2
running test test3
running test test4
    assert high
AssertionError


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01255071-enjarify/

For all our attempts to build enjarify with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/enjarify/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-05-25 13:02:13 UTC
This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 2 Miro Hrončok 2020-05-29 07:12:02 UTC
Python 3.9 update: The f33-python side tag is currently being merged.

New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:

    $ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3

And build the packages normally after that.

Comment 3 Tomáš Hrnčiar 2020-06-09 06:19:28 UTC
*** Bug 1842085 has been marked as a duplicate of this bug. ***

Comment 4 Igor Raits 2020-06-16 18:46:20 UTC
Hello,

This is the first reminder (step 3 from https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/#_package_removal_for_long_standing_ftbfs_and_fti_bugs).

If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.

Comment 5 Igor Raits 2020-07-08 09:24:49 UTC
Hello,

This is the second reminder (step 4 from https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/#_package_removal_for_long_standing_ftbfs_and_fti_bugs).

If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.

Comment 6 Zbigniew Jędrzejewski-Szmek 2020-08-07 09:50:22 UTC
https://github.com/Storyyeller/enjarify/issues/17

Comment 7 Ben Cotton 2020-08-11 13:11:31 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 8 Miro Hrončok 2020-08-22 10:30:43 UTC
Hello. This comment is mass posted to all 16 bugzillas for packages that failed to rebuild with Python 3.9 (and, in most cases, also fail to install on Fedora 33 and 34). Apologies if some of the suggestions or questions don't apply 100%.

Fedora 33 Beta Freeze starts next week on Tuesday, 2020-08-25. I haven't spammed this bugzilla because it is in ASSIGNED, or I've seen upstream has been notified.

Please consider what to do here. Is this failure critical? For example, can a test be temporarily skipped to make this package build? Is it possible to fix or workaround the failure before Fedora 33 Beta? Or before Fedora 33 Final? If not, would it make sense to retire the package (even temporarily) before the release to unblock updates from Fedora 31 or Fedora 32?

In case you need any assistance with a Python-related problem with the build, don't hesitate to let me know and I'll try to have a look.

Thank you!

Comment 9 Zbigniew Jędrzejewski-Szmek 2020-08-22 10:46:23 UTC
I'm waiting for the patch to be applied in python3.9.

Comment 10 Miro Hrončok 2020-08-23 17:03:32 UTC
Do you want me to backport the fix or is it OK to wait for rc2 (scheduled for mid-September)?

Comment 11 Zbigniew Jędrzejewski-Szmek 2020-08-26 07:55:44 UTC
I pushed a build that simply ignores the test results. The package is most likely broken 
in actual use with current python 3.9-rc1, but it's not very widely used, so I think it's
OK to keep it like that for a few weeks.

Comment 12 Miro Hrončok 2020-08-26 09:16:34 UTC
Could you please push a F33 update as well? Either now or after rc2 is out, but let's keep this open.

Comment 13 Fedora Update System 2020-08-26 11:09:48 UTC
FEDORA-2020-702f887360 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-702f887360

Comment 14 Miro Hrončok 2020-08-26 12:03:54 UTC
*** Bug 1871481 has been marked as a duplicate of this bug. ***

Comment 15 Fedora Update System 2020-08-26 17:07:15 UTC
FEDORA-2020-702f887360 has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-702f887360`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-702f887360

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Igor Raits 2020-08-27 05:35:21 UTC
Hello,

Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (ignatenkobrain).

All subpackages of a package against which this bug was filled are now installable or removed from Fedora 34.

Thanks for taking care of it!

Comment 17 Fedora Blocker Bugs Application 2020-09-10 16:49:59 UTC
Proposed as a Freeze Exception for 33-beta by Fedora user churchyard using the blocker tracking app because:

 The package does not install without the proposed update.

Comment 18 Adam Williamson 2020-09-11 21:06:01 UTC
+3 votes in https://pagure.io/fedora-qa/blocker-review/issue/77 , setting accepted. We usually accept FTBFS / fails to install as FEs.

Comment 19 Fedora Update System 2020-09-11 23:01:16 UTC
FEDORA-2020-702f887360 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 20 Miro Hrončok 2020-09-18 10:07:15 UTC
https://src.fedoraproject.org/rpms/enjarify/pull-request/1 to enable the tests again with 3.9.0rc2


Note You need to log in before you can comment on or make changes to this bug.