Bug 2154872 - nftables fails to build with Python 3.12: ModuleNotFoundError: No module named 'distutils'
Summary: nftables fails to build with Python 3.12: ModuleNotFoundError: No module nam...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: nftables
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Kevin Fenzi
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.12
TreeView+ depends on / blocked
 
Reported: 2022-12-19 13:07 UTC by Tomáš Hrnčiar
Modified: 2023-04-22 15:25 UTC (History)
4 users (show)

Fixed In Version: nftables-1.0.7-1.fc39
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-04-01 19:22:11 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2022-12-19 13:07:28 UTC
nftables fails to build with Python 3.12.0a3.

Traceback (most recent call last):
  File "/builddir/build/BUILD/nftables-1.0.5/py/setup.py", line 2, in <module>
    from distutils.core import setup
ModuleNotFoundError: No module named 'distutils'

Remove the distutils package. It was deprecated in Python 3.10 by PEP 632 “Deprecate distutils module”. For projects still using distutils and cannot be updated to something else, the setuptools project can be installed: it still provides distutils. (Contributed by Victor Stinner in gh-92584.)

If your package is listed in [0], you may workaround this issue by BuildRequiring python-setuptools, note that adding such BuildRequires might however hide some transitive dependency problem, if the distutils import comes from a dependency.
Cooperation with upstream is recommended. Additional context [1].

[0] https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/6BHNAWHE7M5VY3YQVJLOYHLY4M7KIFFN/

[1] https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/N6ITYHLRWIDNYNXGPYG2ZHF3ZLQWZN7L/

https://docs.python.org/3.12/whatsnew/3.12.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-rawhide-x86_64/05129266-nftables/

For all our attempts to build nftables with Python 3.12, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/package/nftables/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.12:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/

Let us know here if you have any questions.

Python 3.12 is planned to be included in Fedora 39. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.12.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Ben Cotton 2023-02-07 15:01:36 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.

Comment 2 Miro Hrončok 2023-03-31 11:58:35 UTC
Bumping severity to high, because this is required for the @core group to install.

Comment 3 Miro Hrončok 2023-03-31 12:16:28 UTC
I've tried to BR setuptools or even backport this commit:

https://git.netfilter.org/nftables/commit/py?id=1acc2fd48c755a8931fa87b8d0560b750316059f

However, the build fails because setuptools will create an egg:

+ sed -i -e 's/\(sofile=\)".*"/\1"libnftables.so.1.1.0"/' /builddir/build/BUILDROOT/nftables-1.0.5-2.fc39.x86_64//usr/lib/python3.12/site-packages/nftables/nftables.py
sed: can't read /builddir/build/BUILDROOT/nftables-1.0.5-2.fc39.x86_64//usr/lib/python3.12/site-packages/nftables/nftables.py: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.F6MSMx (%install)



$ pwd
/var/lib/mock/fedora-rawhide-python312/root/builddir/build/BUILDROOT/nftables-1.0.5-2.fc39.x86_64/usr/lib

$ tree python3.12/
python3.12/
└── site-packages
    └── nftables-0.1-py3.12.egg
        ├── EGG-INFO
        │   ├── dependency_links.txt
        │   ├── not-zip-safe
        │   ├── PKG-INFO
        │   ├── SOURCES.txt
        │   └── top_level.txt
        └── nftables
            ├── __init__.py
            ├── nftables.py
            ├── __pycache__
            │   ├── __init__.cpython-312.pyc
            │   └── nftables.cpython-312.pyc
            └── schema.json


The problem is that the Makefile calls setup.py directly with:

		$(PYTHON_BIN) setup.py build --build-base $(abs_builddir) \
		install --prefix $(DESTDIR)$(prefix)

And setuptools will install an egg if invoked like this.

This sed fixes that:

  sed -i 's/--prefix $(DESTDIR)$(prefix)/--root $(DESTDIR) --prefix $(prefix)/' py/Makefile*

I'd appreciate it if you could submit that upstream, I don't even have an account in their bugzilla.


Downstream PR: https://src.fedoraproject.org/rpms/nftables/pull-request/5

Comment 4 Kevin Fenzi 2023-04-01 19:05:23 UTC
I pulled it in. 

In the distant past, I just posted patches to their mailing list, I also don't have any bugzilla account there. 

Perhaps Phil does and could submit this?

Comment 5 Fedora Update System 2023-04-01 19:20:13 UTC
FEDORA-2023-d32742130c has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-d32742130c

Comment 6 Fedora Update System 2023-04-01 19:22:11 UTC
FEDORA-2023-d32742130c has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 7 Phil Sutter 2023-04-21 17:26:32 UTC
(In reply to Kevin Fenzi from comment #4)
> I pulled it in. 
> 
> In the distant past, I just posted patches to their mailing list, I also
> don't have any bugzilla account there. 
> 
> Perhaps Phil does and could submit this?

No account needed, netfilter development is still largely email-based and
netfilter-devel.org accepts emails from non-subscribers, too.

I had a look at the proposed changes myself, but the setup.py parameter change
didn't work for me because I don't set DESTDIR when invoking 'make install'.
(It's not needed, --prefix argument in configure is sufficient for me.)

Miro, can you tell what should happen if DESTDIR is unset? Not pass '--root' or
something else?

Comment 8 Miro Hrončok 2023-04-21 18:14:58 UTC
When DESTDIR is not set, I'd not pass --root.

Comment 9 Phil Sutter 2023-04-22 10:22:32 UTC
(In reply to Miro Hrončok from comment #8)
> When DESTDIR is not set, I'd not pass --root.

OK, thanks. Turns out Makefile.am conditionals is not nice, so I ended up with this for now:

| diff --git a/py/Makefile.am b/py/Makefile.am
| index 215ecd9e47513..2acc4b3f6210d 100644
| --- a/py/Makefile.am
| +++ b/py/Makefile.am
| @@ -5,9 +5,9 @@ EXTRA_DIST = setup.py __init__.py nftables.py schema.json
|                 $(PYTHON_BIN) setup.py build --build-base $(abs_builddir)
|  
|  install-exec-local:
| -       cd $(srcdir) && \
| +       destdir="$(DESTDIR)"; cd $(srcdir) && \
|                 $(PYTHON_BIN) setup.py build --build-base $(abs_builddir) \
| -               install --prefix $(DESTDIR)$(prefix)
| +               install $${destdir:+--root "$$destdir"} --prefix "$(prefix)"
|  
|  uninstall-local:
|         rm -rf $(DESTDIR)$(prefix)/lib*/python*/site-packages/nftables

Unless you have a better idea. ;)

What bothers me is there's still this deprecation warning:

| /usr/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.

Do you by chance know how to avoid it? For some reason, Python module packaging feels like rocket science to me. :(

Comment 10 Miro Hrončok 2023-04-22 15:25:39 UTC
Yes, setup.py install is deprecated and yes, Python packaging wrapped in autotools/cmake etc. is indeed rocket science.

This tutorial describes how to create a simple Python package https://packaging.python.org/en/latest/tutorials/packaging-projects/ -- but it covers a "Python only" package, not something that builds multiple things + Python bindings etc.


Note You need to log in before you can comment on or make changes to this bug.