Bug 1889726 - unbound fails to build with Python 3.10: undefined reference to `PyParser_SimpleParseFile'
Summary: unbound fails to build with Python 3.10: undefined reference to `PyParser_Sim...
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: unbound
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Paul Wouters
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2020-10-20 13:13 UTC by Tomáš Hrnčiar
Modified: 2020-11-12 15:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github NLnetLabs unbound issues 344 None open PyParser_SimpleParseFile has been removed in Python 3.10 2020-11-20 10:51:19 UTC

Description Tomáš Hrnčiar 2020-10-20 13:13:41 UTC
unbound fails to build with Python 3.10.0a1.

This report is automated and not very verbose, but we'll try to get back here with details.

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01713325-unbound/

For all our attempts to build unbound with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/unbound/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-10-20 13:20:12 UTC
The problem is:

/builddir/build/BUILD/unbound-1.10.1/unbound-1.10.1/pythonmod/pythonmod.c:371: undefined reference to `PyParser_SimpleParseFile'

I don't see this explicitly mentioned in https://docs.python.org/3.10/whatsnew/3.10.html but I see it mentioned in in https://docs.python.org/3.9/whatsnew/3.9.html#deprecated

Comment 2 Miro Hrončok 2020-11-10 12:41:04 UTC
Marking as high severity, because unbound is needed by gnutls.

Comment 3 Petr Menšík 2020-11-10 18:21:00 UTC
Hi Tomáš,

I lack python expertise to see obvious fix. It seems offending line [1] is executed only to obtain Exception to print into log.

There is already PyRun_SimpleFile, which runs the script. It would print execption to stderr.
Second time, it runs the same code to include it in log. It should prefix the second message with:

./unbound-checkconf unbound.conf 
  File "./pythonmod/examples/log.py", line 63
    print "-"*100
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("-"*100)?
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: can't parse Python script ./pythonmod/examples/log.py
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: python error:   File "./pythonmod/examples/log.py", line 63
    print "-"*100
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("-"*100)?

This part is repeated just to be catched also in logs under unbound. Could we instead modify PyRun_SimpleFile call to produce just one exception,
then print it to stderr once and once into the log? We could then remove deprecated function just for that purpose.

[1605031806] unbound-checkconf[2797510:0] error: pythonmod: can't parse Python script ./pythonmod/examples/log.py
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: python error:   File "./pythonmod/examples/log.py", line 63

But it seems PyRun_SimpleFile does not throw Exception. Can you recommend variant or flags, which would make it raise an Exception, which log_py_err() would then to log file?
After commenting out PyParser_SimpleParseFile it reports None, so it did not already raise an exception.

[1605032274] unbound-checkconf[2798279:0] error: pythonmod: python error: NoneType: None

I could not find any information about exceptions in documentation [2]. Is there better place to look into?

Reproducing the error is quite simple. Enter unbound build directory and unbound-1.* in it.

cd unbound-1.*/unbound-1.*

cat <<EOF > unbound.conf
server:
	module-config: "python iterator"

python:
	python-script: ./pythonmod/examples/log.py
EOF

./unbound-checkconf unbound.conf

Because examples are written for python2, they emit error.

1. https://github.com/NLnetLabs/unbound/blob/master/pythonmod/pythonmod.c#L371
2. https://docs.python.org/3.9/c-api/veryhigh.html#c.PyParser_SimpleParseFile

Comment 4 Miro Hrončok 2020-11-10 19:30:40 UTC
Victor is better suited to answer your question.

Comment 5 Victor Stinner 2020-11-12 15:19:00 UTC
> I don't see this explicitly mentioned in https://docs.python.org/3.10/whatsnew/3.10.html but I see it mentioned in in https://docs.python.org/3.9/whatsnew/3.9.html#deprecated

I asked to explicitly document removed functions:
https://bugs.python.org/issue40939#msg380826

unbound used the removed function in pythonmod/pythonmod.c. Extract of unbound-1.12.0.tar.gz:

   if (PyRun_SimpleFile(script_py, pe->fname) < 0) {
      log_err("pythonmod: can't parse Python script %s", pe->fname);
      /* print the error to logs too, run it again */
      fseek(script_py, 0, SEEK_SET);
      /* we don't run the file, like this, because then side-effects
       *    s = PyRun_File(script_py, pe->fname, Py_file_input, 
       *        PyModule_GetDict(PyImport_AddModule("__main__")), pe->dict);
       * could happen (again). Instead we parse the file again to get
       * the error string in the logs, for when the daemon has stderr
       * removed.  SimpleFile run already printed to stderr, for then
       * this is called from unbound-checkconf or unbound -dd the user
       * has a nice formatted error.
      */
      /* ignore the NULL return of _node, it is NULL due to the parse failure
       * that we are expecting */
      (void)PyParser_SimpleParseFile(script_py, pe->fname, Py_file_input);
      log_py_err();
      PyGILState_Release(gil);
      fclose(script_py);
      return 0;
   }


> But it seems PyRun_SimpleFile does not throw Exception. Can you recommend variant or flags, which would make it raise an Exception, which log_py_err() would then to log file?

I'm not aware of any replacement for PyParser_SimpleParseFile() :-( I asked on the usptream issue:
https://bugs.python.org/issue40939#msg380828


Note You need to log in before you can comment on or make changes to this bug.