python-jenkins-job-builder fails to build with Python 3.13.0a1. ______________ test_yaml_snippet[parameter-override-ordering-001] ______________ check_job = <function check_job.<locals>.check at 0x7f162227e980> def test_yaml_snippet(check_job): # Some tests using config with 'include_path' expect JJB root to be current directory. os.chdir(Path(__file__).parent / "../..") > check_job() tests/yamlparser/test_jobs.py:41: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def check(): roots = Roots(jjb_config) load_files(jjb_config, roots, [scenario.in_path]) registry.set_macros(roots.macros) job_data_list = roots.generate_jobs() registry.amend_job_dicts(job_data_list) generator = XmlJobGenerator(registry) job_xml_list = generator.generateXML(job_data_list) job_xml_list.sort(key=AlphanumSort) pretty_xml = ( "\n".join(job.output().decode() for job in job_xml_list) .strip() .replace("\n\n", "\n") ) if expected_output is None: return stripped_expected_output = ( expected_output.strip().replace("<BLANKLINE>", "").replace("\n\n", "\n") ) > assert stripped_expected_output == pretty_xml E assert '<?xml versio...>\n</project>' == '<?xml versio...>\n</project>' E Skipping 579 identical leading characters in diff, use -v to show E n E # E - print("Doing something cool with python") E ? ^ ^ E + print("Doing something cool with python") E ? ^^^^^^ ^^^^^^... E E ...Full output truncated (145 lines hidden), use '-vv' to show tests/conftest.py:189: AssertionError ------------------------------ Captured log setup ------------------------------ WARNING jenkins_jobs.config:config.py:121 Config file, /builddir/build/BUILD/jenkins-job-builder-5.0.4/tests/yamlparser/job_fixtures/trigger_parameterized_builds/parameter-override-ordering-001.conf, not found. Using default config values. INFO jenkins_jobs.config:config.py:339 Will use anonymous access to Jenkins if needed. ------------------------------ Captured log call ------------------------------- INFO root:registry.py:165 Caching type builders of builders = jenkins_jobs.modules.builders:Builders INFO root:registry.py:165 Caching type publishers of publishers = jenkins_jobs.modules.publishers:Publishers =========================== short test summary info ============================ FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds008] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds006] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds003] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds001] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds-configfactory-multi] FAILED tests/builders/test_builders.py::test_yaml_snippet[multibuild] - asser... FAILED tests/builders/test_builders.py::test_yaml_snippet[jms-messaging-full] FAILED tests/builders/test_builders.py::test_yaml_snippet[docker-builder002] FAILED tests/builders/test_builders.py::test_yaml_snippet[docker-builder001] FAILED tests/builders/test_builders.py::test_yaml_snippet[complete-1.x] - ass... FAILED tests/builders/test_builders.py::test_yaml_snippet[complete-2.x] - ass... FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-remote001] FAILED tests/jsonparser/test_jsonparser.py::test_yaml_snippet[complete001] - ... FAILED tests/loader/test_loader.py::test_include_job[include-raw001-template] FAILED tests/loader/test_loader.py::test_include_job[include-raw001-job] - as... FAILED tests/loader/test_loader.py::test_include_job[include-raw-multi001] - ... FAILED tests/loader/test_loader.py::test_include_job[include-raw-escaped001-template] FAILED tests/loader/test_loader.py::test_include_job[include-raw-escaped001-job] FAILED tests/loader/test_loader.py::test_include_job[include-raw-escaped-multi001] FAILED tests/loader/test_loader.py::test_include_job[deprecated-include-raw-escaped001] FAILED tests/macros/test_macros.py::test_yaml_snippet[multi-line-strings001] FAILED tests/parameters/test_parameters.py::test_yaml_snippet[extended-choice-param-full] FAILED tests/properties/test_properties.py::test_yaml_snippet[batch-task] - a... FAILED tests/publishers/test_publishers.py::test_yaml_snippet[trigger_parameterized_builds001] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript004] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript003] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript002] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript-full] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[post-tasks002] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[post-tasks001] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[jms-messaging-full] FAILED tests/scm/test_scm.py::test_yaml_snippet[repo001] - assert '<?xml vers... FAILED tests/triggers/test_triggers.py::test_yaml_snippet[jms-messaging002] FAILED tests/triggers/test_triggers.py::test_yaml_snippet[jms-messaging001] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[version-number001] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[pre-scm-buildstep002] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[pre-scm-buildstep001] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[artifactory005] - a... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[variable_escaping] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[variable_default_values] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[unicode001] - assert ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_ids] - asser... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_honor_defaults_project_type] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_default_variables] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[string_join] - assert... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[regression-2006254] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template003] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_triggers] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template006] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template003] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_concurrent] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_flow_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_flow_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project-matrix002] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project-matrix001] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs002] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs001] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs-multi002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs-multi001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jobgroups_multi_use] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[job_instantiation_with_null_params] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[job_honor_defaults_project_type] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jinja-yaml04-deepcopy] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jinja-yaml03] - asser... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include001] - assert ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-rawunicode001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-raw001] - ass... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-raw-escape001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-param] - asse... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[dont_overwrite_desc] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[default_template_variables_id_multi] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[complete001] - assert... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[allow_empty_variables_include] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[allow_empty_variables] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[add_managed_to_desc] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[parameter-override-ordering-001] = 80 failed, 1672 passed, 2 skipped, 8 xfailed, 4 xpassed, 19 warnings in 34.40s = https://docs.python.org/3.13/whatsnew/3.13.html For the build logs, see: https://copr-be.cloud.fedoraproject.org/results/@python/python3.13/fedora-rawhide-x86_64/06578891-python-jenkins-job-builder/ For all our attempts to build python-jenkins-job-builder with Python 3.13, see: https://copr.fedorainfracloud.org/coprs/g/python/python3.13/package/python-jenkins-job-builder/ Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.13: https://copr.fedorainfracloud.org/coprs/g/python/python3.13/ Let us know here if you have any questions. Python 3.13 is planned to be included in Fedora 41. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.13. A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon. We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
From the error message it looks like the escaping behaviour has changed in Python 3.13: double quotes in the output are now HTML-escaped. I'll have to dig into this; any pointers are appreciated.
This bug appears to have been reported against 'rawhide' during the Fedora Linux 40 development cycle. Changing version to 40.
*** Bug 2291747 has been marked as a duplicate of this bug. ***
This package has failed to build with Python 3.13 so far. Fedora 41 is already in the Beta Freeze phase. What's your plan going forward? Will the package be fixed in time for the Final Freeze of Fedora 41 (planned on Oct 15th 2024)? If you don't plan or can't fix the package before Fedora 41 goes into the Final Freeze phase, please retire the package. You can unretire it anytime during the Fedora 41 lifecycle - doing this now will simply ensure that we don't release Fedora with a broken package. Thank you!
I won't be able to investigate this further until https://bugzilla.redhat.com/show_bug.cgi?id=2301155 (python-jenkins: FTBFS in Fedora rawhide/f41) is fixed.
=========================== short test summary info ============================ FAILED tests/builders/test_builders.py::test_yaml_snippet[docker-builder001] FAILED tests/builders/test_builders.py::test_yaml_snippet[docker-builder002] FAILED tests/builders/test_builders.py::test_yaml_snippet[jms-messaging-full] FAILED tests/builders/test_builders.py::test_yaml_snippet[multibuild] - asser... FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds-configfactory-multi] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds001] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds003] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds006] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-builds008] FAILED tests/builders/test_builders.py::test_yaml_snippet[trigger-remote001] FAILED tests/builders/test_builders.py::test_yaml_snippet[complete-1.x] - ass... FAILED tests/builders/test_builders.py::test_yaml_snippet[complete-2.x] - ass... FAILED tests/jsonparser/test_jsonparser.py::test_yaml_snippet[complete001] - ... FAILED tests/loader/test_loader.py::test_include_job[include-raw-expand-template] FAILED tests/loader/test_loader.py::test_include_job[include-raw-verbatim-job] FAILED tests/loader/test_loader.py::test_include_job[include-raw-verbatim-multi-job] FAILED tests/loader/test_loader.py::test_include_job[include-raw-verbatim-multi-template] FAILED tests/loader/test_loader.py::test_include_job[include-raw-verbatim-template] FAILED tests/macros/test_macros.py::test_yaml_snippet[multi-line-strings001] FAILED tests/parameters/test_parameters.py::test_yaml_snippet[extended-choice-param-full] FAILED tests/properties/test_properties.py::test_yaml_snippet[batch-task] - a... FAILED tests/publishers/test_publishers.py::test_yaml_snippet[jms-messaging-full] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[post-tasks001] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[post-tasks002] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript-full] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript002] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript003] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[postbuildscript004] FAILED tests/publishers/test_publishers.py::test_yaml_snippet[trigger_parameterized_builds001] FAILED tests/scm/test_scm.py::test_yaml_snippet[repo001] - assert '<?xml vers... FAILED tests/triggers/test_triggers.py::test_yaml_snippet[jms-messaging001] FAILED tests/triggers/test_triggers.py::test_yaml_snippet[jms-messaging002] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[artifactory005] - a... FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[pre-scm-buildstep-no-buildsteps] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[pre-scm-buildstep001] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[pre-scm-buildstep002] FAILED tests/wrappers/test_wrappers.py::test_yaml_snippet[version-number001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[add_managed_to_desc] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[allow_empty_variables] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[allow_empty_variables_include] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[complete001] - assert... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[default_template_variables_id_multi] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[deprecated-include-raw-escaped001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[deprecated-include-raw-escaped002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[deprecated-include-raw001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[deprecated-include-raw002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[dont_overwrite_desc] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-param] - asse... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-raw-expand-unicode] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-raw-expand] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include-raw-verbatim] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[include001] - assert ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jinja-yaml03] - asser... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jinja-yaml04-deepcopy] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[job_honor_defaults_project_type] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[job_instantiation_with_null_params] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[jobgroups_multi_use] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs-multi001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs-multi002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs001] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[lazy-load-jobs002] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[macro-uses-custom-defaults] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[macro-uses-global-defaults] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project-matrix001] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project-matrix002] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_flow_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_flow_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_concurrent] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template003] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_template006] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_pipeline_triggers] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template001] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template002] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[project_workflow_template003] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[regression-2006254] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[string_join] - assert... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_default_variables] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_honor_defaults_project_type] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[template_ids] - asser... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[unicode001] - assert ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[variable_default_values] FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[variable_escaping] - ... FAILED tests/yamlparser/test_jobs.py::test_yaml_snippet[parameter-override-ordering-001] = 85 failed, 1781 passed, 2 skipped, 8 xfailed, 4 xpassed, 43 warnings in 72.02s (0:01:12) =
The strange HTML escaping of double quotes is still there, and it causes the tests to fail. I'll try to find out what the root cause is.
Oh, it's the other way round! The expected output (e.g. in `tests/builders/fixtures/docker-builder001.xml`) does contain escaped quotes but the actual output contains plain double quotes.
The root cause must be somewhere inside `xml.dom.minidom`. Python 3.12.: ``` $ python3 -c 'import xml.dom.minidom; print(xml.dom.minidom.parseString("<foo>\"quoted\" stuff</foo>").toprettyxml())' <?xml version="1.0" ?> <foo>"quoted" stuff</foo> ``` Python 3.13: ``` $ python3 -c 'import xml.dom.minidom; print(xml.dom.minidom.parseString("<foo>\"quoted\" stuff</foo>").toprettyxml())' <?xml version="1.0" ?> <foo>"quoted" stuff</foo> ```
Found the culprit: https://github.com/python/cpython/commit/154477be722ae5c4e18d22d0860e284006b09c4f > Also double quotes (") are now only quoted in attributes. That's sane behaviour; there is no need to escape double quotes except inside attributes. But it breaks the existing tests because the reference output expects the old behaviour. I'll go figure out a way to make the tests work on both Python 3.13 and older.
I have a fix and will push it once the python-jenkins rebuild (https://bodhi.fedoraproject.org/updates/FEDORA-2024-454587d1c0) has hit the mirrors.
Upstream patch: https://review.opendev.org/c/jjb/jenkins-job-builder/+/930634
(In reply to Christoph Erhardt from comment #11) > I have a fix and will push it once the python-jenkins rebuild > (https://bodhi.fedoraproject.org/updates/FEDORA-2024-454587d1c0) has hit the > mirrors. No need to wait for the mirrors the build is already available in Koji. You can use --enablerepo=local in mock, in case you want to test your fix. If you push the fix to distgit and build it in rawhide, I can build it in a F41 side tag and add it to https://bodhi.fedoraproject.org/updates/FEDORA-2024-8f868c9ceb
Thanks for the pointer! `fedpkg mockbuild --enablerepo=local` completed successfully; changes have been pushed to Git for the `rawhide` and `f41` branches.
FEDORA-2024-8f868c9ceb (python-jenkins-1.8.2-4.fc41 and python-jenkins-job-builder-6.4.1-2.fc41) has been submitted as an update to Fedora 41. https://bodhi.fedoraproject.org/updates/FEDORA-2024-8f868c9ceb
FEDORA-2024-8f868c9ceb has been pushed to the Fedora 41 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-8f868c9ceb` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-8f868c9ceb See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Hello, Please note that this comment was generated automatically by https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py If you feel that this output has mistakes, please open an issue at https://pagure.io/releng/ All subpackages of a package against which this bug was filled are now installable or removed from Fedora 42. Thanks for taking care of it!
FEDORA-2024-8f868c9ceb (python-jenkins-1.8.2-4.fc41 and python-jenkins-job-builder-6.4.1-2.fc41) has been pushed to the Fedora 41 stable repository. If problem still persists, please make note of it in this bug report.