Created attachment 1562731 [details] Full log from Copr python-jenkins-job-builder-2.1.0-3 fails to build with Python 3.8 due to syntax warnings for invalid escape sequences during the test run. Snippet: jenkins_jobs/modules/builders.py:2876 /builddir/build/BUILD/jenkins-job-builder-2.1.0/jenkins_jobs/modules/builders.py:2876: SyntaxWarning: invalid escape sequence \* """yaml: dsl This is due to https://bugs.python.org/issue32912 Full log attached
Please respond.
I'll try to fix this package over the weekend.
https://src.fedoraproject.org/rpms/python-jenkins-job-builder/pull-request/1
Thanks Igor! I can locally build your changes with mock on my laptop (along with the additional pytest BR line), so I've merged that now.
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to '31'.
The coordinated rebuild of Python 3.8 has started in the `f32-python` side tag. If you figure out how to rebuild this package, please don't rebuild it in regular rawhide, but use the side tag instead: on branch master: $ fedpkg build --target=f32-python To wait for a build to show up in the side tag, do: $ koji wait-repo f32-python --build=<nvr> Where <nvr> is name-version-release of the source package, e.g. python-foo-1.1-2.fc32. An updated mock config is posted at: http://copr.fedorainfracloud.org/coprs/g/python/python3.8/ Note that it will take a while before the essential packages are rebuilt, so don't expect all your dependencies to be available right away. Thanks. Let us know if you need up to date info, or if you have any questions. PS this message is mass posted to all the bugs that block the PYTHON38 bug. If this is also a Fedora 31 FTBFS bug and you manage to fix it, you can do a f31 build as usual: on branch f31: $ fedpkg build
The f32-python side tag has been merged. In order to rebuild the package, do it in regular rawhide, but please wait until python3-3.8 is tagged: $ koji wait-repo f32-build --build python3-3.8.0~b3-3.fc32 If your built already started in f32-python, after it is finished, please tag it to rawhide with: $ koji tag-build f32-pending <nvr> For example: $ koji tag-build f32-pending libreoffice-6.3.0.4-3.fc32 Thanks! (This comment is mass posted to all bugzillas blocking the PYTHON38 tracking bug.)
(Python 3.8 has landed in the rawhide buildroot.)
==================================== ERRORS ==================================== ________ ERROR at teardown of TestCaseModuleWrappers.test_yaml_snippet _________ NOTE: Incompatible Exception Representation, displaying natively: testtools.testresult.real._StringException: pythonlogging:'': {{{ Config file, /dev/null, not found. Using default config values. Config: <configparser.ConfigParser object at 0xf5e22130> Skipping plugin info retrieval Will use anonymous access to Jenkins if needed. }}} Traceback (most recent call last): File "/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/base.py", line 239, in test_yaml_snippet self.assertThat( File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: Differences (ndiff with -expected +actual): <?xml version="1.0" encoding="utf-8"?> <project> <buildWrappers> <ruby-proxy-object> - <ruby-object pluginid="rbenv" ruby-class="Jenkins::Tasks::BuildWrapperProxy"> ? ----------------- + <ruby-object ruby-class="Jenkins::Tasks::BuildWrapperProxy" pluginid="rbenv"> ? +++++++++++++++++ <pluginid pluginid="rbenv" ruby-class="String">rbenv</pluginid> - <object pluginid="rbenv" ruby-class="RbenvWrapper"> + <object ruby-class="RbenvWrapper" pluginid="rbenv"> - <gem__list pluginid="rbenv" ruby-class="String">bundler,rake</gem__list> ? ----------------- + <gem__list ruby-class="String" pluginid="rbenv">bundler,rake</gem__list> ? +++++++++++++++++ - <rbenv__root pluginid="rbenv" ruby-class="String">$HOME/.rbenv</rbenv__root> ? ----------------- + <rbenv__root ruby-class="String" pluginid="rbenv">$HOME/.rbenv</rbenv__root> ? +++++++++++++++++ - <rbenv__repository pluginid="rbenv" ruby-class="String">https://github.com/rbenv/rbenv</rbenv__repository> ? ----------------- + <rbenv__repository ruby-class="String" pluginid="rbenv">https://github.com/rbenv/rbenv</rbenv__repository> ? +++++++++++++++++ - <rbenv__revision pluginid="rbenv" ruby-class="String">master</rbenv__revision> ? ----------------- + <rbenv__revision ruby-class="String" pluginid="rbenv">master</rbenv__revision> ? +++++++++++++++++ - <ruby__build__repository pluginid="rbenv" ruby-class="String">https://github.com/rbenv/ruby-build</ruby__build__repository> ? ----------------- + <ruby__build__repository ruby-class="String" pluginid="rbenv">https://github.com/rbenv/ruby-build</ruby__build__repository> ? +++++++++++++++++ - <ruby__build__revision pluginid="rbenv" ruby-class="String">master</ruby__build__revision> ? ----------------- + <ruby__build__revision ruby-class="String" pluginid="rbenv">master</ruby__build__revision> ? +++++++++++++++++ - <version pluginid="rbenv" ruby-class="String">1.9.3-p484</version> ? ----------------- + <version ruby-class="String" pluginid="rbenv">1.9.3-p484</version> ? +++++++++++++++++ - <ignore__local__version pluginid="rbenv" ruby-class="FalseClass"/> ? ----------------- + <ignore__local__version ruby-class="FalseClass" pluginid="rbenv"/> ? +++++++++++++++++ </object> </ruby-object> </ruby-proxy-object> </buildWrappers> </project> _______ ERROR at teardown of TestCaseModuleYamlInclude.test_yaml_snippet _______ NOTE: Incompatible Exception Representation, displaying natively: testtools.testresult.real._StringException: pythonlogging:'': {{{ Config file, /dev/null, not found. Using default config values. Config: <configparser.ConfigParser object at 0xf5fd4388> Skipping plugin info retrieval Will use anonymous access to Jenkins if needed. Adding '.' to search path for include tags Expanding project 'project-name' }}} Traceback (most recent call last): File "/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/base.py", line 300, in test_yaml_snippet self.assertThat( File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: Differences (ndiff with -expected +actual): <?xml version="1.0" encoding="utf-8"?> <flow-definition plugin="workflow-job"> - <definition class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition" plugin="workflow-cps"> ? ---------------------- + <definition plugin="workflow-cps" class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition"> ? ++++++++++++++++++++++ <script>stage('Build another job') { build(job: "hello") } </script> <sandbox>false</sandbox> </definition> <actions/> <description><!-- Managed by Jenkins Job Builder --></description> <keepDependencies>false</keepDependencies> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <concurrentBuild>false</concurrentBuild> <canRoam>true</canRoam> <properties/> <scm class="hudson.scm.NullSCM"/> <publishers/> <buildWrappers/> </flow-definition> =================================== FAILURES =================================== __________________ TestCaseModulePublishers.test_yaml_snippet __________________ NOTE: Incompatible Exception Representation, displaying natively: testtools.testresult.real._StringException: plugins-info: {{{[OrderedDict([('longName', 'Flexible Publish Plugin'), ('shortName', 'flexible-publish'), ('version', '0.1')])]}}} plugins-info-filename: {{{/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/publishers/fixtures/conditional-publisher002.plugins_info.yaml}}} pythonlogging:'': {{{ Config file, /dev/null, not found. Using default config values. Config: <configparser.ConfigParser object at 0xf61b3718> Skipping plugin info retrieval Will use anonymous access to Jenkins if needed. }}} Traceback (most recent call last): File "/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/base.py", line 239, in test_yaml_snippet self.assertThat( File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: Differences (ndiff with -expected +actual): <?xml version="1.0" encoding="utf-8"?> <project> <publishers> <org.jenkins__ci.plugins.flexible__publish.FlexiblePublisher> <publishers> <org.jenkins__ci.plugins.flexible__publish.ConditionalPublisher> <condition class="org.jenkins_ci.plugins.run_condition.contributed.ShellCondition"> <command>ls file*</command> </condition> <runner class="org.jenkins_ci.plugins.run_condition.BuildStepRunner$DontRun"/> - <publisher class="com.michelin.cio.hudson.plugins.copytoslave.CopyToMasterNotifier" plugin="copy-to-slave"> ? ----------------------- + <publisher plugin="copy-to-slave" class="com.michelin.cio.hudson.plugins.copytoslave.CopyToMasterNotifier"> ? +++++++++++++++++++++++ <includes>file1,file2*.txt</includes> <excludes>file2bad.txt</excludes> <runAfterResultFinalised>true</runAfterResultFinalised> <destinationFolder/> </publisher> </org.jenkins__ci.plugins.flexible__publish.ConditionalPublisher> <org.jenkins__ci.plugins.flexible__publish.ConditionalPublisher> <condition class="org.jenkins_ci.plugins.run_condition.core.AlwaysRun"/> <runner class="org.jenkins_ci.plugins.run_condition.BuildStepRunner$RunUnstable"/> <publisher class="hudson.tasks.test.AggregatedTestResultPublisher"> <includeFailedBuilds>true</includeFailedBuilds> </publisher> </org.jenkins__ci.plugins.flexible__publish.ConditionalPublisher> </publishers> </org.jenkins__ci.plugins.flexible__publish.FlexiblePublisher> </publishers> </project> ___________________ TestCaseModuleWrappers.test_yaml_snippet ___________________ NOTE: Incompatible Exception Representation, displaying natively: testtools.testresult.real._StringException: pythonlogging:'': {{{ Config file, /dev/null, not found. Using default config values. Config: <configparser.ConfigParser object at 0xf5e25a18> Skipping plugin info retrieval Will use anonymous access to Jenkins if needed. }}} Traceback (most recent call last): File "/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/base.py", line 239, in test_yaml_snippet self.assertThat( File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: Differences (ndiff with -expected +actual): <?xml version="1.0" encoding="utf-8"?> <project> <buildWrappers> <ruby-proxy-object> - <ruby-object pluginid="rbenv" ruby-class="Jenkins::Tasks::BuildWrapperProxy"> ? ----------------- + <ruby-object ruby-class="Jenkins::Tasks::BuildWrapperProxy" pluginid="rbenv"> ? +++++++++++++++++ <pluginid pluginid="rbenv" ruby-class="String">rbenv</pluginid> - <object pluginid="rbenv" ruby-class="RbenvWrapper"> + <object ruby-class="RbenvWrapper" pluginid="rbenv"> - <gem__list pluginid="rbenv" ruby-class="String">bundler,rake</gem__list> ? ----------------- + <gem__list ruby-class="String" pluginid="rbenv">bundler,rake</gem__list> ? +++++++++++++++++ - <rbenv__root pluginid="rbenv" ruby-class="String">$HOME/.rbenv</rbenv__root> ? ----------------- + <rbenv__root ruby-class="String" pluginid="rbenv">$HOME/.rbenv</rbenv__root> ? +++++++++++++++++ - <rbenv__repository pluginid="rbenv" ruby-class="String">https://github.com/rbenv/rbenv</rbenv__repository> ? ----------------- + <rbenv__repository ruby-class="String" pluginid="rbenv">https://github.com/rbenv/rbenv</rbenv__repository> ? +++++++++++++++++ - <rbenv__revision pluginid="rbenv" ruby-class="String">master</rbenv__revision> ? ----------------- + <rbenv__revision ruby-class="String" pluginid="rbenv">master</rbenv__revision> ? +++++++++++++++++ - <ruby__build__repository pluginid="rbenv" ruby-class="String">https://github.com/rbenv/ruby-build</ruby__build__repository> ? ----------------- + <ruby__build__repository ruby-class="String" pluginid="rbenv">https://github.com/rbenv/ruby-build</ruby__build__repository> ? +++++++++++++++++ - <ruby__build__revision pluginid="rbenv" ruby-class="String">master</ruby__build__revision> ? ----------------- + <ruby__build__revision ruby-class="String" pluginid="rbenv">master</ruby__build__revision> ? +++++++++++++++++ - <version pluginid="rbenv" ruby-class="String">2.0.0-p353</version> ? ----------------- + <version ruby-class="String" pluginid="rbenv">2.0.0-p353</version> ? +++++++++++++++++ - <ignore__local__version pluginid="rbenv" ruby-class="TrueClass"/> ? ----------------- + <ignore__local__version ruby-class="TrueClass" pluginid="rbenv"/> ? +++++++++++++++++ </object> </ruby-object> </ruby-proxy-object> </buildWrappers> </project> _________________ TestCaseModuleYamlInclude.test_yaml_snippet __________________ NOTE: Incompatible Exception Representation, displaying natively: testtools.testresult.real._StringException: pythonlogging:'': {{{ Config file, /dev/null, not found. Using default config values. Config: <configparser.ConfigParser object at 0xf5f901d8> Skipping plugin info retrieval Will use anonymous access to Jenkins if needed. Adding '.' to search path for include tags Expanding project 'project-name' }}} Traceback (most recent call last): File "/builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/base.py", line 300, in test_yaml_snippet self.assertThat( File "/usr/lib/python3.8/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: Differences (ndiff with -expected +actual): <?xml version="1.0" encoding="utf-8"?> <flow-definition plugin="workflow-job"> - <definition class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition" plugin="workflow-cps"> ? ---------------------- + <definition plugin="workflow-cps" class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition"> ? ++++++++++++++++++++++ <script>build job: "job1" parallel [ 2a: build job: "job2a", 2b: node "dummynode" { sh "echo hello" } ] </script> <sandbox>true</sandbox> </definition> <actions/> <description><!-- Managed by Jenkins Job Builder --></description> <keepDependencies>false</keepDependencies> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <concurrentBuild>false</concurrentBuild> <canRoam>true</canRoam> <properties/> <scm class="hudson.scm.NullSCM"/> <publishers/> <buildWrappers/> </flow-definition> =============================== warnings summary =============================== /usr/lib/python3.8/site-packages/testtools/matchers/_warnings.py:24 /usr/lib/python3.8/site-packages/testtools/matchers/_warnings.py:24: DeprecationWarning: invalid escape sequence \s """ /usr/lib/python3.8/site-packages/testtools/content.py:99 /usr/lib/python3.8/site-packages/testtools/content.py:99: DeprecationWarning: invalid escape sequence \* """Iterate over the text of the serialised content. /usr/lib/python3.8/site-packages/unittest2/compatibility.py:143 /usr/lib/python3.8/site-packages/unittest2/compatibility.py:143: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working class ChainMap(collections.MutableMapping): /usr/lib/python3.8/site-packages/jinja2/runtime.py:318 /usr/lib/python3.8/site-packages/jinja2/runtime.py:318: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import Mapping tests/cmd/test_config.py::TestConfigs::test_non_existing_config_dir tests/cmd/test_config.py::TestConfigs::test_non_existing_config_file tests/cmd/subcommands/test_test.py::TestTests::test_console_output tests/cmd/subcommands/test_test.py::TestTests::test_non_existing_job tests/cmd/subcommands/test_test.py::TestTests::test_output_dir tests/cmd/subcommands/test_test.py::TestTests::test_plugins_info_stub_option tests/cmd/subcommands/test_test.py::TestTests::test_stream_input_output_ascii_encoding tests/cmd/subcommands/test_test.py::TestTests::test_stream_input_output_no_encoding_exceed_recursion tests/cmd/subcommands/test_test.py::TestTests::test_stream_input_output_utf8_encoding tests/cmd/subcommands/test_test.py::TestTests::test_stream_output_ascii_encoding_invalid_char tests/cmd/subcommands/test_test.py::TestTests::test_valid_job tests/cmd/subcommands/test_test.py::TestJenkinsGetPluginInfoError::test_console_output_jenkins_connection_failure_warning tests/cmd/subcommands/test_test.py::TestTestsMultiPath::test_multi_path tests/cmd/subcommands/test_test.py::TestTestsMultiPath::test_recursive_multi_path_command_line tests/cmd/subcommands/test_test.py::TestTestsMultiPath::test_recursive_multi_path_config_file tests/cmd/subcommands/test_test.py::TestTestsMultiPath::test_recursive_multi_path_with_excludes /builddir/build/BUILD/jenkins-job-builder-3.0.1/jenkins_jobs/cli/subcommand/test.py:54: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead logger.warn('(Deprecated) The default output behavior of' tests/cmd/subcommands/test_test.py::TestTests::test_bogus_plugins_info_stub_option tests/cmd/subcommands/test_test.py::TestTests::test_plugins_info_stub_option /builddir/build/BUILD/jenkins-job-builder-3.0.1/jenkins_jobs/cli/entry.py:104: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. plugins_info = yaml.load(yaml_file) tests/cmd/subcommands/test_test.py::TestTests::test_plugins_info_stub_option /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/cmd/subcommands/test_test.py:198: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. plugins_info_list = yaml.load(yaml_file) tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:35: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. default_data = yaml.load("string: hello") tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:48: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. user_input_data = yaml.load("user-input-string: hello") tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:62: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. required_data = yaml.load("string: hello") tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:74: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. user_input_data = yaml.load("user-input-string: bye") tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. user_input_data = yaml.load("user-input-string: later") tests/modules/test_helpers.py::TestCaseTestHelpers::test_convert_mapping_to_xml /builddir/build/BUILD/jenkins-job-builder-3.0.1/tests/modules/test_helpers.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. user_input_data = yaml.load("user-input-string: world") -- Docs: https://docs.pytest.org/en/latest/warnings.html ==== 3 failed, 87 passed, 2 skipped, 29 warnings, 2 error in 24.66 seconds =====
What is the plan here? Do we try to keep the package or retire it?
Please, respond.
Consider this the reminder described as step 4 in https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/
https://pagure.io/releng/issue/9223
When I build the latest upstream version (3.2.0) on f32, I see the same test failures. When I build 3.2.0 for f29, there are no test failures. This tells me that some change in the underlying libraries causes Jenkins Job Builder to write the XML element attributes in a different order than the test suite expects.
Ok, the problem was a slight XML ordering change in py38. I've cherry-picked https://review.opendev.org/705269 onto upstream's latest 3.2.0 release, and that clears one test failure. The other test failure (with plugin="copy-to-slave") still occurs when I run upstream with `tox -e py38`, so more investigation is needed there.
I've fixed final test failure with with plugin="copy-to-slave" and submitted that change to https://review.opendev.org/706317 . I've un-orphaned this package now. Commit that resolves this FTBFS: https://src.fedoraproject.org/rpms/python-jenkins-job-builder/c/78b70d24cf65a4c7a100d3e56358ae22d3a6eaf6 Rawhide (f32) build: https://koji.fedoraproject.org/koji/buildinfo?buildID=1457501
Thanks!
Thanks for being patient Miro.