Description of problem: Lots of tests fail on Fedora 25: $ prove -lrcf t # Testing RPM::Grill 0.01 t/00.load.t ............................................ ok Possible precedence issue with control flow operator at /code/t/../bin/rpmgrill-fetch-build line 447. Fetch failed! HTTP response: 404 Not Found [404 File `/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' does not exist] at /code/t/../bin/rpmgrill-fetch-build line 413. Could not copy '/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' to 't-FetchBuild.odnY8b/testbuild-0.1-1.fc20.i386.rpm': No such file or directory at /code/t/../bin/rpmgrill-fetch-build line 413. Could not copy '/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' to 't-FetchBuild.odnY8b/testbuild-0.1-1.fc20.i386.rpm': No such file or directory at /code/t/../bin/rpmgrill-fetch-build line 413. Use of uninitialized value in subroutine entry at /code/t/../bin/rpmgrill-fetch-build line 642. t/10fetch-build.t ...................................... ok t/RPM/Grill/10arch-order.t ............................. ok t/RPM/Grill/12major_release_ge.t ....................... ok t/RPM/Grill/20multilib.t ............................... ok t/RPM/Grill/30aggregate_gripes.t ....................... ok t/RPM/Grill/40codes.t .................................. ok t/RPM/Grill/50as_xml.t ................................. ok t/RPM/Grill/60blacklisting.t ........................... ok t/RPM/Grill/Plugin/10order.t ........................... ok t/RPM/Grill/Plugin/20docs.t ............................ ok t/RPM/Grill/Plugin/90real-world.t ...................... ok $VAR1 = 'update.c: In function `update\':'; # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 255 just after 3. t/RPM/Grill/Plugin/BuildLog/01basic.t .................. Dubious, test returned 255 (wstat 65280, 0xff00) All 3 subtests passed $VAR1 = '[Desktop Entry]'; # Looks like your test exited with 25 just after 4. t/RPM/Grill/Plugin/DesktopLint/10basic.t ............... Dubious, test returned 25 (wstat 6400, 0x1900) Failed 17/21 subtests t/RPM/Grill/Plugin/ElfChecks/10basic.t ................. ok t/RPM/Grill/Plugin/ElfChecks/20relro.t ................. ok # Failed test 'binary-needs-manpage : ERROR: $VAR1 = '/usr/sbin/foo'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'manpage-in-another-arch : ERROR: $VAR1 = '/usr/sbin/foo'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'manpage-in-arch-but-configfile-is-in-noarch : ERROR: $VAR1 = '/etc/gitweb.conf'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'manpage-in-only-some-arches : ERROR: $VAR1 = '/etc/gitweb.conf'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'bad-gzip-manpage : ERROR: $VAR1 = ' # gzip: /usr/share/man/man1/foo.1.gz: not in gzip format # (\'file\' classifies this file as: ASCII text)'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'bad-manpage-content : ERROR: $VAR1 = 'a /bin/nosh script, ASCII text executable'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Failed test 'config-file-missing-manpage : ERROR: $VAR1 = '/etc/foo.rc'; # ' # at t/RPM/Grill/Plugin/ManPages/10basic.t line 53. # Looks like you failed 7 tests of 15. t/RPM/Grill/Plugin/ManPages/10basic.t .................. not ok 4 - binary-needs-manpage : ERROR: $VAR1 = '/usr/sbin/foo'; not ok 8 - manpage-in-another-arch : ERROR: $VAR1 = '/usr/sbin/foo'; not ok 10 - manpage-in-arch-but-configfile-is-in-noarch : ERROR: $VAR1 = '/etc/gitweb.conf'; not ok 11 - manpage-in-only-some-arches : ERROR: $VAR1 = '/etc/gitweb.conf'; not ok 12 - bad-gzip-manpage : ERROR: $VAR1 = ' not ok 13 - bad-manpage-content : ERROR: $VAR1 = 'a /bin/nosh script, ASCII text executable'; not ok 15 - config-file-missing-manpage : ERROR: $VAR1 = '/etc/foo.rc'; Dubious, test returned 7 (wstat 1792, 0x700) Failed 7/15 subtests t/RPM/Grill/Plugin/Manifest/10basic.t .................. ok $VAR1 = '/usr/local'; # Looks like your test exited with 255 just after 7. t/RPM/Grill/Plugin/Manifest/20fhs.t .................... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 14/21 subtests t/RPM/Grill/Plugin/Manifest/30systemd.t ................ ok t/RPM/Grill/Plugin/Manifest/50usr.t .................... ok t/RPM/Grill/Plugin/Manifest/60perms.t .................. ok $VAR1 = '%filter_setup'; # Looks like your test exited with 255 just after 5. t/RPM/Grill/Plugin/Multilib/10basic.t .................. Dubious, test returned 255 (wstat 65280, 0xff00) Failed 2/7 subtests $VAR1 = '%define _default_patch_fuzz 2'; # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 2 just after 4. t/RPM/Grill/Plugin/Patches/01basic.t ................... Dubious, test returned 2 (wstat 512, 0x200) All 4 subtests passed $VAR1 = 'groupadd -f -g %{abrt_gid_uid} --system abrtX'; # Looks like your test exited with 255 just after 3. t/RPM/Grill/Plugin/RpmScripts/10basic.t ................ Dubious, test returned 255 (wstat 65280, 0xff00) Failed 16/19 subtests $VAR1 = '<action id="org.fedoraproject.config.users.pkexec.run">'; # Looks like your test exited with 255 just after 4. t/RPM/Grill/Plugin/SecurityPolicy/10basic.t ............ Dubious, test returned 255 (wstat 65280, 0xff00) Failed 1/5 subtests $VAR1 = 'actionpack'; # Looks like your test exited with 255 just after 1. t/RPM/Grill/Plugin/SecurityPolicy/20rubygems_cves.t .... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 26/27 subtests $VAR1 = 'PATH=/home/test_dir'; # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 29 just after 5. t/RPM/Grill/Plugin/SecurityPolicy/30suspicious_path.t .. Dubious, test returned 29 (wstat 7424, 0x1d00) All 5 subtests passed $VAR1 = '/usr/bin/non-wl-suid'; # Looks like your test exited with 255 just after 8. t/RPM/Grill/Plugin/Setxid/10basic.t .................... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 6/14 subtests $VAR1 = '* Thu Jul 1 2010 \'test": Mu<u>ñ</u>oz <u>[80]</u><santiago> 1.0-1'; # Looks like your test exited with 2 just after 5. t/RPM/Grill/Plugin/SpecFileEncoding/01basic.t .......... Dubious, test returned 2 (wstat 512, 0x200) Failed 3/8 subtests # Failed test 'polkit-0.109-2.el7.spec : $VAR1 = '* Mon Jan 7 2013 Matthias Clasen <mclasen> - 0.109-2%{?dist}'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'qpid-cpp-mrg-0.10-5.el5.spec : $VAR1 = '* Thu Apr 28 2011 Ted Ross <tross> - 0.10-5'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'tomcat5-5.5.33-22_patch_07.ep5.el4.spec : $VAR1 = '# '; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-1-badversion-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.1-1'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-2-badrelease-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-5'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-4-comments-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-5-unexpected-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 0:1.2.3-4'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-6-wrong-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 2:1.2.3-4'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-7-pkgname-in-changelog-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> zzz-7-pkgname-in-changelog-1.2.3-4'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-8-cruft-in-changelog-1.2-3.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> blahblah-1.2-3'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Failed test 'zzz-9-macrosurprise-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4'; # ' # at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92. # Looks like you failed 11 tests of 15. t/RPM/Grill/Plugin/SpecFileSanity/10basic.t ............ not ok 4 - polkit-0.109-2.el7.spec : $VAR1 = '* Mon Jan 7 2013 Matthias Clasen <mclasen> - 0.109-2%{?dist}'; not ok 5 - qpid-cpp-mrg-0.10-5.el5.spec : $VAR1 = '* Thu Apr 28 2011 Ted Ross <tross> - 0.10-5'; not ok 6 - tomcat5-5.5.33-22_patch_07.ep5.el4.spec : $VAR1 = '\# '; not ok 7 - zzz-1-badversion-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.1-1'; not ok 8 - zzz-2-badrelease-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-5'; not ok 10 - zzz-4-comments-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4'; not ok 11 - zzz-5-unexpected-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 0:1.2.3-4'; not ok 12 - zzz-6-wrong-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 2:1.2.3-4'; not ok 13 - zzz-7-pkgname-in-changelog-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> zzz-7-pkgname-in-changelog-1.2.3-4'; not ok 14 - zzz-8-cruft-in-changelog-1.2-3.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> blahblah-1.2-3'; not ok 15 - zzz-9-macrosurprise-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4'; Dubious, test returned 11 (wstat 2816, 0xb00) Failed 11/15 subtests # Failed test 'clamav-testdata : ERROR: $VAR1 = 'Clamav.Test.File-6 FOUND'; # ' # at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98. # Failed test 'eicar : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND'; # ' # at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98. # Failed test 'eicar-multiple : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND'; # ' # at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98. # Looks like you failed 3 tests of 6. t/RPM/Grill/Plugin/VirusCheck/01basic.t ................ not ok 4 - clamav-testdata : ERROR: $VAR1 = 'Clamav.Test.File-6 FOUND'; not ok 5 - eicar : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND'; not ok 6 - eicar-multiple : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND'; Dubious, test returned 3 (wstat 768, 0x300) Failed 3/6 subtests t/RPM/Grill/RPM/10basic.t .............................. ok t/RPM/Grill/RPM/20requires.t ........................... ok t/RPM/Grill/RPM/30daemons.t ............................ ok t/RPM/Grill/RPM/Files/10numeric_mode.t ................. ok t/RPM/Grill/RPM/Files/20eu-readelf.t ................... ok t/RPM/Grill/RPM/Files/30file_type.t .................... ok t/RPM/Grill/RPM/Files/40dw_at_producer.t ............... ok t/RPM/Grill/RPM/Specfile/01basic.t ..................... ok $VAR1 = 'Muñ'; # Looks like your test exited with 255 just after 1. t/RPM/Grill/Util/10sanitize_text.t ..................... Dubious, test returned 255 (wstat 65280, 0xff00) Failed 3/4 subtests t/perlcritic.t ......................................... ok t/pod-coverage.t ....................................... skipped: still in development t/pod.t ................................................ skipped: temporary disable for development Test Summary Report ------------------- t/RPM/Grill/Plugin/BuildLog/01basic.t (Wstat: 65280 Tests: 3 Failed: 0) Non-zero exit status: 255 Parse errors: No plan found in TAP output t/RPM/Grill/Plugin/DesktopLint/10basic.t (Wstat: 6400 Tests: 4 Failed: 0) Non-zero exit status: 25 Parse errors: Bad plan. You planned 21 tests but ran 4. t/RPM/Grill/Plugin/ManPages/10basic.t (Wstat: 1792 Tests: 15 Failed: 7) Failed tests: 4, 8, 10-13, 15 Non-zero exit status: 7 t/RPM/Grill/Plugin/Manifest/20fhs.t (Wstat: 65280 Tests: 7 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 21 tests but ran 7. t/RPM/Grill/Plugin/Multilib/10basic.t (Wstat: 65280 Tests: 5 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 7 tests but ran 5. t/RPM/Grill/Plugin/Patches/01basic.t (Wstat: 512 Tests: 4 Failed: 0) Non-zero exit status: 2 Parse errors: No plan found in TAP output t/RPM/Grill/Plugin/RpmScripts/10basic.t (Wstat: 65280 Tests: 3 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 19 tests but ran 3. t/RPM/Grill/Plugin/SecurityPolicy/10basic.t (Wstat: 65280 Tests: 4 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 5 tests but ran 4. t/RPM/Grill/Plugin/SecurityPolicy/20rubygems_cves.t (Wstat: 65280 Tests: 1 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 27 tests but ran 1. t/RPM/Grill/Plugin/SecurityPolicy/30suspicious_path.t (Wstat: 7424 Tests: 5 Failed: 0) Non-zero exit status: 29 Parse errors: No plan found in TAP output t/RPM/Grill/Plugin/Setxid/10basic.t (Wstat: 65280 Tests: 8 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 14 tests but ran 8. t/RPM/Grill/Plugin/SpecFileEncoding/01basic.t (Wstat: 512 Tests: 5 Failed: 0) Non-zero exit status: 2 Parse errors: Bad plan. You planned 8 tests but ran 5. t/RPM/Grill/Plugin/SpecFileSanity/10basic.t (Wstat: 2816 Tests: 15 Failed: 11) Failed tests: 4-8, 10-15 Non-zero exit status: 11 t/RPM/Grill/Plugin/VirusCheck/01basic.t (Wstat: 768 Tests: 6 Failed: 3) Failed tests: 4-6 Non-zero exit status: 3 t/RPM/Grill/Util/10sanitize_text.t (Wstat: 65280 Tests: 1 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 4 tests but ran 1. Files=44, Tests=661, 37 wallclock secs ( 0.15 usr 0.03 sys + 36.88 cusr 1.38 csys = 38.44 CPU) Result: FAIL I got stuck fixing these in Util.pm, because the decode function from Encode returned only a substring from the non-utf8 input (10sanitize_text.t) rather then the full string. I couldn't figure out if that was a bug (probably since the documentation claims it just replaces invalid characters) or a missing argument the function needs to be invoked with.
I've put my changes in a branch published under: https://github.com/romanofski/rpmgrill/tree/tests_work_on_F25
A few patches to address the problems with the tests: https://github.com/default-to-open/rpmgrill/commit/8373f34456cdb092f86e3e41cd33819a4c7fbd29 https://github.com/default-to-open/rpmgrill/commit/2cc8552644d91d54bac9f8465d81100f4d65aec7 https://github.com/default-to-open/rpmgrill/commit/81789759e6785662232cf55e05846ed6f5312a69 https://github.com/default-to-open/rpmgrill/commit/c33a016c8521300140914f34ba1e20bf2f0600bb
This has been released with rpmgrill-0.30.