Bug 1409683 - Lots of tests fail on Fedora 25
Summary: Lots of tests fail on Fedora 25
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: rpmgrill
Classification: Community
Component: Tests
Version: 0.29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 0.30
Assignee: Roman Joost
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-03 00:39 UTC by Roman Joost
Modified: 2017-01-23 00:48 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-01-23 00:48:10 UTC
Embargoed:


Attachments (Terms of Use)

Description Roman Joost 2017-01-03 00:39:50 UTC
Description of problem:

Lots of tests fail on Fedora 25:

$ prove -lrcf t
# Testing RPM::Grill 0.01
t/00.load.t ............................................ ok
Possible precedence issue with control flow operator at /code/t/../bin/rpmgrill-fetch-build line 447.
Fetch failed! HTTP response: 404 Not Found [404 File `/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' does not exist] at /code/t/../bin/rpmgrill-fetch-build line 413.
Could not copy '/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' to 't-FetchBuild.odnY8b/testbuild-0.1-1.fc20.i386.rpm': No such file or directory  at /code/t/../bin/rpmgrill-fetch-build line 413.
Could not copy '/code/t-FetchBuild.odnY8b/packages/testbuild/0.1/1.fc20/i386/testbuild-0.1-1.fc20.i386.rpm' to 't-FetchBuild.odnY8b/testbuild-0.1-1.fc20.i386.rpm': No such file or directory  at /code/t/../bin/rpmgrill-fetch-build line 413.
Use of uninitialized value in subroutine entry at /code/t/../bin/rpmgrill-fetch-build line 642.
t/10fetch-build.t ...................................... ok
t/RPM/Grill/10arch-order.t ............................. ok
t/RPM/Grill/12major_release_ge.t ....................... ok
t/RPM/Grill/20multilib.t ............................... ok
t/RPM/Grill/30aggregate_gripes.t ....................... ok
t/RPM/Grill/40codes.t .................................. ok
t/RPM/Grill/50as_xml.t ................................. ok
t/RPM/Grill/60blacklisting.t ........................... ok
t/RPM/Grill/Plugin/10order.t ........................... ok
t/RPM/Grill/Plugin/20docs.t ............................ ok
t/RPM/Grill/Plugin/90real-world.t ...................... ok
$VAR1 = 'update.c: In function `update\':';
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 255 just after 3.
t/RPM/Grill/Plugin/BuildLog/01basic.t .................. 
Dubious, test returned 255 (wstat 65280, 0xff00)
All 3 subtests passed 
$VAR1 = '[Desktop Entry]';
# Looks like your test exited with 25 just after 4.
t/RPM/Grill/Plugin/DesktopLint/10basic.t ............... 
Dubious, test returned 25 (wstat 6400, 0x1900)
Failed 17/21 subtests 
t/RPM/Grill/Plugin/ElfChecks/10basic.t ................. ok
t/RPM/Grill/Plugin/ElfChecks/20relro.t ................. ok

#   Failed test 'binary-needs-manpage : ERROR: $VAR1 = '/usr/sbin/foo';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'manpage-in-another-arch : ERROR: $VAR1 = '/usr/sbin/foo';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'manpage-in-arch-but-configfile-is-in-noarch : ERROR: $VAR1 = '/etc/gitweb.conf';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'manpage-in-only-some-arches : ERROR: $VAR1 = '/etc/gitweb.conf';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'bad-gzip-manpage : ERROR: $VAR1 = '
# gzip: /usr/share/man/man1/foo.1.gz: not in gzip format
#  (\'file\' classifies this file as: ASCII text)';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'bad-manpage-content : ERROR: $VAR1 = 'a /bin/nosh script, ASCII text executable';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.

#   Failed test 'config-file-missing-manpage : ERROR: $VAR1 = '/etc/foo.rc';
# '
#   at t/RPM/Grill/Plugin/ManPages/10basic.t line 53.
# Looks like you failed 7 tests of 15.
t/RPM/Grill/Plugin/ManPages/10basic.t .................. 
not ok 4 - binary-needs-manpage : ERROR: $VAR1 = '/usr/sbin/foo';
not ok 8 - manpage-in-another-arch : ERROR: $VAR1 = '/usr/sbin/foo';
not ok 10 - manpage-in-arch-but-configfile-is-in-noarch : ERROR: $VAR1 = '/etc/gitweb.conf';
not ok 11 - manpage-in-only-some-arches : ERROR: $VAR1 = '/etc/gitweb.conf';
not ok 12 - bad-gzip-manpage : ERROR: $VAR1 = '
not ok 13 - bad-manpage-content : ERROR: $VAR1 = 'a /bin/nosh script, ASCII text executable';
not ok 15 - config-file-missing-manpage : ERROR: $VAR1 = '/etc/foo.rc';
Dubious, test returned 7 (wstat 1792, 0x700)
Failed 7/15 subtests 
t/RPM/Grill/Plugin/Manifest/10basic.t .................. ok
$VAR1 = '/usr/local';
# Looks like your test exited with 255 just after 7.
t/RPM/Grill/Plugin/Manifest/20fhs.t .................... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 14/21 subtests 
t/RPM/Grill/Plugin/Manifest/30systemd.t ................ ok
t/RPM/Grill/Plugin/Manifest/50usr.t .................... ok
t/RPM/Grill/Plugin/Manifest/60perms.t .................. ok
$VAR1 = '%filter_setup';
# Looks like your test exited with 255 just after 5.
t/RPM/Grill/Plugin/Multilib/10basic.t .................. 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 2/7 subtests 
$VAR1 = '%define _default_patch_fuzz 2';
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 2 just after 4.
t/RPM/Grill/Plugin/Patches/01basic.t ................... 
Dubious, test returned 2 (wstat 512, 0x200)
All 4 subtests passed 
$VAR1 = 'groupadd -f -g %{abrt_gid_uid} --system abrtX';
# Looks like your test exited with 255 just after 3.
t/RPM/Grill/Plugin/RpmScripts/10basic.t ................ 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 16/19 subtests 
$VAR1 = '<action id="org.fedoraproject.config.users.pkexec.run">';
# Looks like your test exited with 255 just after 4.
t/RPM/Grill/Plugin/SecurityPolicy/10basic.t ............ 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 1/5 subtests 
$VAR1 = 'actionpack';
# Looks like your test exited with 255 just after 1.
t/RPM/Grill/Plugin/SecurityPolicy/20rubygems_cves.t .... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 26/27 subtests 
$VAR1 = 'PATH=/home/test_dir';
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 29 just after 5.
t/RPM/Grill/Plugin/SecurityPolicy/30suspicious_path.t .. 
Dubious, test returned 29 (wstat 7424, 0x1d00)
All 5 subtests passed 
$VAR1 = '/usr/bin/non-wl-suid';
# Looks like your test exited with 255 just after 8.
t/RPM/Grill/Plugin/Setxid/10basic.t .................... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 6/14 subtests 
$VAR1 = '* Thu Jul  1 2010 \'test": Mu<u>&ntilde;</u>oz <u>[80]</u><santiago> 1.0-1';
# Looks like your test exited with 2 just after 5.
t/RPM/Grill/Plugin/SpecFileEncoding/01basic.t .......... 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 3/8 subtests 

#   Failed test 'polkit-0.109-2.el7.spec : $VAR1 = '* Mon Jan  7 2013 Matthias Clasen <mclasen> - 0.109-2%{?dist}';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'qpid-cpp-mrg-0.10-5.el5.spec : $VAR1 = '* Thu Apr 28 2011 Ted Ross <tross> - 0.10-5';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'tomcat5-5.5.33-22_patch_07.ep5.el4.spec : $VAR1 = '# ';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-1-badversion-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.1-1';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-2-badrelease-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-5';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-4-comments-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-5-unexpected-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 0:1.2.3-4';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-6-wrong-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 2:1.2.3-4';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-7-pkgname-in-changelog-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> zzz-7-pkgname-in-changelog-1.2.3-4';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-8-cruft-in-changelog-1.2-3.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> blahblah-1.2-3';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.

#   Failed test 'zzz-9-macrosurprise-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4';
# '
#   at t/RPM/Grill/Plugin/SpecFileSanity/10basic.t line 92.
# Looks like you failed 11 tests of 15.
t/RPM/Grill/Plugin/SpecFileSanity/10basic.t ............ 
not ok 4 - polkit-0.109-2.el7.spec : $VAR1 = '* Mon Jan  7 2013 Matthias Clasen <mclasen> - 0.109-2%{?dist}';
not ok 5 - qpid-cpp-mrg-0.10-5.el5.spec : $VAR1 = '* Thu Apr 28 2011 Ted Ross <tross> - 0.10-5';
not ok 6 - tomcat5-5.5.33-22_patch_07.ep5.el4.spec : $VAR1 = '\# ';
not ok 7 - zzz-1-badversion-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.1-1';
not ok 8 - zzz-2-badrelease-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-5';
not ok 10 - zzz-4-comments-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4';
not ok 11 - zzz-5-unexpected-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 0:1.2.3-4';
not ok 12 - zzz-6-wrong-epoch-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 2:1.2.3-4';
not ok 13 - zzz-7-pkgname-in-changelog-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> zzz-7-pkgname-in-changelog-1.2.3-4';
not ok 14 - zzz-8-cruft-in-changelog-1.2-3.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> blahblah-1.2-3';
not ok 15 - zzz-9-macrosurprise-1.2.3-4.spec : $VAR1 = '* Fri May 20 2011 Eduardo Santiago <santiago> 1.2.3-4';
Dubious, test returned 11 (wstat 2816, 0xb00)
Failed 11/15 subtests 

#   Failed test 'clamav-testdata : ERROR: $VAR1 = 'Clamav.Test.File-6 FOUND';
# '
#   at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98.

#   Failed test 'eicar : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND';
# '
#   at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98.

#   Failed test 'eicar-multiple : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND';
# '
#   at t/RPM/Grill/Plugin/VirusCheck/01basic.t line 98.
# Looks like you failed 3 tests of 6.
t/RPM/Grill/Plugin/VirusCheck/01basic.t ................ 
not ok 4 - clamav-testdata : ERROR: $VAR1 = 'Clamav.Test.File-6 FOUND';
not ok 5 - eicar : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND';
not ok 6 - eicar-multiple : ERROR: $VAR1 = 'Eicar-Test-Signature FOUND';
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/6 subtests 
t/RPM/Grill/RPM/10basic.t .............................. ok
t/RPM/Grill/RPM/20requires.t ........................... ok
t/RPM/Grill/RPM/30daemons.t ............................ ok
t/RPM/Grill/RPM/Files/10numeric_mode.t ................. ok
t/RPM/Grill/RPM/Files/20eu-readelf.t ................... ok
t/RPM/Grill/RPM/Files/30file_type.t .................... ok
t/RPM/Grill/RPM/Files/40dw_at_producer.t ............... ok
t/RPM/Grill/RPM/Specfile/01basic.t ..................... ok
$VAR1 = 'Mu&#241;';
# Looks like your test exited with 255 just after 1.
t/RPM/Grill/Util/10sanitize_text.t ..................... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 3/4 subtests 
t/perlcritic.t ......................................... ok
t/pod-coverage.t ....................................... skipped: still in development
t/pod.t ................................................ skipped: temporary disable for development

Test Summary Report
-------------------
t/RPM/Grill/Plugin/BuildLog/01basic.t                (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
t/RPM/Grill/Plugin/DesktopLint/10basic.t             (Wstat: 6400 Tests: 4 Failed: 0)
  Non-zero exit status: 25
  Parse errors: Bad plan.  You planned 21 tests but ran 4.
t/RPM/Grill/Plugin/ManPages/10basic.t                (Wstat: 1792 Tests: 15 Failed: 7)
  Failed tests:  4, 8, 10-13, 15
  Non-zero exit status: 7
t/RPM/Grill/Plugin/Manifest/20fhs.t                  (Wstat: 65280 Tests: 7 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 21 tests but ran 7.
t/RPM/Grill/Plugin/Multilib/10basic.t                (Wstat: 65280 Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 7 tests but ran 5.
t/RPM/Grill/Plugin/Patches/01basic.t                 (Wstat: 512 Tests: 4 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
t/RPM/Grill/Plugin/RpmScripts/10basic.t              (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 19 tests but ran 3.
t/RPM/Grill/Plugin/SecurityPolicy/10basic.t          (Wstat: 65280 Tests: 4 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 5 tests but ran 4.
t/RPM/Grill/Plugin/SecurityPolicy/20rubygems_cves.t  (Wstat: 65280 Tests: 1 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 27 tests but ran 1.
t/RPM/Grill/Plugin/SecurityPolicy/30suspicious_path.t (Wstat: 7424 Tests: 5 Failed: 0)
  Non-zero exit status: 29
  Parse errors: No plan found in TAP output
t/RPM/Grill/Plugin/Setxid/10basic.t                  (Wstat: 65280 Tests: 8 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 14 tests but ran 8.
t/RPM/Grill/Plugin/SpecFileEncoding/01basic.t        (Wstat: 512 Tests: 5 Failed: 0)
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 8 tests but ran 5.
t/RPM/Grill/Plugin/SpecFileSanity/10basic.t          (Wstat: 2816 Tests: 15 Failed: 11)
  Failed tests:  4-8, 10-15
  Non-zero exit status: 11
t/RPM/Grill/Plugin/VirusCheck/01basic.t              (Wstat: 768 Tests: 6 Failed: 3)
  Failed tests:  4-6
  Non-zero exit status: 3
t/RPM/Grill/Util/10sanitize_text.t                   (Wstat: 65280 Tests: 1 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 4 tests but ran 1.
Files=44, Tests=661, 37 wallclock secs ( 0.15 usr  0.03 sys + 36.88 cusr  1.38 csys = 38.44 CPU)
Result: FAIL


I got stuck fixing these in Util.pm, because the decode function from Encode returned only a substring from the non-utf8 input (10sanitize_text.t) rather then the full string. I couldn't figure out if that was a bug (probably since the documentation claims it just replaces invalid characters) or a missing argument the function needs to be invoked with.

Comment 1 Roman Joost 2017-01-03 00:41:37 UTC
I've put my changes in a branch published under:

https://github.com/romanofski/rpmgrill/tree/tests_work_on_F25

Comment 3 Roman Joost 2017-01-23 00:48:10 UTC
This has been released with rpmgrill-0.30.


Note You need to log in before you can comment on or make changes to this bug.