Bug 747871 - The test suite fails in several ways
The test suite fails in several ways
Status: CLOSED CURRENTRELEASE
Product: Publican
Classification: Community
Component: publican (Show other bugs)
2.8
Unspecified Unspecified
unspecified Severity unspecified
: 3.0
: ---
Assigned To: Jeff Fearn
tools-bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-10-21 04:39 EDT by Raphaël Hertzog
Modified: 2012-10-30 23:11 EDT (History)
3 users (show)

See Also:
Fixed In Version: 3.0.0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-10-30 23:11:28 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Patch to fix the test suite (1.88 KB, patch)
2011-10-21 06:28 EDT, Raphaël Hertzog
no flags Details | Diff
compiler output for ./Build local (994.09 KB, text/plain)
2012-06-15 22:05 EDT, Tim Hildred
no flags Details
Compiler output for ./Build authortest (14.16 KB, text/plain)
2012-06-15 22:06 EDT, Tim Hildred
no flags Details
Compiler output for ./Build local(mark 2) (2.18 MB, text/plain)
2012-07-03 00:48 EDT, Tim Hildred
no flags Details

  None (edit)
Description Raphaël Hertzog 2011-10-21 04:39:31 EDT
I'm not sure what changed compared to 2.7 but the test suite fails during the build of the Debian package:

Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140.
t/000.load.t .................. 1/1 Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140, <DATA> line 1662.
# Testing Publican 2.8
t/000.load.t .................. ok   
t/100.CreateBook.t ............ 1/8 Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140.
# Testing Publican::CreateBook 0.2
t/100.CreateBook.t ............ ok   
t/200.XmlClean.t .............. 1/5 Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140.
# Testing Publican::XmlClean 0.2
t/200.XmlClean.t .............. ok   
t/300.Builder.t ............... 2/4 Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140.
# Testing Publican::Builder 0.2
Use of uninitialized value $value in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 536.
Use of uninitialized value $value in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 536.
Use of uninitialized value $value in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 536.
Use of uninitialized value $string in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 648.
Use of uninitialized value $string in pattern match (m//) at /usr/share/perl5/Config/Simple.pm line 650.
Use of uninitialized value in join or string at /usr/share/perl5/Config/Simple.pm line 624.
Use of uninitialized value $string in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 648.
Use of uninitialized value $string in pattern match (m//) at /usr/share/perl5/Config/Simple.pm line 650.
Use of uninitialized value in join or string at /usr/share/perl5/Config/Simple.pm line 624.
Use of uninitialized value $string in substitution (s///) at /usr/share/perl5/Config/Simple.pm line 648.
Use of uninitialized value $string in pattern match (m//) at /usr/share/perl5/Config/Simple.pm line 650.
Use of uninitialized value in join or string at /usr/share/perl5/Config/Simple.pm line 624.
t/300.Builder.t ............... Failed 1/4 subtests 
t/410.Translate.t ............. 1/3 Name "Win32::Locale::Lexicon" used only once: possible typo at /usr/share/perl/5.12/I18N/LangTags/Detect.pm line 140.
# Testing Publican::Translate 0.2
t/410.Translate.t ............. ok   
t/900.publican.t .............. # 
# Testing blib/script/publican
/home/rhertzog/deb/pkg/publican/blib/script/publican syntax OK
t/900.publican.t .............. 4/13 <standard input>:620: warning [p 5, 13.8i]: can't break line
t/900.publican.t .............. 10/13 Writing pr01s02.html for section
Writing pref-foo2-Preface.html for preface(pref-foo2-Preface)
Writing sect-foo2-Test_Chapter-Test_Section_2.html for section(sect-foo2-Test_Chapter-Test_Section_2)
Writing chap-foo2-Test_Chapter.html for chapter(chap-foo2-Test_Chapter)
Writing appe-foo2-Revision_History.html for appendix(appe-foo2-Revision_History)
Writing ix01.html for index
Writing index.html for book
Making portrait pages on A4 paper (210mmx297mm)
[warning] /usr/bin/fop: Unable to locate servlet-api in /usr/share/java
Couldn't find hyphenation pattern en
t/900.publican.t .............. Failed 1/13 subtests 
t/910.publican.Users_Guide.t .. # Testing bin/publican on the Users_Guide
Failed to load brand file: /usr/share/publican/Common_Content/common/publican.cfg at /home/rhertzog/deb/pkg/publican/blib/script/publican line 647
# result 1 = 512
t/910.publican.Users_Guide.t .. 1/5 
#   Failed test 'Run print_tree'
#   at t/910.publican.Users_Guide.t line 34.
#          got: '512'
#     expected: '0'
Failed to load brand file: /usr/share/publican/Common_Content/common/publican.cfg at /home/rhertzog/deb/pkg/publican/blib/script/publican line 647
# 
# result 2 = 512
t/910.publican.Users_Guide.t .. 2/5 
#   Failed test 'Update POT file'
#   at t/910.publican.Users_Guide.t line 39.
#          got: '512'
#     expected: '0'
Failed to load brand file: /usr/share/publican/Common_Content/common/publican.cfg at /home/rhertzog/deb/pkg/publican/blib/script/publican line 647
# 
# result 3 = 512

#   Failed test 'Update German PO files'
#   at t/910.publican.Users_Guide.t line 48.
#          got: '512'
#     expected: '0'
Failed to load brand file: /usr/share/publican/Common_Content/common/publican.cfg at /home/rhertzog/deb/pkg/publican/blib/script/publican line 647
# 
# result 4 = 512
t/910.publican.Users_Guide.t .. 4/5 
#   Failed test 'build the Users Guide'
#   at t/910.publican.Users_Guide.t line 57.
#          got: '512'
#     expected: '0'
Failed to load brand file: /usr/share/publican/Common_Content/common/publican.cfg at /home/rhertzog/deb/pkg/publican/blib/script/publican line 647
# 
# result 5 = 512

#   Failed test 'build the Users Guide in all formats'
#   at t/910.publican.Users_Guide.t line 69.
#          got: '512'
#     expected: '0'
# Looks like you failed 5 tests of 5.
t/910.publican.Users_Guide.t .. Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/5 subtests 
t/perlcritic.t ................ ok     
t/pod-coverage.t .............. ok   
t/pod.t ....................... ok     

Test Summary Report
-------------------
t/300.Builder.t             (Wstat: 0 Tests: 3 Failed: 0)
  Parse errors: Tests out of sequence.  Found (4) but expected (3)
                Bad plan.  You planned 4 tests but ran 3.
t/900.publican.t            (Wstat: 0 Tests: 12 Failed: 0)
  Parse errors: Bad plan.  You planned 13 tests but ran 12.
t/910.publican.Users_Guide.t (Wstat: 1280 Tests: 5 Failed: 5)
  Failed tests:  1-5
  Non-zero exit status: 5
Files=10, Tests=66, 31 wallclock secs ( 0.12 usr  0.00 sys + 31.88 cusr  1.58 csys = 33.58 CPU)
Result: FAIL
Failed 3/10 test programs. 5/66 subtests failed.


I don't understand why it looks in /usr/share/publican/Common_Content/common/publican.cfg while the test code passes the --common_content option pointing to ../blib/... If you have some hints on what's going wrong...

Note that I don't have the required docbook-xsl 1.76.1 installed yet (I only have 1.75.2) but that did not result in a test suite failure last time either so I don't think it explains anything.
Comment 1 Raphaël Hertzog 2011-10-21 06:28:16 EDT
Created attachment 529465 [details]
Patch to fix the test suite

OK I have found the various problems. First of you need to use --quiet because it looks like publican is outputting stuff that is confusing the test system (it uses stdout to communicate test resultst). That's the reason why it doesn't see some of the test results.

Then you made a mistake assigning an array reference [] to an array @common_opts so instead of the expected result, the command line ends up embedding some random text representation of the pointer instead of the wanted options.

Attached is a patch.
Comment 2 Jeff Fearn 2011-11-17 00:56:09 EST
Applied patches, fixed Config::Simple complaint about setting fields to undef (t/300 errors).

Committed revision 1953.
Comment 3 Michael Hideo 2012-06-07 21:46:18 EDT
build publican 3 from source. look at compiler output. verify that there are no failed perl test on output.
Comment 4 Tim Hildred 2012-06-15 22:05:27 EDT
Created attachment 592271 [details]
compiler output for ./Build local

In this attachment, you can see that the test suite does not run by default.
Comment 5 Tim Hildred 2012-06-15 22:06:50 EDT
Created attachment 592272 [details]
Compiler output for ./Build authortest

In this attachment, you can see that when the authortest is invoked manually, it fails.
Comment 6 Tim Hildred 2012-06-15 22:08:32 EDT
Failing both because 1)according to the verification instructions the test suite should be run as part of the compilation and it isn't and 2)when the test suite is run manually it fails.
Comment 7 Jeff Fearn 2012-06-22 21:04:40 EDT
Fixed authortest failing. Note that author tests are not for general testing and are specifically for non-functional tests and should not be run by default. The default tests are run using `./Build test`.

Switched flag in spec file to run functional tests when building RPM.

FYI for extra fun try running `./Build testcover` ... wait ... wait ... still waiting ... yeah it takes a while ... `firefox cover_db/coverage.html` and witness how many more tests really need to be written :(

To ssh://git.fedorahosted.org/git/publican.git
   8c64c85..d60b0f1  master -> master
Comment 8 Tim Hildred 2012-07-03 00:37:54 EDT
"All tests successful. Files=7, Tests=41, 122 wallclock secs ( 0.15 usr  0.04 sys + 102.62 cusr  3.82 csys = 106.63 CPU)
Result: PASS"
Comment 9 Tim Hildred 2012-07-03 00:48:19 EDT
Created attachment 595876 [details]
Compiler output for ./Build local(mark 2)

Just in case someone wants to see exactly what I got.

Note You need to log in before you can comment on or make changes to this bug.