Bug 517215

Summary: over-limit test-generated files can cause a traceback in post-run file attachment code
Product: [Retired] Red Hat Hardware Certification Program Reporter: Greg Nichols <gnichols>
Component: Test Suite (harness)Assignee: Greg Nichols <gnichols>
Status: CLOSED ERRATA QA Contact: Lawrence Lim <llim>
Severity: medium Docs Contact:
Priority: low    
Version: 1.0CC: rlandry, smaitra, tools-bugs, yoyzhang
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-10-14 13:29:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
rundocument.py patch to file warning message formatting
none
Log files and result rpm file
none
Attached log file for V7 USB test
none
Output log for v7 usb test, the fixed version. none

Description Greg Nichols 2009-08-13 02:54:58 UTC
Description of problem:

Files generated by the test that exceed the limit set in the v7 environment file (/etc/v7.xml) cause a traceback in the warning message that the file will be skipped:

copying attachments...
checking directory /tmp/v7-fv_storage-Ptjdjn/mnt/tests/V7/v7/fv_storage 
Skipping test-result.xml
Skipping Makefile
Traceback (most recent call last):
  File "/usr/bin/v7", line 49, in ?
    success = v7.do(args)
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 166, in do
    result = self.doRun()
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 499, in doRun
    self._doRun(tests)
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 569, in _doRun
    run.getResults(self.environment, directory+"/test-result.xml")
  File "/usr/share/v7/lib/v7/rundocument.py", line 156, in getResults
    self.getResultsAttachments(environment, resultsFile)
  File "/usr/share/v7/lib/v7/rundocument.py", line 174, in getResultsAttachments
    print "Warning: " + file + " size: " + os.path.getsize(filePath) + " is too big to attach to the results."
TypeError: cannot concatenate 'str' and 'long' objects


Version-Release number of selected component (if applicable):

v7 1.0 R14

Comment 1 Greg Nichols 2009-08-13 02:57:11 UTC
Created attachment 357267 [details]
rundocument.py patch to file warning message formatting

Comment 2 Rob Landry 2009-08-17 21:45:17 UTC
Should this be a release blocker?

Comment 3 Greg Nichols 2009-08-28 14:44:35 UTC
No. overlimit attachment files shouldn't happen.

Comment 5 zhanghaiyan 2009-09-27 09:57:17 UTC
I have 2 Qs to confirm with your for reproduce step. Thanks a lot

This bug happend on which command? After issue # v7 run -t fv_storage ?
In what condition this bug happens ? After results.xml is bigger than 9M or any others ?

Comment 7 Greg Nichols 2009-10-01 15:59:44 UTC
The bug happens during test runs via "run", or "certify".

I found the this bug when a different bug cause a large downloaded image file to be left in the test directory.   One method to reproduce this bug is to use an interactive test, such as the usb test.   When the test is waiting at a prompt, copy a large file (over 64MB) into the working test directory (under /tmp/v7-usb-*/mnt/tests/V7/v7/usb).   Then let the test continue.   When the test is finishing, you should see a warning message about the file being too big to copy as an attachment, but no traceback.

Comment 8 Satyabrata Maitra 2009-10-05 06:18:39 UTC
Hi Greg

How are you? Few questions :

Do you mean this bug happens during executing the command "v7 certify --test <hardware_name>" ?

For which hardware like audio, video, memory, network, storage, cdrom socket, usb, floppy, core, its showing this traceback? Any specific test or for all the tests or for storage test only?

I think, its shown in terminal at the time of running the test. right? OR its shown at the time of catalog submission in hwcert page with the result RPM?

Which large image file you are talking about?

I am also running the same command. Tests are still running. Lets see.

I expect a specific Test Steps with Expected and Actual Results and "When Reproducible" part to be included with this bug. Its ease the effort of Testing really! :)

Regards
Satya

Comment 9 Satyabrata Maitra 2009-10-05 07:25:08 UTC
Created attachment 363642 [details]
Log files and result rpm file

In my test, I did not find any traceback. Requesting you to check the attached logs and .rpm file for test results.

Version of the component : v7-1.0-18.el5

In my /etc/v7.xml, the maximum attachment size is written "67108864".

Test Steps :

Command executed : v7 certify --test all

                   v7 sumbit

******
Please tell me if the test steps i followed is correct to reproduce this bug if not, please provide the test steps!

Comment 10 Greg Nichols 2009-10-05 14:26:03 UTC

1) install v7 1.0 release 15
2) v7 plan
3) v7 run --test usb
4) at the test prompt, copy a file with a size greater than 64 MB to:
    /tmp/v7-usb-*/mnt/tests/V7/v7/usb
5) continue the test by answering the prompts
6) If you see a traceback as the tests finishes, you've reproduced the bug in R15
7) upgrade v7 to R18
8) repeat steps 3) though 5)
9) As the test completes, you should see a warning message about the file, but no traceback.

Comment 11 Satyabrata Maitra 2009-10-06 07:28:24 UTC
Hi Greg

I followed the test steps from 1 to 3. But
as you said in 4th step, the test prompt really did not give me such place where I can copy a file.

After step 3, its asked "how many USB sockets are required to be tested?"

My Input was 1 (before that I plugged in a USB Pen drive with the system)

System response was "Device not found"

Pasting the terminal message below, printed during the testing :-
**************************************
[root@dhcp1-42 Desktop]# v7 run --test usbOS Version: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Kernel RPM: kernel-2.6.18-164.el5
v7 version 1.0, release 15
the controller /org/freedesktop/Hal/devices/usb_device_204_6025_03110400AC07FA11_if0_scsi_host is already in testplan, will skip the block device /dev/sdd
Hardware: Dell Inc. unknown OptiPlex 755
OS: Tikanga 5.4
saved configuration to /var/v7/results.xml
Created a new plan with 14 tests on 136 devices
saved test plan to /var/v7/results.xml

Running usb:
---------------------------------
usb                   pnp_PNP0103                          
info                                                       

running usb on /org/freedesktop/Hal/devices/pnp_PNP0103
mkdir -p /tmp/v7-usb-32Gdll/mnt/tests/V7/v7/usb
cp -a runtest.sh usb.py Makefile /tmp/v7-usb-32Gdll/mnt/tests/V7/v7/usb
install -m 0755 runtest.sh /tmp/v7-usb-32Gdll/mnt/tests/V7/v7/usb
make OUTPUTFILE=/var/log/v7/runs/1/usb/output.log RUNMODE=normal UDI=/org/freedesktop/Hal/devices/pnp_PNP0103 DEVICE= TESTSERVER=unknown run
chmod a+x ./runtest.sh ./usb.py
./runtest.sh
/tmp/v7-usb-32Gdll/mnt/tests/V7/v7/usb/usb.py
Running ./usb.py:
USB test:
USB Mass Storage Interface appears to be plugged into bus 1 port 4
CBM2080 Flash drive controller appears to be plugged into bus 1 port 4
USB Mass Storage Interface appears to be plugged into bus 4 port 1
Triplet GSM Phone (storage) appears to be plugged into bus 4 port 1
USB Hub Interface appears to be plugged into bus 3 port 0
USB Hub Interface appears to be plugged into bus 4 port 0
USB Hub Interface appears to be plugged into bus 1 port 0
USB Hub Interface appears to be plugged into bus 1 port 2
USB Video Interface appears to be plugged into bus 1 port 2
USB Video Interface appears to be plugged into bus 1 port 2
USB Audio Interface appears to be plugged into bus 1 port 2
USB Audio Interface appears to be plugged into bus 1 port 2
Monitor Webcam appears to be plugged into bus 1 port 2
USB Hub Interface appears to be plugged into bus 1 port 2
Unknown (0x2514) appears to be plugged into bus 1 port 2
Unknown (0x2512) appears to be plugged into bus 1 port 2
USB Hub Interface appears to be plugged into bus 5 port 0
USB Hub Interface appears to be plugged into bus 6 port 0
USB HID Interface appears to be plugged into bus 6 port 2
USB HID Interface appears to be plugged into bus 6 port 2
USB Multimedia Keyboard appears to be plugged into bus 6 port 2
USB Hub Interface appears to be plugged into bus 7 port 0
USB HID Interface appears to be plugged into bus 7 port 1
Dell Optical Mouse appears to be plugged into bus 7 port 1
USB Hub Interface appears to be plugged into bus 2 port 0
How many USB sockets are required to be tested? 1
response: 1
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) y
response: y
device not found
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) y
response: y
device not found
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) y
response: y
device not found
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) n
Skipping runtest.sh
Skipping usb.py
Skipping Makefile
saveOutput: /var/log/v7/runs/1/usb/output.log
Return value was 0
running info on 
mkdir -p /tmp/v7-info-k8TlbX/mnt/tests/V7/v7/info
cp -a testinfo.desc runtest.sh info.py Makefile /tmp/v7-info-k8TlbX/mnt/tests/V7/v7/info
install -m 0755 runtest.sh /tmp/v7-info-k8TlbX/mnt/tests/V7/v7/info
make OUTPUTFILE=/var/log/v7/runs/1/info/output.log RUNMODE=forced UDI= DEVICE= TESTSERVER=unknown run
chmod a+x ./runtest.sh ./info.py
./runtest.sh
/tmp/v7-info-k8TlbX/mnt/tests/V7/v7/info/info.py
Running ./info.py:
OS Version: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Kernel RPM: kernel-2.6.18-164.el5
v7 version 1.0, release 15
+ rpm -ql kernel-2.6.18-164.el5
Boot Parameters: ro root=/dev/VolGroup01/LogVol00 rhgb quiet
Kernel check passed.
v7 Verify passed
checking modules...

Module check passed

This utility will go through and collect some detailed information
about the hardware and setup of your Red Hat Linux system.
This information will be used to diagnose problems with your system
and will be considered confidential information.  Red Hat will use
this information for diagnostic purposes ONLY.

Please wait while we collect information about your system.

This process may take a while to complete....
No changes will be made to your system during this process.

NOTE: You can safely ignore a failed message. This only means a file
we were checking for did not exist.

If your system hangs while gathering rpm information, please abort
the script with CTRL-C and run it again after adding -norpm to the
sysreport command line

Press ENTER to continue, or CTRL-C to quit.

Getting system configuration information.

Determining Red Hat Linux version:                         [  OK  ]
Determinding your current hostname:                        [  OK  ]
Getting the date:                                          [  OK  ]
Checking your systems current uptime and load average:     [  OK  ]
Checking available memory:                                 [  OK  ]
Checking free disk space:                                  [  OK  ]
Checking currently running processes:                      [  OK  ]
Checking current process tree:                             [  OK  ]
Collecting information about ld.so.conf:                   [  OK  ]
Collecting information about ld.so.conf.d:                 [  OK  ]
Collecting IPC-related information:                        [  OK  ]
Collecting information about chkconfig --list:             [  OK  ]
Collecting information about /etc/rc.d:                    [  OK  ]
Lists information about files opened (lsof)                [  OK  ]
Collecting interprocess communication facilities status    [  OK  ]
Collecting interprocess communication facilities status    [  OK  ]
Collecting information about currently installed packages:
*********************************************

I am not quite clear with your 4th step of test. Please explain a bit, what exactly to do. Attaching the log file too for this test run.

Comment 12 Satyabrata Maitra 2009-10-06 07:31:25 UTC
RPM version tested : v7-1.0-15.el5

OS & Arch : RHEL5.4-Server & x86_

Log file is attached below.

Comment 13 Satyabrata Maitra 2009-10-06 07:32:53 UTC
Created attachment 363788 [details]
Attached log file for V7 USB test

Comment 14 Satyabrata Maitra 2009-10-06 09:19:03 UTC
Tested this bug in the older version of V7. Bug Reproduced here in older version.

Test Report on the basis of Older Version :

Bug Reproduced.

Test Step Followed :

1) install v7 1.0 release 15
2) v7 plan
3) v7 run --test usb
4. Test Prompt will ask answer from you, Pause yourself and
5. Open another terminal
6. Copy A big file (larger than 68MB) to /tmp/v7-usb-*/mnt/tests/V7/v7/usb
   Here to find the value or name of (*), you have to look at the terminal.
   Please find the directory name like below :
   "running usb on /org/freedesktop/Hal/devices/pnp_PNP0103
   mkdir -p /tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb"
   Here for my test, (*) is "ZFSEHc" so for example, to copy a big file to the 
   proper directory, I had to execute the command "cp abc.avi /tmp/v7-
   usb-ZFSEHc/mnt/tests/V7/v7/usb".
   
   ** Change your dynamic directory accordingly (This ZFSEHc dir created at the 
   time of test by the auto test script)

5) Now, continue the test by answering the prompts (you can input "0" also)
6) If you see a traceback as the tests finishes, you've reproduced the bug in 
   R15.

Terminal Messages and Traceback at the time of Testing :
[root@dhcp1-42 Desktop]# v7 run --test usb
loaded results /var/v7/results.xml
OS Version: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Kernel RPM: kernel-2.6.18-164.el5
v7 version 1.0, release 15
the controller /org/freedesktop/Hal/devices/usb_device_204_6025_03110400AC07FA11_if0_scsi_host is already in testplan, will skip the block device /dev/sdd
saved test plan to /var/v7/results.xml

Running usb:
---------------------------------
usb                   pnp_PNP0103                          
info                                                       

running usb on /org/freedesktop/Hal/devices/pnp_PNP0103
mkdir -p /tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb
cp -a runtest.sh usb.py Makefile /tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb
install -m 0755 runtest.sh /tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb
make OUTPUTFILE=/var/log/v7/runs/1/usb/output.log RUNMODE=normal UDI=/org/freedesktop/Hal/devices/pnp_PNP0103 DEVICE= TESTSERVER=unknown run
chmod a+x ./runtest.sh ./usb.py
./runtest.sh
/tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb/usb.py
Running ./usb.py:
USB test:
USB Mass Storage Interface appears to be plugged into bus 1 port 4
CBM2080 Flash drive controller appears to be plugged into bus 1 port 4
USB Hub Interface appears to be plugged into bus 3 port 0
USB Hub Interface appears to be plugged into bus 4 port 0
USB Hub Interface appears to be plugged into bus 1 port 0
USB Hub Interface appears to be plugged into bus 1 port 2
USB Video Interface appears to be plugged into bus 1 port 2
USB Video Interface appears to be plugged into bus 1 port 2
USB Audio Interface appears to be plugged into bus 1 port 2
USB Audio Interface appears to be plugged into bus 1 port 2
Monitor Webcam appears to be plugged into bus 1 port 2
USB Hub Interface appears to be plugged into bus 1 port 2
Unknown (0x2514) appears to be plugged into bus 1 port 2
Unknown (0x2512) appears to be plugged into bus 1 port 2
USB Hub Interface appears to be plugged into bus 5 port 0
USB Hub Interface appears to be plugged into bus 6 port 0
USB HID Interface appears to be plugged into bus 6 port 2
USB HID Interface appears to be plugged into bus 6 port 2
USB Multimedia Keyboard appears to be plugged into bus 6 port 2
USB Hub Interface appears to be plugged into bus 7 port 0
USB HID Interface appears to be plugged into bus 7 port 1
Dell Optical Mouse appears to be plugged into bus 7 port 1
USB Hub Interface appears to be plugged into bus 2 port 0
How many USB sockets are required to be tested? 1
response: 1
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) y
response: y
device not found
testing socket 1 of 1...
Please plug in a USB device - continue?  (y|n) n
response: n
...finished running ./usb.py, exit code=1
recovered exit code=1
v7-report-result /V7/v7/usb FAIL /var/log/v7/runs/1/usb/output.log 
copying attachments...
checking directory /tmp/v7-usb-ZFSEHc/mnt/tests/V7/v7/usb 
Skipping test-result.xml
Skipping runtest.sh
Skipping usb.py
Traceback (most recent call last):
  File "/usr/bin/v7", line 49, in ?
    success = v7.do(args)
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 166, in do
    result = self.doRun()
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 499, in doRun
    self._doRun(tests)
  File "/usr/share/v7/lib/v7/hardwaretest.py", line 569, in _doRun
    run.getResults(self.environment, directory+"/test-result.xml")
  File "/usr/share/v7/lib/v7/rundocument.py", line 156, in getResults
    self.getResultsAttachments(environment, resultsFile)
  File "/usr/share/v7/lib/v7/rundocument.py", line 174, in getResultsAttachments
    print "Warning: " + file + " size: " + os.path.getsize(filePath) + " is too big to attach to the results."
TypeError: coercing to Unicode: need string or buffer, int found

********************************************************************************

Will update the test result with latest proposed component version shortly.

Comment 15 Satyabrata Maitra 2009-10-06 09:53:17 UTC
Created attachment 363807 [details]
Output log for v7 usb test, the fixed version.

Tested this bug against the proposed version of the component.

Traceback not Found. Fixed in this version.

Component Version tested : v7-1.0-19.el5
OS & Arch : RHEL5.4-Server & x86_64, i386.

Output file is attached below with this bug.

Terminal Message at the end of test :
*******************************************

...finished running ./info.py, exit code=0
recovered exit code=0
v7-report-result info PASS /var/log/v7/runs/1/info/output.log 
copying attachments...
checking directory /tmp/v7-info-qyWcvI/mnt/tests/V7/v7/info 
Adding /tmp/v7-info-qyWcvI/mnt/tests/V7/v7/info/dhcp1-42.20091006145632.tar.bz2
copied attachment file dhcp1-42.20091006145632.tar.bz2
Skipping testinfo.desc
Skipping test-result.xml
Skipping runtest.sh
Skipping Makefile
Skipping info.py
saveOutput: /var/log/v7/runs/1/info/output.log
Return value was 0
*****************************************************

Comment 16 Satyabrata Maitra 2009-10-06 09:55:42 UTC
Hi Greg

Sorry, pls ignore the Comment #11. I got the tricks! ;)
Thanks.

Comment 18 Satyabrata Maitra 2009-10-06 11:46:03 UTC
Sorry, It should be "its NOT showing Traceback Error" in Comment #17

Comment 20 errata-xmlrpc 2009-10-14 13:29:02 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1498.html