Bug 1615003

Summary:	Not getting logs if test case is time out on build server
Product:	[Community] GlusterFS	Reporter:	Mohit Agrawal <moagrawa>
Component:	project-infrastructure	Assignee:	bugs <bugs>
Status:	CLOSED UPSTREAM	QA Contact:
Severity:	high	Docs Contact:
Priority:	medium
Version:	mainline	CC:	amukherj, bugs, gluster-infra, srangana
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-12 12:42:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Mohit Agrawal 2018-08-11 04:42:19 UTC

Description of problem:
I am not able to extract logs if any particular test case timed out
on a build server, in log dump .tar file is showing but I am not able to extract any log from the tar file.


Version-Release number of selected component (if applicable):


How reproducible:
For more please refer this https://build.gluster.org/job/line-coverage/455/console, I am not able to extract log for test case
tests/bugs/core/multiplex-limit-issue-151.t 

Tar files are present in the log dumps
-rw-r--r--. 1 mohit84 mohit84 10K Aug 11 06:59 multiplex-limit-issue-151-01:34:27.tar
-rw-r--r--. 1 mohit84 mohit84 10K Aug 11 07:09 multiplex-limit-issue-151.tar

Steps to Reproduce:
1.
2.
3.

Actual results:
Not able to extract log from tar

Expected results:
The log should be accessible

Additional info:

Comment 1 Atin Mukherjee 2018-08-11 04:49:51 UTC

Any specific reason why in case of time out we've not taken a dump of all the log files?

Comment 2 Shyamsundar 2018-08-11 18:20:13 UTC

This is because the test is calling cleanup twice in its exit bash traps.

- First, the test itself sets a trap to cleanup at
https://github.com/gluster/glusterfs/blob/master/tests/bugs/core/multiplex-limit-issue-151.t#L30

- There is an additional trap set to cleanup in include.rc,
https://github.com/gluster/glusterfs/blob/master/tests/include.rc#L719

The tar ball is generated in the cleanup routine, and also ensures that
on content in the tar balls is between 2 invocations. Thus, calling
cleanup twice will result in an empty tarball.

This can be seen running the test locally as,
`./tests/bugs/distribute/bug-1042725.t`

There are a few things in that test we need clarified,
1. why trap this:
https://github.com/gluster/glusterfs/blob/master/tests/bugs/core/multiplex-limit-issue-151.t#L29
2. Why trap cleanup, rather than invoke it at the end of the test as is
normal

Also, in the merged patch sets 2/4/6/7/8 (of [1]) I had added a cleanup at the
end (as I traced the failure of ./tests/bugs/distribute/bug-1042725.t to
incorrect cleanup by the previous test (or timeout in cleanup)). I did
not do the same in patch set 9.

So, I will post a patch that removes the traps set by this test (so we
get logs from this test), and hence add the manual cleanup at the end of
the test. (see bug#1615037 for the same)

Finally, I do not see an infra bug in this.

[1] https://review.gluster.org/c/glusterfs/+/20637

Comment 3 Nigel Babu 2018-08-13 01:26:55 UTC

This is not an infra bug. Going to move this over to tests.

Comment 4 Worker Ant 2020-03-12 12:42:11 UTC

This bug is moved to https://github.com/gluster/project-infrastructure/issues/22, and will be tracked there from now on. Visit GitHub issues URL for further details