Bug 1403657

Summary:	rpm-test-trigger should leave a more detailed comment on Errata Tool showing what it tested
Product:	[Community] rpm-test-trigger	Reporter:	Dan Callaghan <dcallagh>
Component:	general	Assignee:	beaker-dev-list
Status:	MODIFIED ---	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unreleased	CC:	jorris
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Dan Callaghan 2016-12-12 06:16:32 UTC

Currently when rpm-test-trigger has finished running tests on an advisory it writes a comment like this:

rpmdeplint: test has passed

but that does not make it easy to know *what* tests it actually decided to run (to check if they are valid) nor can the reader easily re-run the tests themselves locally using their own copy of rpmdeplint. And in case the rpmdeplint run found an error, the comment will be:

rpmdeplint: test has failed

which doesn't show what problems rpmdeplint actually found.

Ultimately we will have some more structured way to store each test run and its results but in the meantime, while we are still just recording Errata Tool comments, we should record more detail in the comment. Ideally showing the rpmdeplint commands that were invoked and the problems reported by each one (if any).

Comment 1 Dan Callaghan 2016-12-12 06:24:49 UTC

ET comments can't contain any markup or Markdown formatting so that limits us somewhat... but I propose the following format for comments:

rpmdeplint overall result: FAIL
------------------------------------------------------------------------
rpmdeplint check \
    --repo=RHEL0,http://something/content/dist/rhel/server/7/7.3/x86_64/os \
    binutils-2.25.1-1.el7.x86_64.rpm
(success)
------------------------------------------------------------------------
rpmdeplint check \
    --repo=RHEL0,http://something/content/dist/rhel/server/7/7.3/x86_64/debug \
    binutils-debuginfo-2.25.1-1.el7.x86_64.rpm
(failed)
Undeclared file conflicts:
x-0.1-1.i386 provides /usr/share/thing which is also provided by b-0.1-1.i386

Comment 2 Dan Callaghan 2016-12-12 06:28:14 UTC

So the idea is that the commands in the comment can be copy-pasted to be run directly, assuming that the user has first used `koji download-build` or similar to get the necessary rpms onto disk.

We would want to strip out the /tmp/somethingsomething prefix from the rpms that rpm-test-trigger has downloaded.

Comment 3 Dan Callaghan 2016-12-12 06:35:53 UTC

Tyrone pointed out that there is a character limit on Errata Tool comments (currently 4000 to fit into some db column I suppose).

So we will need to ensure that any comment we post is truncated to 4000 characters or less. The overall result will always be at the top, and that's the most important piece of information. We should also ensure that any failures are listed before successful runs, so that if the end of the comment is truncated we are still showing why it failed.

Comment 4 Dan Callaghan 2016-12-14 07:49:53 UTC

https://gerrit.beaker-project.org/5544