Bug 844762 - [RFE] In Beaker add options for formatting email notifications sent at end of job
[RFE] In Beaker add options for formatting email notifications sent at end of...
Status: NEW
Product: Beaker
Classification: Community
Component: web UI (Show other bugs)
0.9
Unspecified Unspecified
unspecified Severity unspecified (vote)
: ---
: ---
Assigned To: beaker-dev-list
UX
: FutureFeature, Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-07-31 13:15 EDT by Scott Poore
Modified: 2015-09-13 22:03 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-07 02:24:31 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Scott Poore 2012-07-31 13:15:56 EDT
It would be useful if different formats could be selected for the email notification sent at the end of a beaker job.  

These formats could be selected as preferences on a per user basis?

Or, maybe set in the job xml?
Comment 1 Dan Callaghan 2012-10-02 01:48:17 EDT
What kind of formats are we talking about? Is the current notification e-mail format not suitable?

Or are you talking about allowing users to set a preference for a format string or template, into which Beaker interpolates the recipe info?
Comment 2 Scott Poore 2012-10-03 17:17:07 EDT
Something more like the latter you mentioned.  


Something like this is what we see now:

JobID: 266715 Status: Completed Result: Fail<https://beaker.engineering.redhat.com/jobs/266715>
        RecipeSetID: 466501
                RecipeID: 571959 Arch: x86_64 System: ipaqavmg.idm.lab.bos.redhat.com Distro: RHEL-6.2 OSVersion: RedHatEnterpriseLinux6.2 Status: Completed Result: Fail
                        TaskID: 7108519 TaskName: /CoreOS/sssd-RHEL6.3/Functional/Tests-against-OPENLDAP-SERVER StartTime: 2012-07-26 01:12:09 Duration: 0:57:31.068154 Status: Completed Result: Fail
                RecipeID: 571960 Arch: x86_64 System: ipaqa64vmh.idm.lab.bos.redhat.com Distro: RHEL-6.3 OSVersion: RedHatEnterpriseLinux6.3 Status: Completed Result: Fail
                        TaskID: 7108522 TaskName: /CoreOS/sssd-RHEL6.3/Functional/Tests-against-OPENLDAP-SERVER StartTime: 2012-07-26 01:14:12 Duration: 0:55:26 Status: Completed Result: Fail

I think we're looking for more flexibility.  Either in template, or maybe an API or lib we could use for sending out our own reports?

I'm also adding Yi to this ticket in case he has any specific suggestions/requests here.

Thanks,
Scott
Comment 3 Min Shin 2012-11-07 02:24:31 EST
This bugs is closed as it is either not in the current Beaker scope or we could not find sufficient data in the bug report for consideration.
Please feel free to reopen the bug with additional information and/or business cases behind it.
Comment 4 Scott Poore 2012-11-07 09:56:44 EST
What data is missing?
Comment 5 Scott Poore 2012-11-07 10:02:14 EST
Business justification here I guess would be that we need to be able to restructure email notifications in a more human readable and report friendly format.  Also, to define what information gets sent in notification could make the email notifications FAR more useful at least to my team.

I understand if this is a more complex request that could take a larger effort.  Leaving the bug as long term while it is worked on seems more logical to me.
Comment 6 Dan Callaghan 2012-11-07 17:32:38 EST
The current notification e-mails are not really friendly to machine parsing, nor are they particularly good for humans to read. But I think devising a system to allow users to customise the format is probably overkill (it would need to use a complete template language with looping constructs).

I think the right answer is to make the notification e-mail as human-friendly as possible, and send a proper structured message on a message bus of some sort for scripting/automation.

We've tried that once already and failed due to various issues. But I think it is something we should persist with.
Comment 7 Scott Poore 2012-11-09 12:24:46 EST
Dan,

A more human readable format would definitely help.  Maybe showing the basic tasks test results?  Maybe something like listing the final pass/fail results for each task for each recipe?

Yi, 

Was there something specific you were looking for in Email notifications?

Thanks,
Scott
Comment 8 Yi Zhang 2013-01-08 09:17:01 EST
Let's go for an example here:

Subject: [Beaker Job Completion] [Completed/Fail] 359207: ipa-server - cert automatic renew i386 (rhel6.4 release) [(#105) (svn: ) 2013-01-07 04:00:03 PST]

=== subject format and content are fine. I can not think of anything better


Email body:
JobID: 359207 Status: Completed Result: Fail <https://beaker.engineering.redhat.com/jobs/359207>
	RecipeSetID: 616847
		RecipeID: 756645 Arch: i386 System: storm.idm.lab.bos.redhat.com Distro: RHEL6.4-20130106.n.0 OSVersion: RedHatEnterpriseLinux6.4 Status: Completed Result: Fail
			TaskID: 9949716 TaskName: /CoreOS/ipa-server/acceptance/ipa-autorenewcert StartTime: 2013-01-07 12:43:17 Duration: 0:57:03.631652 Status: Completed Result: Fail

==== my comments are below ====
1. Text format vs HTML format: Do we absolutely have to use "text" format? because if the link is click-able, it would be a lot better
2. Readability: fixed indent and new lines would be great help, like: 
    RecipeID: 756645 
    Arch: i386 
    System: storm.idm.lab.bos.redhat.com 
    Distro: RHEL6.4-20130106.n.0 
    OSVersion: RedHatEnterpriseLinux6.4 
    Status: Completed Result: Fail
3. content of email: is it possible to generate a summary report for beaker test result based on parsing of journal? I have done this 

sample report:
========================== Final Pass/Fail Report ===========================
  Test Date: Sun Jan  4 19:20:44 EST 2015 
     Total : [37] 
     passed: [33] 
     failed: [4] 
 Unfinished: [0] 
     Abort : [0]
     Crash : [0]
 ---------------------------------------------------------
   [   PASS   ]      autorenewcert startup  Check for ipa-server package
   [   PASS   ]      autorenewcert round [1] - find_soon_to_be_renewed_certs
   [   PASS   ]      autorenewcert round [1] - test_ipa_via_kinit_as_admin (Before auto renew triggered)
   [   PASS   ]      autorenewcert round [1] - test_dirsrv_via_ssl_based_ldapsearch (Before auto renew triggered)
   [   PASS   ]      autorenewcert round [1] - calculate_autorenew_date
   [   PASS   ]      autorenewcert round [1] - stop_ipa_certmonger_server (Before autorenew, stop ipa, adjust system to trigger automatic cert renew)
   [   PASS   ]      autorenewcert round [1] - adjust_system_time autorenew
...
...
   [   FAIL   ]      autorenewcert round [2] - check_actually_renewed_certs
   [   FAIL   ]      autorenewcert round [2] - compare epoch time of certs expires date round [2]
   [   FAIL   ]      autorenewcert round [2] - compare_expected_renewal_certs_with_actual_renewed_certs
   [   FAIL   ]      /CoreOS/ipa-server/acceptance/ipa-autorenewcert





code:
makereport()
{ 
    local report=$1
    if [ -n "$report" ];then
        touch $report
    else
        if [ ! -w "$report" ];then
            report=/tmp/rhts.report.$RANDOM.txt
            touch $report
        else
            touch $report
        fi
    fi
    # capture the result and make a simple report
    local total=`rlJournalPrintText | grep "RESULT" | wc -l`
    local unfinished=`rlJournalPrintText | grep "RESULT" | grep "\[unfinished\]" | wc -l`
    local pass=`rlJournalPrintText | grep "RESULT" | grep "\[   PASS   \]" | wc -l`
    local fail=`rlJournalPrintText | grep "RESULT" | grep "\[   FAIL   \]" | wc -l`
    local abort=`rlJournalPrintText | grep "RESULT" | grep "\[  ABORT   \]" | wc -l`
    if rlJournalPrintText | grep "^:: \[   FAIL   \] :: RESULT: $"
    then
        total=$((total-1))
        fail=$((fail-1))
    fi
    echo "========================== Final Pass/Fail Report ===========================" > $report
    echo "  Test Date: `date` " >> $report
    echo "     Total : [$total] "  >> $report
    echo "     Passed: [$pass] "   >> $report
    echo "     Failed: [$fail] "   >> $report
    echo " Unfinished: [$unfinished] "   >> $report
    echo "     Abort : [$abort]"   >> $report
    echo "     Crash : [$crashes]" >> $report
    echo " ---------------------------------------------------------" >> $report
    rlJournalPrintText | grep "RESULT" | grep "\[   PASS   \]"| sed -e 's/:/ /g' -e 's/RESULT//g' >> $report
    echo "" >> $report
    rlJournalPrintText | grep "RESULT" | grep "\[   FAIL   \]"| grep -v "^:: \[   FAIL   \] :: RESULT: $" | sed -e 's/:/ /g' -e 's/RESULT//g'  >> $report
    echo "" >> $report
    rlJournalPrintText | grep "RESULT" | grep "\[unfinished\]"| sed -e 's/:/ /g' -e 's/RESULT//g' >> $report
    echo "" >> $report
    rlJournalPrintText | grep "RESULT" | grep "\[  ABORT   \]"| sed -e 's/:/ /g' -e 's/RESULT//g' >> $report
    echo "===========================[$report]===============================" >> $report
    cat $report
    echo "[`date`] test summary report saved as: $report"
    echo ""
} #makereport

Note You need to log in before you can comment on or make changes to this bug.