Bug 1291130

Summary: [RFE] provide a stable "bouncer" URL for job logs, which always redirects to the current storage location
Product: [Retired] Beaker Reporter: Dan Callaghan <dcallagh>
Component: generalAssignee: Dan Callaghan <dcallagh>
Status: CLOSED CURRENTRELEASE QA Contact: tools-bugs <tools-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 21CC: dcallagh, dowang, mjia, rjoost
Target Milestone: 22.0Keywords: FutureFeature, Patch, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-14 05:33:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 915319    

Description Dan Callaghan 2015-12-14 04:34:11 UTC
In Beaker sites which are using beaker-transfer with an archive server (like Red Hat's production Beaker deployment) job logs live on the lab controller while the job is running, and for a short time afterwards, but then they are moved to the archive server by beaker-transfer. At that point all the URLs change.

Beaker always knows what the current location for the logs is. But there are still some situations where we link someone to the logs stored on the lab controller but by the time they click the link, it is invalid and the logs are stored on the archive server instead.

One example is a human opening the job page while the job is still running, leaving it alone for a few minutes (during which time beaker-transfer moves the logs) and then clicking on one of the logs. At that point they will see a 404 message (albeit with a hint about why, due to bug 811241).

Another scenario is a script which uses bkr job-watch to wait the end of a job, followed by bkr job-logs to immediately fetch the log files, or bkr job-results --format=junit-xml to fetch JUnit XML with log URLs. Most likely Beaker will return the lab controller URLs for the logs but shortly aftewards the URLs will become invalid when the logs are moved.

Instead of linking to the logs at their current storage location, we could provide new URLs which will always return a 307 Temporary Redirect to the current storage location. Then every place where we provide a link to the logs (job results in the web UI, taskactions.logs XMLRPC call, JUnit XML results, and in future Beaker job results XML) we would use this "bouncer" URL instead. That ensures if the log storage moves in future the links won't break.

There are endpoints already defined in the lab controller API which we could re-use on the server side:

GET /recipes/(recipe_id)/logs/(path: path)
GET /recipes/(recipe_id)/tasks/(task_id)/logs/(path: path)
GET /recipes/(recipe_id)/tasks/(task_id)/results/(result_id)/logs/(path: path)

Comment 1 Dan Callaghan 2015-12-14 23:59:43 UTC
I would like to get this into 22.0 since it's necessary to make the JUnit XML output actually usable.

Comment 2 Dan Callaghan 2015-12-15 01:49:11 UTC
http://gerrit.beaker-project.org/4542

Comment 5 Dan Callaghan 2016-01-14 05:33:32 UTC
Beaker 22.0 has been released.