Bug 1128008 - track log file size in Beaker
Summary: track log file size in Beaker
Keywords:
Status: NEW
Alias: None
Product: Beaker
Classification: Community
Component: scheduler
Version: 0.17
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: ---
Assignee: beaker-dev-list
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-08 06:01 UTC by Dan Callaghan
Modified: 2018-11-09 23:12 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Dan Callaghan 2014-08-08 06:01:38 UTC
Beaker does not currently record the size of job logs. This would be useful information, for finding logs consuming a large amount of disk space and for implementing a log storage quota system in future.

We don't want to upload the log file size on every chunk (since that would mean an extra call LC->scheduler for every chunk). beaker-transfer could record the log file size when it moves the files to the archive server. The downside to that approach is that we won't have any log file size information while the recipe is running (only after it is finished), and Beaker installations which aren't using beaker-transfer will never have log file size information at all.

Comment 3 Nick Coghlan 2014-08-08 06:08:22 UTC
Could beaker-watchdog track it?

Comment 4 Dan Callaghan 2014-08-08 06:14:17 UTC
The problem is that the LC daemons are (quite intentionally) stateless, so the only place to track it is in the Beaker database itself. Updating that requires a call to the scheduler, which we don't want to do for every log chunk.

Comment 5 Nick Coghlan 2014-08-11 08:06:47 UTC
Could we have a scheme where we rate limited the size updates based on the different between the current size and the last reported size? It would mean some runtime state in the watchdog daemon (to remember the last reported size), but it could safely reset to zero if the daemon was restarted.

Comment 6 Dan Callaghan 2014-08-11 08:40:50 UTC
(In reply to Nick Coghlan from comment #5)
> Could we have a scheme where we rate limited the size updates based on the
> different between the current size and the last reported size? It would mean
> some runtime state in the watchdog daemon (to remember the last reported
> size), but it could safely reset to zero if the daemon was restarted.

I'm not sure that would be worth the effort. It would mean that the recorded size in Beaker would not be reliable. We don't know when a log is "complete" (there's no such notion) so beaker-watchdog would have to just decide after a timeout that a log has stopped growing and its size should be updated -- but if beaker-watchdog is stopped before that happens, and no more chunks are uploaded after that, then the size will never be updated. So beaker-transfer would still need to update the size on log archiving anyway, at which point I don't think you have gained much by making beaker-watchdog do it as well.


Note You need to log in before you can comment on or make changes to this bug.