Bug 1289384

Summary: Include foreman-rake foreman_tasks:cleanup with Satellite
Product: Red Hat Satellite Reporter: Chris Roberts <chrobert>
Component: Tasks PluginAssignee: Katello Bug Bin <katello-bugs>
Status: CLOSED ERRATA QA Contact: Stephen Benjamin <stbenjam>
Severity: high Docs Contact:
Priority: high    
Version: NightlyCC: acontant, aperotti, bbuckingham, chartwel, chorn, cwelton, daniele, ldelouw, mlinden, oshtaier, pmoravec, riehecky, stbenjam
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 09:21:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1353215, 1296845    

Description Chris Roberts 2015-12-08 00:31:55 UTC
Description of problem:
upstream we have the following:

https://github.com/theforeman/foreman-tasks

Tasks cleanup

Although, the history of tasks has an auditing value, some kinds of tasks can grow up in number quite soon. Therefore there is a mechanism how to clean the tasks, using a rake command. When running without any arguments, the tasks are deleted based on the default parameters defined in the code.

foreman-rake foreman_tasks:cleanup
To see what tasks would be deleted, without actually deleting the records, you can run

foreman-rake foreman_tasks:cleanup NOOP=true
By default, only the actions explicitly defined with expiration time in the code, will get cleaned. One can configure new actions, or override the default configuration inside the configuration config/settings.plugins.d/foreman_tasks.yaml, such as:

:foreman-tasks:
  :cleanup:
# the period after witch to delete all the tasks (by default all tasks are not being deleted after some period)
    :after: 365d
# per action settings to override the default defined in the actions (cleanup_after method)
    :actions:
      - :name: Actions::Foreman::Host::ImportFacts
        :after: 10d

The foreman_tasks:cleanup script also accepts additional parameters to specify the search criteria for the cleanup manually:

FILTER: scoped search filter (example: 'label = "Actions::Foreman::Host::ImportFacts"')
AFTER: delete tasks created after AFTER period. Expected format is a number followed by the time unit (s, h, m, y), such as 10d for 10 days (applicable only when the FILTER option is specified)
STATES: comma separated list of task states to touch with the cleanup, by default only stopped tasks are affected (applicable only when the FILTER option is specified)
NOOP: set to "true" if the task should not actually perform the deletion, only report the actions the script would perform
VERBOSE: set to "true" for more verbose output
BATCH_SIZE: the size of batches the tasks get processed in (1000 by default)
To see the current configuration (what actions get cleaned automatically and what is their after period), this script can be used:

foreman-rake foreman_tasks:cleanup:config

Version-Release number of selected component (if applicable):

Installed Packages

candlepin-0.9.49.9-1.el7.noarch
candlepin-common-1.0.22-1.el7.noarch
candlepin-guice-3.0-2_redhat_1.el7.noarch
candlepin-scl-1-5.el7.noarch
candlepin-scl-quartz-2.1.5-6.el7.noarch
candlepin-scl-rhino-1.7R3-3.el7.noarch
candlepin-scl-runtime-1-5.el7.noarch
candlepin-selinux-0.9.49.9-1.el7.noarch
candlepin-tomcat-0.9.49.9-1.el7.noarch
elasticsearch-0.90.10-7.el7.noarch
katello-2.2.0.16-1.el7sat.noarch
katello-certs-tools-2.2.1-1.el7sat.noarch
katello-common-2.2.0.16-1.el7sat.noarch
katello-debug-2.2.0.16-1.el7sat.noarch
katello-default-ca-1.0-1.noarch
katello-installer-2.3.20-1.el7sat.noarch
katello-installer-base-2.3.20-1.el7sat.noarch
katello-server-ca-1.0-1.noarch
katello-service-2.2.0.16-1.el7sat.noarch
libqpid-dispatch-0.4-10.el7.x86_64
pulp-docker-plugins-0.2.5-1.el7sat.noarch
pulp-katello-0.5-1.el7sat.noarch
pulp-nodes-common-2.6.0.17-1.el7sat.noarch
pulp-nodes-parent-2.6.0.17-1.el7sat.noarch
pulp-puppet-plugins-2.6.0.17-1.el7sat.noarch
pulp-puppet-tools-2.6.0.17-1.el7sat.noarch
pulp-rpm-plugins-2.6.0.17-1.el7sat.noarch
pulp-selinux-2.6.0.17-1.el7sat.noarch
pulp-server-2.6.0.17-1.el7sat.noarch
python-gofer-qpid-2.6.6-2.el7sat.noarch
python-isodate-0.5.0-4.pulp.el7sat.noarch
python-kombu-3.0.24-10.pulp.el7sat.noarch
python-pulp-bindings-2.6.0.17-1.el7sat.noarch
python-pulp-common-2.6.0.17-1.el7sat.noarch
python-pulp-docker-common-0.2.5-1.el7sat.noarch
python-pulp-puppet-common-2.6.0.17-1.el7sat.noarch
python-pulp-rpm-common-2.6.0.17-1.el7sat.noarch
python-qpid-0.30-6.el7.noarch
python-qpid-qmf-0.30-5.el7.x86_64
qpid-cpp-client-0.30-9.el7.x86_64
qpid-cpp-client-devel-0.30-9.el7.x86_64
qpid-cpp-server-0.30-9.el7.x86_64
qpid-cpp-server-linearstore-0.30-9.el7.x86_64
qpid-dispatch-router-0.4-10.el7.x86_64
qpid-java-client-0.30-3.el7.noarch
qpid-java-common-0.30-3.el7.noarch
qpid-proton-c-0.9-7.el7.x86_64
qpid-qmf-0.30-5.el7.x86_64
qpid-tools-0.30-4.el7.noarch
ruby193-rubygem-katello-2.2.0.76-1.el7sat.noarch
ruby193-rubygem-qpid_messaging-0.30.0-1.el7sat.x86_64
rubygem-hammer_cli_katello-0.0.7.18-1.el7sat.noarch
rubygem-smart_proxy_pulp-1.0.1.2-1.el7sat.noarch
satellite.toledo.satellite.lab.eng.rdu2.redhat.com-qpid-broker-1.0-1.noarch
satellite.toledo.satellite.lab.eng.rdu2.redhat.com-qpid-client-cert-1.0-1.noarch
satellite.toledo.satellite.lab.eng.rdu2.redhat.com-qpid-router-client-1.0-1.noarch
satellite.toledo.satellite.lab.eng.rdu2.redhat.com-qpid-router-server-1.0-1.noarch

Actual results:

[root@foreman ~]# foreman-rake foreman_tasks:cleanup NOOP=true --trace
rake aborted!
Don't know how to build task 'foreman_tasks:cleanup'
/opt/rh/ruby193/root/usr/share/ruby/rake/task_manager.rb:49:in `[]'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:115:in `invoke_task'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block (2 levels) in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `each'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:88:in `top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:66:in `block in run'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:63:in `run'
/opt/rh/ruby193/root/usr/bin/rake:32:in `<main>'

Expected results:
Tasks to be cleaned up

Additional info:

link to cleanup.rake:

https://github.com/theforeman/foreman-tasks/blob/master/lib/foreman_tasks/tasks/cleanup.rake

Comment 3 Pavel Moravec 2015-12-09 12:00:44 UTC
+1 we need to do some cleanup to have benefits like:

- shrink psql tables *a lot*
- make task list faster to load (due to much less data to process)
- task-export generation much faster, tarball is much smaller, easier to investigate it

Comment 6 Pavel Moravec 2016-02-15 08:38:31 UTC
This improvement is great, but NikStSG found a gotcha:

cleaning tasks this way, a repository synced long time ago is newly shown as Never synced

see https://bugzilla.redhat.com/show_bug.cgi?id=1308430 filed for that

Comment 8 Stephen Benjamin 2016-03-24 16:32:57 UTC
Verified, task is included in Satellite 6.2 Snap 5.1:

[root@sat-r220-06 ~]# foreman-rake foreman_tasks:cleanup NOOP=true
[noop] deleting all tasks matching filter label = Actions::Foreman::Host::ImportFacts AND started_at < "2016-02-23 16:32:02" AND state = stopped
[noop] 0 tasks would be deleted
[noop] deleting all tasks matching filter label = Actions::Foreman::Puppetclass::Import AND started_at < "2016-02-23 16:32:02" AND state = stopped
[noop] 0 tasks would be deleted

Comment 13 errata-xmlrpc 2016-07-27 09:21:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501