Red Hat Bugzilla – Bug 1276443
Package installation via the host errata page times out after ~120 seconds
Last modified: 2017-02-23 14:44:02 EST
I have 2 cases opened in relation to this and bug 1269509, both of which are regarding timeouts and patching performance in 6.1.3. Case 01531872 is related to this timeout issue and 01530408 is related as this same thing applies to repo synch's. What I've found to be the underlying problem (which is also mentioned in bug 1269509) is the reliance on timeouts being a bad idea. Katello / Foreman provide the GUI and Pulp runs the work in these cases, it's a really bad design, IMHO, to have katello tell pulp what to do and then simply forget about it/say it failed if the task isn't completed soon enough. To add insult to injury here, the task isn't cancelled in pulp when Katello times out so, pulp will still get around to doing the work ... eventually. These 2 products need to talk together to provide the end user a much better idea of what's actually going on. After 2 weeks of digging into this, here are the recommendations I'd make ... hope you guys are receptive to ideas: 1. Add a pulp status page that will show the queue depth and at least what tasks are currently processing. Adding the ability to prioritize tasks in pulp, batch cancel in pulp and drill down to see what tasks are what in the queue would be awesome. As critical as pulp is, it cannot be so detached from the GUI. (EX: ALWAYS move patching activities to the front of the queue ahead of repo synch's) 2. Add some logic into Katello or pulp to not queue duplicate tasks. If a repo is synching now, don't add another task to synch it again until after this synch has completed. Same for client updates, etc. All this does is allows the queue to back up to unmanageable proportions and slow the whole system down. 3. Allow a "max queue depth" setting for katello and/or pulp. don't accept more work if there's a big backlog. 4. Repo synch's ... check for changes, if there are none, don't regenerate metadata or synch the capsules, after all, nothing's changed. All this does is adds VERY long running tasks into the katello and pulp queues. This is how I ended up with 11,000 tasks to cancel in pulp.
Please see this comment here for HOTFIX instructions on how to apply large # of errata with Satellite 6: https://bugzilla.redhat.com/show_bug.cgi?id=1269509#c18
VERIFIED: # rpm -qa | grep foreman foreman-compute-1.7.2.48-1.el7sat.noarch ruby193-rubygem-foreman-redhat_access-0.2.4-1.el7sat.noarch rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch ruby193-rubygem-foreman_docker-1.2.0.24-1.el7sat.noarch rubygem-hammer_cli_foreman_docker-0.0.3.10-1.el7sat.noarch foreman-debug-1.7.2.48-1.el7sat.noarch foreman-postgresql-1.7.2.48-1.el7sat.noarch ruby193-rubygem-foreman_openscap-0.3.2.10-1.el7sat.noarch ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch foreman-gce-1.7.2.48-1.el7sat.noarch ruby193-rubygem-foreman_discovery-2.0.0.22-1.el7sat.noarch rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch puppet-foreman_scap_client-0.3.3-9.el7sat.noarch foreman-selinux-1.7.2.17-1.el7sat.noarch foreman-vmware-1.7.2.48-1.el7sat.noarch ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch foreman-proxy-1.7.2.6-1.el7sat.noarch ibm-x3655-04.ovirt.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch foreman-discovery-image-3.0.5-3.el7sat.noarch foreman-1.7.2.48-1.el7sat.noarch ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch foreman-libvirt-1.7.2.48-1.el7sat.noarch ibm-x3655-04.ovirt.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch ibm-x3655-04.ovirt.rhts.eng.bos.redhat.com-foreman-proxy-1.0-1.noarch foreman-ovirt-1.7.2.48-1.el7sat.noarch rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch steps: 1. Select multiple errata(s) to create a transaction that will take at least 120 seconds to apply 2. Apply selected errata Id: e41062bf-bfd1-52db-c67c-f623cff8egg7 Label: Actions::Katello::System::Erratum::Install Name: Install erratum Owner: admin Started at: 2015-12-07 11:52:07 UTC Ended at: 2015-12-07 11:57:10 UTC State: stopped Result: success Params: RHSA-2015:2504, RHBA-2015:2014, RHBA-2015:2018, RHBA-2015:2006, RHBA-2015:2011; system 'ibm-hs22-02.lab.bos.redhat.com'; organization 'Default Organization' perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 libreport-python-2.0.9-25.el6_7.x86_64 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 3:perl-version-0.77-141.el6_7.1.x86_64 4:perl-devel-5.10.1-141.el6_7.1.x86_64 4:perl-5.10.1-141.el6_7.1.x86_64 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 libreport-plugin-kerneloops-2.0.9-25.el6_7.x86_64 perl-CGI-3.51-141.el6_7.1.x86_64 libreport-plugin-mailx-2.0.9-25.el6_7.x86_64 zip-3.0-1.el6_7.1.x86_64 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 libreport-compat-2.0.9-25.el6_7.x86_64 libreport-plugin-rhtsupport-2.0.9-25.el6_7.x86_64 selinux-policy-3.7.19-279.el6_7.7.noarch 4:perl-libs-5.10.1-141.el6_7.1.x86_64 libreport-2.0.9-25.el6_7.x86_64 perl-Test-Simple-0.92-141.el6_7.1.x86_64 libreport-cli-2.0.9-25.el6_7.x86_64 perl-ExtUtils-MakeMaker-6.55-141.el6_7.1.x86_64 1:perl-ExtUtils-ParseXS-2.2003.0-141.el6_7.1.x86_64 libreport-plugin-logger-2.0.9-25.el6_7.x86_64 perl-Test-Harness-3.17-141.el6_7.1.x86_64 grep-2.20-3.el6_7.1.x86_64 libreport-plugin-reportuploader-2.0.9-25.el6_7.x86_64 selinux-policy-targeted-3.7.19-279.el6_7.7.noarch perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 json-c-0.11-12.el6.x86_64 libreport-filesystem-2.0.9-25.el6_7.x86_64 satyr-0.16-2.el6.x86_64 augeas-libs-1.0.0-10.el6.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:2622