Bug 1275183
| Summary: | Timeout in errata calculation with multiple concurrent ContentViews containing RedHat repos and Date Filter | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Peter Vreman <peter.vreman> | ||||
| Component: | Errata Management | Assignee: | Justin Sherrill <jsherril> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Katello QA List <katello-qa-list> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 6.1.3 | CC: | bbuckingham, bkearney, cwelton, jcallaha, jsherril, sauchter | ||||
| Target Milestone: | Unspecified | Keywords: | PrioBumpPM, Triaged | ||||
| Target Release: | Unused | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-09-14 21:03:07 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1122832, 1317008 | ||||||
| Attachments: |
|
||||||
I have applied the recommendation "if you're getting "Request Timeout" in the sub-tasks then the work around is as follows"
- Edit /etc/foreman/plugins/katello.yaml and set rest_client_timeout: 3600 (default is 120)
- katello-service restart
Now i can publish 3 CVs with 6 RedHat channels at the same time without a Rest timeout. See below that it still worked even that a request took 17 Minutes and an other took 11 Minutes to complete:
build 02-Dec-2015 10:08:54 2015-12-02 10:08:54 [DEBUG]Requesting GET '[KT]content_views' ("organization_id":3,"name":"product-HOIRHEL-6.6-d20151201")
build 02-Dec-2015 10:08:56 2015-12-02 10:08:56 [INFO ]Publishing Content View 'product-HOIRHEL-6.6-d20151201' to 'Library'
build 02-Dec-2015 10:08:56 2015-12-02 10:08:56 [DEBUG]Requesting POST '[KT]content_views/105/publish' ()
build 02-Dec-2015 10:25:31 2015-12-02 10:25:31 [INFO ]Waiting for 1 Tasks
build 02-Dec-2015 10:25:31 2015-12-02 10:25:31 [INFO ][TASKS]Waiting for 1 tasks, completed 0 tasks, elapsed 0 minutes, waiting 0 seconds
build 02-Dec-2015 10:25:31 2015-12-02 10:25:31 [DEBUG][TASKS]Requesting POST '[TASKS]tasks/bulk_search' ("searches":[{"type":"task","task_id":"88d17eb1-4d94-48b1-94ab-7c7be953e97e"}])
build 02-Dec-2015 09:36:57 2015-12-02 09:36:57 [DEBUG]Requesting GET '[KT]content_views' ("organization_id":3,"name":"product-HOIRHEL-6Server-d20151201")
build 02-Dec-2015 09:37:00 2015-12-02 09:37:00 [INFO ]Publishing Content View 'product-HOIRHEL-6Server-d20151201' to 'Library'
build 02-Dec-2015 09:37:00 2015-12-02 09:37:00 [DEBUG]Requesting POST '[KT]content_views/106/publish' ()
build 02-Dec-2015 09:48:53 2015-12-02 09:48:53 [INFO ]Waiting for 1 Tasks
build 02-Dec-2015 09:48:53 2015-12-02 09:48:53 [INFO ][TASKS]Waiting for 1 tasks, completed 0 tasks, elapsed 0 minutes, waiting 0 seconds
build 02-Dec-2015 09:48:53 2015-12-02 09:48:53 [DEBUG][TASKS]Requesting POST '[TASKS]tasks/bulk_search' ("searches":[{"type":"task","task_id":"c32968f5-8e05-4068-a1e0-9657e1f277a7"}])
Please include the setting of this rest_client_timeout from the API. Currently it can only be set manual and is removed again in the next upgrade
Logging fragment of upgrade to show that the upgrade is removing it: [DEBUG 2015-11-30 13:23:19 main] /Stage[main]/Katello::Config/File[/usr/share/foreman/bundler.d/katello.rb]: The container Class[Katello::Config] will propagate my refresh event [DEBUG 2015-11-30 13:23:19 main] /Stage[main]/Katello::Config/File[/usr/share/foreman/bundler.d/katello.rb]: The container Class[Katello::Config] will propagate my refresh event [DEBUG 2015-11-30 13:23:19 main] Executing 'diff -u /etc/foreman/plugins/katello.yaml /tmp/puppet-file20151130-3834-1y7ig9o-0' [ WARN 2015-11-30 13:23:20 main] /Stage[main]/Katello::Config/File[/etc/foreman/plugins/katello.yaml]/content: [ INFO 2015-11-30 13:23:20 main] --- /etc/foreman/plugins/katello.yaml 2015-11-30 13:15:17.531078004 +0000 [ INFO 2015-11-30 13:23:20 main] +++ /tmp/puppet-file20151130-3834-1y7ig9o-0 2015-11-30 13:23:19.421038148 +0000 [ INFO 2015-11-30 13:23:20 main] @@ -13,7 +13,7 @@ [ INFO 2015-11-30 13:23:20 main] [ INFO 2015-11-30 13:23:20 main] common: [ INFO 2015-11-30 13:23:20 main] [ INFO 2015-11-30 13:23:20 main] - rest_client_timeout: 3600 [ INFO 2015-11-30 13:23:20 main] + rest_client_timeout: 120 [ INFO 2015-11-30 13:23:20 main] [ INFO 2015-11-30 13:23:20 main] post_sync_url: https://localhost/katello/api/v2/repositories/sync_complete?token=M3g6xdbsJkUgpNkQ5H9emr2f77Di4nKA [ INFO 2015-11-30 13:23:20 main] [ INFO 2015-11-30 13:23:20 main] /Stage[main]/Katello::Config/File[/etc/foreman/plugins/katello.yaml]: Filebucketed /etc/foreman/plugins/katello.yaml to puppet with sum 533bc6fc4aee6b580589ec3500bb039e *** Bug 1172556 has been marked as a duplicate of this bug. *** This is fixed in Sat6.2Beta2, the default has been increased to 3600: [crash] root@li-lc-1578:~# grep rest_client_timeout /usr/share/katello-installer-base/modules/katello/templates/katello.yaml.erb :rest_client_timeout: 3600 After installing scenario 'satellite' [crash] root@li-lc-1578:~# grep rest_client_timeout /usr/share/foreman/config/settings.plugins.d/katello.yaml :rest_client_timeout: 3600 *** Bug 1161639 has been marked as a duplicate of this bug. *** It appears that this was moved from a pulp query (in 6.1) to a db query in 6.2. Thus the rest client timeout isn't actually relevant anymore and it seems to only be taking a few seconds to initiate the publish task. moving to on_qa for verification Verified in Satellite 6.2.2 Snap 1.1 Followed the steps outlined in comment #1. Kicked off all content view publishes in rapid succession. [root@rhsm-qe-1 ~]# for i in {9..16}; do hammer -u admin -p changeme content-view publish --id $i --async --organization-id 1; done; Content view is being published with task 6e77516b-8fe2-4c61-b296-2d5e59cd9856 Content view is being published with task 9c9d0d79-c1e6-4812-aaf1-b9104b8252b9 Content view is being published with task 587019b2-dbbc-43c9-82b0-91fa1dcfad5c Content view is being published with task b8005eab-d283-4adc-9a85-be96248386a0 Content view is being published with task f1f11e50-e5c6-4a3b-ade4-bb1912590c12 Content view is being published with task 48a15777-ed34-4cdb-941c-173f436b1610 Content view is being published with task f58876e9-efa3-40b1-bd3a-6d1630d7471d Content view is being published with task be69e830-8a3d-42b5-b002-b5f6107d41af All publishes completed successfully. No timeouts seen in any logs. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1885 |
Created attachment 1086414 [details] Task export with timeout of errata calculation Description of problem: When publishing multiple ContentViews containing the RedHat repositories there is a timeout in the errata claus generation Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Sync RedHat repos of 6,5,6.6,6.7,7.1 releases with at least the following repos: - Kickstart - Server - Optional - RHSCL - Oracle Java - Sat6.1 Tools 2. Create for each RedHat minor a ContentView - E.g. CV-RHEL6.5, CV-RHEL6.6, CV-RHEL6.7 3. Add to each Content View the RedHat repositories of that release 3. Configure on each repository a dDate Filters - Include all Packages without Errata - Include by date everything upto day 2015.10.26 4. Start publishing all repositories in parallel Use Hammer with --async or in multiple terminals. Clicking and waiting in the UI might not be concurrent enough Actual results: timeouts happening Expected results: publish successful Additional info: