Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1275183 - Timeout in errata calculation with multiple concurrent ContentViews containing RedHat repos and Date Filter
Summary: Timeout in errata calculation with multiple concurrent ContentViews containin...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Errata Management
Version: 6.1.3
Hardware: All
OS: Linux
high
medium
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Katello QA List
URL:
Whiteboard:
: 1161639 1172556 (view as bug list)
Depends On:
Blocks: 1122832 1317008
TreeView+ depends on / blocked
 
Reported: 2015-10-26 08:45 UTC by Peter Vreman
Modified: 2020-01-17 15:35 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-14 21:03:07 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Task export with timeout of errata calculation (3.07 MB, application/x-gzip)
2015-10-26 08:45 UTC, Peter Vreman 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1161639 0 unspecified CLOSED Content View Publish planning may time out using original packages on multiple repos 2022-10-10 12:49:09 UTC
Red Hat Bugzilla 1172556 0 unspecified CLOSED Rest Timeout with publishing Content view 2022-10-10 12:49:09 UTC
Red Hat Knowledge Base (Solution) 2135931 0 None None None 2016-03-07 11:43:20 UTC
Red Hat Product Errata RHBA-2016:1885 0 normal SHIPPED_LIVE Satellite 6.2.2 bug fix update 2016-09-15 00:57:56 UTC

Internal Links: 1172556

Description Peter Vreman 2015-10-26 08:45:27 UTC 
Created attachment 1086414 [details]
Task export with timeout of errata calculation

Description of problem:
When publishing multiple ContentViews containing the RedHat repositories there is a timeout in the errata claus generation

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Sync RedHat repos of 6,5,6.6,6.7,7.1 releases with at least the following repos:
   - Kickstart
   - Server
   - Optional
   - RHSCL
   - Oracle Java
   - Sat6.1 Tools
2. Create for each RedHat minor a ContentView
   - E.g. CV-RHEL6.5, CV-RHEL6.6, CV-RHEL6.7
3. Add to each Content View the RedHat repositories of that release
3. Configure on each repository a dDate Filters
   - Include all Packages without Errata
   - Include by date everything upto day 2015.10.26
4. Start publishing all repositories in parallel
   Use Hammer with --async or in multiple terminals. Clicking and waiting in the UI might not be concurrent enough

Actual results:
timeouts happening

Expected results:
publish successful

Additional info:
Comment 2 Peter Vreman 2015-12-02 11:03:30 UTC 
I have applied the recommendation "if you're getting "Request Timeout" in the sub-tasks then the work around is as follows"
- Edit /etc/foreman/plugins/katello.yaml and set rest_client_timeout: 3600 (default is 120)
- katello-service restart

Now i can publish 3 CVs with 6 RedHat channels at the same time without a Rest timeout. See below that it still worked even that a request took 17 Minutes and an other took 11 Minutes to complete:

build	02-Dec-2015 10:08:54	2015-12-02 10:08:54 [DEBUG]Requesting GET '[KT]content_views' ("organization_id":3,"name":"product-HOIRHEL-6.6-d20151201")
build	02-Dec-2015 10:08:56	2015-12-02 10:08:56 [INFO ]Publishing Content View 'product-HOIRHEL-6.6-d20151201' to 'Library'
build	02-Dec-2015 10:08:56	2015-12-02 10:08:56 [DEBUG]Requesting POST '[KT]content_views/105/publish' ()
build	02-Dec-2015 10:25:31	2015-12-02 10:25:31 [INFO ]Waiting for 1 Tasks
build	02-Dec-2015 10:25:31	2015-12-02 10:25:31 [INFO ][TASKS]Waiting for 1 tasks, completed 0 tasks, elapsed 0 minutes, waiting 0 seconds
build	02-Dec-2015 10:25:31	2015-12-02 10:25:31 [DEBUG][TASKS]Requesting POST '[TASKS]tasks/bulk_search' ("searches":[{"type":"task","task_id":"88d17eb1-4d94-48b1-94ab-7c7be953e97e"}])

build	02-Dec-2015 09:36:57	2015-12-02 09:36:57 [DEBUG]Requesting GET '[KT]content_views' ("organization_id":3,"name":"product-HOIRHEL-6Server-d20151201")
build	02-Dec-2015 09:37:00	2015-12-02 09:37:00 [INFO ]Publishing Content View 'product-HOIRHEL-6Server-d20151201' to 'Library'
build	02-Dec-2015 09:37:00	2015-12-02 09:37:00 [DEBUG]Requesting POST '[KT]content_views/106/publish' ()
build	02-Dec-2015 09:48:53	2015-12-02 09:48:53 [INFO ]Waiting for 1 Tasks
build	02-Dec-2015 09:48:53	2015-12-02 09:48:53 [INFO ][TASKS]Waiting for 1 tasks, completed 0 tasks, elapsed 0 minutes, waiting 0 seconds
build	02-Dec-2015 09:48:53	2015-12-02 09:48:53 [DEBUG][TASKS]Requesting POST '[TASKS]tasks/bulk_search' ("searches":[{"type":"task","task_id":"c32968f5-8e05-4068-a1e0-9657e1f277a7"}])

Please include the setting of this rest_client_timeout from the API. Currently it can only be set manual and is removed again in the next upgrade
Comment 3 Peter Vreman 2015-12-02 11:06:39 UTC 
Logging fragment of upgrade to show that the upgrade is removing it:

[DEBUG 2015-11-30 13:23:19 main]  /Stage[main]/Katello::Config/File[/usr/share/foreman/bundler.d/katello.rb]: The container Class[Katello::Config] will propagate
 my refresh event
[DEBUG 2015-11-30 13:23:19 main]  /Stage[main]/Katello::Config/File[/usr/share/foreman/bundler.d/katello.rb]: The container Class[Katello::Config] will propagate
 my refresh event
[DEBUG 2015-11-30 13:23:19 main]  Executing 'diff -u /etc/foreman/plugins/katello.yaml /tmp/puppet-file20151130-3834-1y7ig9o-0'
[ WARN 2015-11-30 13:23:20 main]  /Stage[main]/Katello::Config/File[/etc/foreman/plugins/katello.yaml]/content:
[ INFO 2015-11-30 13:23:20 main] --- /etc/foreman/plugins/katello.yaml  2015-11-30 13:15:17.531078004 +0000
[ INFO 2015-11-30 13:23:20 main] +++ /tmp/puppet-file20151130-3834-1y7ig9o-0    2015-11-30 13:23:19.421038148 +0000
[ INFO 2015-11-30 13:23:20 main] @@ -13,7 +13,7 @@
[ INFO 2015-11-30 13:23:20 main]
[ INFO 2015-11-30 13:23:20 main]  common:
[ INFO 2015-11-30 13:23:20 main]
[ INFO 2015-11-30 13:23:20 main] -  rest_client_timeout: 3600
[ INFO 2015-11-30 13:23:20 main] +  rest_client_timeout: 120
[ INFO 2015-11-30 13:23:20 main]
[ INFO 2015-11-30 13:23:20 main]    post_sync_url: https://localhost/katello/api/v2/repositories/sync_complete?token=M3g6xdbsJkUgpNkQ5H9emr2f77Di4nKA
[ INFO 2015-11-30 13:23:20 main]
[ INFO 2015-11-30 13:23:20 main]  /Stage[main]/Katello::Config/File[/etc/foreman/plugins/katello.yaml]: Filebucketed /etc/foreman/plugins/katello.yaml to puppet with sum 533bc6fc4aee6b580589ec3500bb039e
Comment 4 Peter Vreman 2016-03-07 09:50:17 UTC 
*** Bug 1172556 has been marked as a duplicate of this bug. ***
Comment 8 Peter Vreman 2016-06-16 12:12:22 UTC 
This is fixed in Sat6.2Beta2, the default has been increased to 3600:

[crash] root@li-lc-1578:~# grep rest_client_timeout /usr/share/katello-installer-base/modules/katello/templates/katello.yaml.erb
  :rest_client_timeout: 3600

After installing scenario 'satellite'

[crash] root@li-lc-1578:~# grep rest_client_timeout /usr/share/foreman/config/settings.plugins.d/katello.yaml
  :rest_client_timeout: 3600
Comment 9 Justin Sherrill 2016-07-22 20:15:14 UTC 
*** Bug 1161639 has been marked as a duplicate of this bug. ***
Comment 11 Justin Sherrill 2016-07-26 19:24:34 UTC 
It appears that this was moved from a pulp query (in 6.1) to a db query in 6.2.  Thus the rest client timeout isn't actually relevant anymore and it seems to only be taking a few seconds to initiate the publish task.  

moving to on_qa for verification
Comment 12 jcallaha 2016-09-12 20:34:03 UTC 
Verified in Satellite 6.2.2 Snap 1.1

Followed the steps outlined in comment #1. Kicked off all content view publishes in rapid succession.

[root@rhsm-qe-1 ~]# for i in {9..16}; do hammer -u admin -p changeme content-view publish --id $i --async --organization-id 1; done;
Content view is being published with task 6e77516b-8fe2-4c61-b296-2d5e59cd9856
Content view is being published with task 9c9d0d79-c1e6-4812-aaf1-b9104b8252b9
Content view is being published with task 587019b2-dbbc-43c9-82b0-91fa1dcfad5c
Content view is being published with task b8005eab-d283-4adc-9a85-be96248386a0
Content view is being published with task f1f11e50-e5c6-4a3b-ade4-bb1912590c12
Content view is being published with task 48a15777-ed34-4cdb-941c-173f436b1610
Content view is being published with task f58876e9-efa3-40b1-bd3a-6d1630d7471d
Content view is being published with task be69e830-8a3d-42b5-b002-b5f6107d41af

All publishes completed successfully. No timeouts seen in any logs.
Comment 14 errata-xmlrpc 2016-09-14 21:03:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1885
Note You need to log in before you can comment on or make changes to this bug.