RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2142992 - Very large or long playbook executions still seem to trigger the DeadlineExceeded error code
Summary: Very large or long playbook executions still seem to trigger the DeadlineExce...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: rhc-worker-playbook
Version: 8.6
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: 8.8
Assignee: Link Dupont
QA Contact: Pavol Kotvan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-11-15 17:55 UTC by Link Dupont
Modified: 2023-05-16 09:24 UTC (History)
2 users (show)

Fixed In Version: rhc-worker-playbook-0.1.8-5.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-16 08:26:58 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-139488 0 None None None 2022-11-15 19:34:25 UTC
Red Hat Product Errata RHBA-2023:2776 0 None None None 2023-05-16 08:27:04 UTC

Description Link Dupont 2022-11-15 17:55:47 UTC
Description of problem:
Very large or long playbook executions still seem to trigger the DeadlineExceeded error code.

Version-Release number of selected component (if applicable):
rhc-0.2.1-9.el8_6.x86_64
rhc-worker-playbook-0.1.8-2.el8_6.x86_64


How reproducible:
always

Steps to Reproduce:
1. Register host with subscription-manager
2. Register host with insights-client
3. Get host UUID from inventory
4. Make sure your user account has the Tasks administrator RBAC role
5. Execute the leapp-preupgrade playbook via the tasks API by sending '{"task":"leapp-preupgrade","hosts":["YOUR HBI UUID"]}' to https://console.redhat.com/api/tasks/v1/executed_task
6. Watch the rhcd logs for the playbook to complete
7. Execute the leapp-preupgrade playbook again.

Actual results:
The dispatch times out with the DeadlineExceeded error message

Expected results:
The dispatch should not time out. The second playbook should execute just like the first.

Additional info:

Original report: https://issues.redhat.com/browse/ADVISOR-2872

Comment 1 Link Dupont 2022-11-16 19:13:25 UTC
I believe I have found a solution to this.

_run_data is not async-aware, so declaring it as an async function and
running it in an event loop results in synchronous execution of the
entire function. This means the Send function only returns once the
entire _run_data function has completed. Running it on the event loop
using run_in_executor runs the function call in a thread executor pool,
ensuring that it doesn't block the calling thread. This is a fairly
naive solution; a better implementation might be able to pass a future
into _run_data, immediately fulfill that future, let the calling
function (Send) return the RPC receipt, and continue running _run_data.

Comment 10 errata-xmlrpc 2023-05-16 08:26:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rhc-worker-playbook bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2776


Note You need to log in before you can comment on or make changes to this bug.