Description of problem: With the new delay feature the inventory/insights sync jobs can take 5 hours instead of <5 seconds. This is not good from a troubleshooting/analyzing perspective because you cannot determine if the 5 hours is expected. Also for the 5 hours a dynflow worker is occupied See attached screenshot. Version-Release number of selected component (if applicable): Satellite-6.12.1 How reproducible: Always Steps to Reproduce: 1. Insights scheduled sync Actual results: The inventory/insights scheduled sync jobs can take 5 hours Expected results: Should be completed in few seconds. Additional info: Expectation is that the delay is used in the planning step and scheduling the job at time "X+<randomdelay>"
While there is a running task in the Dynflow worker, it's not consuming any resources. It's dehydrated and waiting for a timer event. From debugging perspective, opening the steps will show you whether the task is waiting the delay or tries to perform: there is a separate "DelayAction" step that generates the delayed start effect. If the action is executing this step, it means it waits. If the action is performing InventoryFullSync - it's actually working. There is an option to force the delay instead of having it randomized for you: you can set SATELLITE_RH_CLOUD_REQUESTS_DELAY=<seconds> ENV variable to force the delay to any value. This is a good option if you want to run the task synchronously: ~~~ echo "::ForemanTasks.sync_task(InventorySync::Async::InventoryScheduledSync)" | SATELLITE_RH_CLOUD_REQUESTS_DELAY=0 foreman-rake console ~~~ For technical reference, the commit that introduced the delay lives here: https://github.com/theforeman/foreman_rh_cloud/commit/8c38ff76b38f146baea755bc85d333da09d0c327. If it would make more sense, we can add a parameter to the InventoryScheduledSync task directly to set the delay value once. So the call would look like: ~~~ echo "::ForemanTasks.sync_task(InventorySync::Async::InventoryScheduledSync, delay: 0)" | foreman-rake console ~~~ We have considered an option of randomizing the actual start time, but it created a lot of edge cases, so the behavior became unpredictable. The preferred solution was to introduce the delay step, which is apparent to the user (including the delay time, it's an input parameter passed to DelayAction and is available in the dynflow UI).