Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
It looks this leftover package was causing the issue.
python2-nectar.noarch 1.6.0-1.el7sat @rhel-7-server-satellite-6.6-rpms
updated it to python-nectar-1.6.2-1.el7sat.noarch and problem solved.
But it's interesting to know why this package wasn't updated during the initial 6.6 -> 6.7 upgrade.
And why on 6.7.2 - 6.7.4 it was working.
The problem is 6.6 is using python2-nectar whereas in 6.5, 6.7 and 6.8 it's python-nectar. During 6.6 -> 6.7 upgrade, yum for some reason didn't consider it for an upgrade probably. 6.7.5 introduces this change https://github.com/pulp/pulp_docker/commit/859ae5589b7bd5152814f001f4566d4a270cd3e8 which is conflict with python2-nectar.
On the Satellite system I manage, python-nectar was replaced by python2-nectar when I upgraded from 6.5 to 6.6. When I upgraded to 6.7, python2-nectar was not replaced with python-nectar. I resolved the synchronization problem by manually installing python-nectar, which replaced python2-nectar.
# foreman-maintain packages install python-nectar
The linked Foreman issue (31275) rejected this issue because the cause was a transient problem with Docker Hub. Clearly there is still a problem with Docker repository synchronization so a new Foreman issue should be opened.
Comment 8Tanya Tereshchenko
2021-04-19 18:41:36 UTC
It looks like a packaging issue to me. Moving to the corresponding component.
Closing this bugzilla as a duplicate of bug 1904227. That bug was cloned and delivered in 6.8.2, which left this bugzilla not associated with a release.
The fix that was delivered in 6.8.2 is also being delivered in Satellite 6.9.0 and has been confirmed with internal Satellite installs. The build of python-nectar in that release is : python-nectar-1.6.4-1.el7sat.noarch
*** This bug has been marked as a duplicate of bug 1904227 ***
Description of problem: Docker repo example ~~~ https://registry.redhat.io Upstream Repository Name amq7/amq-streams-bridge-rhel7 ~~~ getting following error and stopped syncing ~~~ nectar.downloaders.threaded:INFO: [c177ac1c] Download failed: Download of https://registry.redhat.io/v2/ failed with code 401: Unauthorized ~~~ after upgrading to 6.7.5. example same repo worked on previous versions example 6.7.2 and 6.7.4 Version-Release number of selected component (if applicable): 6.7.5 How reproducible: Create docker repo set the account and sync Steps to Reproduce: 1. install 6.7.5 2. create the repo Basic Information Name docker Label docker Description Backend Identifier 9e5eacfa-007f-4047-967f-d5769998ebcd Type docker Sync Settings Registry URL https://registry.redhat.io Upstream Repository Name amq7/amq-streams-bridge-rhel7 Verify SSL Yes Upstream Authorization rhn-support-username / ******** HTTP Proxy Global Default (None) Publish via HTTPS Yes Publish via HTTP Yes Published At aganbat67.usersys.redhat.com:5000/operations-docker-docker SSL CA Cert SSL Client Cert SSL Client Key Limit Sync Tags 3. sync Actual results: Nov 1 15:09:27 aganbat67 pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS ==> /var/log/httpd/foreman-ssl_access_ssl.log <== 10.10.181.186 - - [01/Nov/2020:15:09:27 -0500] "GET /pulp/api/v2/status HTTP/1.1" 301 - "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" 10.10.181.186 - - [01/Nov/2020:15:09:27 -0500] "GET /pulp/api/v2/status/ HTTP/1.1" 200 261 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" 10.10.181.186 - admin [01/Nov/2020:15:09:27 -0500] "GET /pulp/api/v2/users/ HTTP/1.1" 200 140 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" ==> /var/log/httpd/foreman-ssl_error_ssl.log <== [Sun Nov 01 15:09:27.710730 2020] [ssl:warn] [pid 26848] [client 10.10.181.186:39280] AH02227: Failed to set r->user to 'SSL_CLIENT_S_DN_CN' [Sun Nov 01 15:09:27.725069 2020] [ssl:warn] [pid 30887] [client 10.10.181.186:39282] AH02227: Failed to set r->user to 'SSL_CLIENT_S_DN_CN' ==> /var/log/foreman/production.log <== 2020-11-01T15:09:27 [I|app|53ce4ec4] Started POST "/katello/api/v2/repositories/39/sync?organization_id=1" for 10.3.112.87 at 2020-11-01 15:09:27 -0500 2020-11-01T15:09:27 [I|app|53ce4ec4] Processing by Katello::Api::V2::RepositoriesController#sync as JSON 2020-11-01T15:09:27 [I|app|53ce4ec4] Parameters: {"id"=>"39", "organization_id"=>"1", "api_version"=>"v2", "repository"=>{"id"=>"39"}} 2020-11-01T15:09:27 [I|bac|53ce4ec4] Task {label: Actions::Katello::Repository::Sync, id: b48ab3ff-b4a5-4962-8479-853304c9c2b4, execution_plan_id: a8a4c479-a039-456c-8bd1-a7af2a8897df} state changed: planning ==> /var/log/messages <== Nov 1 15:09:28 aganbat67 pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS Nov 1 15:09:28 aganbat67 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[2402c056-38db-476a-b7f6-e712da54349b] Nov 1 15:09:28 aganbat67 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[c177ac1c-7c70-4c17-b866-7b19f53e0df3] Nov 1 15:09:28 aganbat67 pulp: urllib3.connectionpool:INFO: [c177ac1c] Starting new HTTPS connection (1): registry.redhat.io Nov 1 15:09:28 aganbat67 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[da11e30d-031f-4789-ae3c-4b2e6cc638b3] Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:INFO: [2402c056] Task pulp.server.async.tasks._queue_reserved_task[2402c056-38db-476a-b7f6-e712da54349b] succeeded in 0.257714610547s: None Nov 1 15:09:28 aganbat67 pulp: nectar.downloaders.threaded:INFO: [c177ac1c] Download failed: Download of https://registry.redhat.io/v2/ failed with code 401: Unauthorized Nov 1 15:09:28 aganbat67 pulp: pulp.server.async.tasks:INFO: [c177ac1c] Task failed : [c177ac1c-7c70-4c17-b866-7b19f53e0df3] Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) Task pulp.server.managers.repo.sync.sync[c177ac1c-7c70-4c17-b866-7b19f53e0df3] raised unexpected: AttributeError("'HTTPThreadedDownloader' object has no attribute 'extra_headers'",) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) Traceback (most recent call last): Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) R = retval = fun(*args, **kwargs) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 686, in __call__ Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) return super(Task, self).__call__(*args, **kwargs) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 108, in __call__ Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) return super(PulpTask, self).__call__(*args, **kwargs) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__ Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) return self.run(*args, **kwargs) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 770, in sync Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) sync_report = sync_repo(transfer_repo, conduit, call_config) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 894, in wrap_f Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) return f(*args, **kwargs) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 84, in sync_repo Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) self.sync_step = sync.SyncStep(repo=repo, conduit=sync_conduit, config=config) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py", line 79, in __init__ Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) v2_found = v2_enabled and self.index_repository.api_version_check() Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/registry.py", line 330, in api_version_check Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) headers, body = self._get_path(self.API_VERSION_CHECK_PATH) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/registry.py", line 516, in _get_path Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) auth_header, self.name) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/auth_util.py", line 93, in request_token Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) downloader.extra_headers.pop('Authorization', None) Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:ERROR: [c177ac1c] (26243-24736) AttributeError: 'HTTPThreadedDownloader' object has no attribute 'extra_headers' Nov 1 15:09:28 aganbat67 pulp: celery.app.trace:INFO: [da11e30d] Task pulp.server.async.tasks._release_resource[da11e30d-031f-4789-ae3c-4b2e6cc638b3] succeeded in 0.00188999436796s: None ==> /var/log/httpd/foreman-ssl_access_ssl.log <== 10.3.112.87 - - [01/Nov/2020:15:09:27 -0500] "POST /katello/api/v2/repositories/39/sync?organization_id=1 HTTP/1.1" 202 1527 "https://aganbat67.usersys.redhat.com/products//repositories/39" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0" 10.3.112.87 - - [01/Nov/2020:15:09:28 -0500] "GET /tasks/views/task-details.html HTTP/1.1" 304 - "https://aganbat67.usersys.redhat.com/products//repositories/39" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0" 10.10.181.186 - admin [01/Nov/2020:15:09:28 -0500] "POST /pulp/api/v2/repositories/9e5eacfa-007f-4047-967f-d5769998ebcd/actions/sync/ HTTP/1.1" 202 172 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" 10.10.181.186 - admin [01/Nov/2020:15:09:28 -0500] "GET /pulp/api/v2/tasks/c177ac1c-7c70-4c17-b866-7b19f53e0df3/ HTTP/1.1" 200 389 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" 10.3.112.87 - - [01/Nov/2020:15:09:28 -0500] "POST /foreman_tasks/api/tasks/bulk_search HTTP/1.1" 200 729 "https://aganbat67.usersys.redhat.com/products//repositories/39" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0" ==> /var/log/foreman/production.log <== 2020-11-01T15:09:27 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: b48ab3ff-b4a5-4962-8479-853304c9c2b4, execution_plan_id: a8a4c479-a039-456c-8bd1-a7af2a8897df} state changed: planned 2020-11-01T15:09:27 [I|app|] Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource 2020-11-01T15:09:28 [I|app|] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource (35.5ms) 2020-11-01T15:09:28 [I|app|] Completed 202 Accepted in 470ms (Views: 35.5ms | ActiveRecord: 65.4ms) 2020-11-01T15:09:28 [I|app|fc8865f8] Started GET "/tasks/views/task-details.html" for 10.3.112.87 at 2020-11-01 15:09:28 -0500 2020-11-01T15:09:28 [I|app|87176d75] Started POST "/foreman_tasks/api/tasks/bulk_search" for 10.3.112.87 at 2020-11-01 15:09:28 -0500 2020-11-01T15:09:28 [I|app|87176d75] Processing by ForemanTasks::Api::TasksController#bulk_search as JSON 2020-11-01T15:09:28 [I|app|87176d75] Parameters: {"searches"=>[{"type"=>"task", "task_id"=>"b48ab3ff-b4a5-4962-8479-853304c9c2b4", "search_id"=>"1"}], "task"=>{}} 2020-11-01T15:09:28 [I|app|87176d75] Completed 200 OK in 48ms (Views: 1.1ms | ActiveRecord: 5.2ms) ==> /var/log/httpd/foreman-ssl_access_ssl.log <== 10.10.181.186 - admin [01/Nov/2020:15:09:28 -0500] "GET /pulp/api/v2/tasks/c177ac1c-7c70-4c17-b866-7b19f53e0df3/ HTTP/1.1" 200 993 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" 10.3.112.87 - - [01/Nov/2020:15:09:29 -0500] "POST /foreman_tasks/api/tasks/bulk_search HTTP/1.1" 200 794 "https://aganbat67.usersys.redhat.com/products//repositories/39/tasks/b48ab3ff-b4a5-4962-8479-853304c9c2b4" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0" ==> /var/log/foreman/production.log <== 2020-11-01T14:52:41 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: 8a007fce-a41d-4276-8546-9f85a243f4a8, execution_plan_id: d4ec0488-48d2-4633-8ab6-3baf5e9c326e} state changed: stopped result: warning 2020-11-01T14:52:41 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: 8a007fce-a41d-4276-8546-9f85a243f4a8, execution_plan_id: d4ec0488-48d2-4633-8ab6-3baf5e9c326e} state changed: stopped result: warning 2020-11-01T15:09:28 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: b48ab3ff-b4a5-4962-8479-853304c9c2b4, execution_plan_id: a8a4c479-a039-456c-8bd1-a7af2a8897df} state changed: running 2020-11-01T15:09:28 [E|bac|] PLP0000: 'HTTPThreadedDownloader' object has no attribute 'extra_headers' (Katello::Errors::PulpError) /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/pulp/abstract_async_task.rb:123:in `block in external_task=' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/pulp/abstract_async_task.rb:121:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/pulp/abstract_async_task.rb:121:in `external_task=' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/pulp/repository/sync.rb:28:in `external_task=' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action/polling.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action/cancellable.rb:14:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/pulp/abstract_async_task.rb:45:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:563:in `block (3 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:32:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/remote_action.rb:16:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/models/katello/concerns/user_extensions.rb:21:in `cp_config' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/remote_action.rb:16:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action/progress.rb:31:in `with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action/progress.rb:17:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/keep_locale.rb:11:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.32/app/lib/actions/middleware/keep_locale.rb:11:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_request_id.rb:49:in `restore_current_request_id' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_user.rb:44:in `restore_curent_user' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_user.rb:15:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5.8/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:27:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:19:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware.rb:32:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/stack.rb:23:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/middleware/world.rb:31:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:562:in `block (2 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:561:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:561:in `block in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:475:in `block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:475:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:475:in `with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:556:in `execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/action.rb:285:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/director.rb:93:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors.rb:12:in `run_user_code' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/worker.rb:14:in `on_message' [ concurrent-ruby ] /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/pool.rb:99:in `distribute_jobs' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/pool.rb:52:in `schedule_work' [ concurrent-ruby ] /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/core.rb:48:in `block in feed_pool' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/core.rb:46:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel/core.rb:46:in `feed_pool' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/abstract/core.rb:123:in `handle_work' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/abstract/core.rb:35:in `handle_event' [ concurrent-ruby ] /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/abstract/core.rb:113:in `on_message' [ concurrent-ruby ] /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/executors/parallel.rb:42:in `delayed_event' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/clock.rb:36:in `apply' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/clock.rb:75:in `run_ready_timers' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/clock.rb:57:in `tick' [ concurrent-ruby ] /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.1/lib/dynflow/clock.rb:96:in `block in sleep_to' [ concurrent-ruby ] 2020-11-01T15:09:29 [I|app|4a246924] Started POST "/foreman_tasks/api/tasks/bulk_search" for 10.3.112.87 at 2020-11-01 15:09:29 -0500 2020-11-01T15:09:29 [I|app|4a246924] Processing by ForemanTasks::Api::TasksController#bulk_search as JSON 2020-11-01T15:09:29 [I|app|4a246924] Parameters: {"searches"=>[{"type"=>"task", "task_id"=>"b48ab3ff-b4a5-4962-8479-853304c9c2b4", "search_id"=>"1"}], "task"=>{}} 2020-11-01T15:09:29 [I|app|4a246924] Completed 200 OK in 42ms (Views: 0.8ms | ActiveRecord: 5.0ms) ^C Expected results: repo must sync Additional info: