Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Is the issue related to searching by "installed_package_name" only? This search term may need to get optimized. Or are there other search terms that cause problems?
I can't answer regarding other search terms, as this was my specific need.
The search itself is something that should not be so resource intensive as to cause OOM. I understand the "runaway query" scenario, but this was very simple.
RH support provided this SQL which has no noticible performance impact:
...
select h.name,kip.name,kip.nvra
from hosts as h,
katello_host_installed_packages as khip,
katello_installed_packages as kip
where kip.id=khip.installed_package_id
and khip.host_id=h.id
and kip.name like '$1'
...
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team. Thank you.
Comment 12Charles Slivkoff
2023-07-26 19:48:13 UTC
This issue persists on 6.13.2
There does not seem to be much interest in the satellite reporting features.
I will mention as I have in other BZs, that features such as this, if they are not to be usable in any practical sense, should be removed from future releases.
Charles,
I appreciate your feedback and interest in Satellite's reporting. I am going to update the bugzilla so that it doesn't go through auto-closure so that additional investigation can be performed. There was a private comment where an engineer was looking to confirm if this persisted on 6.13 and I appreciate your feedback that it does.
To your point regarding 'interest in the satellite reporting features', the Reporting feature has had over 150 bugzillas resolved since it was introduced, including over 50 enhancement requests. While it is unfortunate that this issue does exist, the data does indicate that there is interest and activity in this area.
Thank you for your understanding.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Important: Satellite 6.15.0 release), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2024:2010
Description of problem: Customer is running the Registered Hosts report in Satellite 6.8 and using a search term : installed_package_name = splunkforwarder. Customer has about 1200 hosts registered to satellite. The report errors out and seems to cause a OOM error killing off a sidekiq process. Version-Release number of selected component (if applicable): How reproducible: Every time certain search terms are used. Not all search terms error out. Jun 24 11:57:26 rhnsat-03 kernel: Out of memory: Kill process 1323 (sidekiq) score 454 or sacrifice child Jun 24 11:57:26 rhnsat-03 kernel: Killed process 1323 (sidekiq), UID 994, total-vm:18395528kB, anon-rss:17486112kB, file-rss:0kB, shmem-rss:0kB Jun 24 11:57:28 rhnsat-03 systemd: dynflow-sidekiq: main process exited, code=killed, status=9/KILL Jun 24 11:57:28 rhnsat-03 systemd: Unit dynflow-sidekiq entered failed state. Jun 24 11:57:28 rhnsat-03 systemd: dynflow-sidekiq failed. Jun 24 11:57:29 rhnsat-03 systemd: dynflow-sidekiq holdoff time over, scheduling restart. Jun 24 11:57:29 rhnsat-03 systemd: Stopped Foreman jobs daemon - worker on sidekiq. Jun 24 11:57:29 rhnsat-03 systemd: Starting Foreman jobs daemon - worker on sidekiq... Jun 24 11:57:32 rhnsat-03 dynflow-sidekiq@worker: 2021-06-24T15:57:32.486Z 43034 TID-b8smm INFO: GitLab reliable fetch activated! Jun 24 11:57:32 rhnsat-03 dynflow-sidekiq@worker: 2021-06-24T15:57:32.486Z 43034 TID-mnx0u INFO: Booting Sidekiq 5.2.7 with redis options {:id=>"Sidekiq-server-PID-43034", :url=>"redis://localhost:6379/0"} Jun 24 11:57:42 rhnsat-03 dynflow-sidekiq@worker: /usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP Jun 24 11:57:42 rhnsat-03 dynflow-sidekiq@worker: /usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here Jun 24 11:57:55 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-7.0.1.5/app/models/concerns/distributor_version.rb:5: warning: already initialized constant Katello::Glue::Provider::DISTRIBUTOR_VERSION Jun 24 11:57:55 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.27/app/models/katello/glue/provider.rb:3: warning: previous definition of DISTRIBUTOR_VERSION was here Jun 24 11:58:01 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_azure_rm-2.1.3/app/models/foreman_azure_rm/azure_rm_compute.rb:13: warning: circular argument reference - sdk Jun 24 11:58:17 rhnsat-03 systemd: Started Foreman jobs daemon - worker on sidekiq. Task export data: INput: --- job_class: TemplateRenderJob queue: default job_data: job_class: TemplateRenderJob job_id: beebaa52-1352-4772-be55-80e2ef0f7abb provider_job_id: beebaa52-1352-4772-be55-80e2ef0f7abb queue_name: default priority: arguments: - template_id: 108-Host - Registered Content Hosts gzip: false send_mail: false mail_to: casl.edu format: html input_values: '5': value: installed_package_name = splunkforwarder _aj_hash_with_indifferent_access: true _aj_hash_with_indifferent_access: true _aj_hash_with_indifferent_access: true - user_id: 35 _aj_symbol_keys: - user_id executions: 0 exception_executions: {} locale: en timezone: America/New_York enqueued_at: '2021-06-24T15:56:42Z' current_request_id: 84477708-32cb-4383-96c5-c1459b574e60 current_timezone: America/New_York current_user_id: 35 current_organization_id: 5 current_location_id: 2 output: Error: StandardError Could not transition step from pending to running, step already in running. --- - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/error.rb:13:in `new'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:501:in `set_error'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:559:in `execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:68:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in `run_user_code'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in `block in '" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'" Actual results: report fails to complete, OOM error Expected results: Report generates as expected