Description of problem:
Satellite may create duplicate CreateRssNotifications, CreatePulpDiskSpaceNotifications or SendExpireSoonNotifications when restart foreman-tasks.
# su - postgres -c "psql -d foreman -c 'select label,count(label),state from foreman_tasks_tasks where state <> '\''stopped'\'' group by label,state;'"
label | count | state
----------------------------------------+-------+-----------
CreateRssNotifications | 2 | scheduled
CreatePulpDiskSpaceNotifications | 1 | scheduled
SendExpireSoonNotifications | 1 | scheduled
The "spawn_if_missing" function is looking for task that is in "scheduled" state during the dynflow startup. If no "CreateRssNotifications" is in "schdeduled" state, then create one. I suspect that the task was not in "scheduled" state previously before Dynflow got terminated and cause Dynflow to create a duplicate task during startup.
/usr/share/foreman/app/jobs/application_job.rb
def self.spawn_if_missing(world)
return if (Foreman.in_rake? && !Foreman.in_rake?('dynflow:executor')) || Rails.env.test?
pending_jobs = world.persistence.find_execution_plans(filters: { :state => 'scheduled' })
scheduled_job = pending_jobs.select do |job|
delayed_plan = world.persistence.load_delayed_plan job.id
next unless delayed_plan.present?
arg = delayed_plan.to_hash[:serialized_args].first
arg.is_a?(Hash) && arg['job_class'] == self.to_s
end
# Schedule the job only if it doesn't exit yet
self.perform_later if scheduled_job.blank?
end
I think I can see how this might happen, but I need to know many executors they have.
Running "grep EXECUTORS_COUNT /etc/sysconfig/{dynflowd,foreman-tasks}" should show that.
Verified on Satellite 6.8 snap 1, on fresh machine after several restarts of dynflow-sidekiq@ services the count of the aforementioned tasks remains one
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2020:4366
Description of problem: Satellite may create duplicate CreateRssNotifications, CreatePulpDiskSpaceNotifications or SendExpireSoonNotifications when restart foreman-tasks. # su - postgres -c "psql -d foreman -c 'select label,count(label),state from foreman_tasks_tasks where state <> '\''stopped'\'' group by label,state;'" label | count | state ----------------------------------------+-------+----------- CreateRssNotifications | 2 | scheduled CreatePulpDiskSpaceNotifications | 1 | scheduled SendExpireSoonNotifications | 1 | scheduled The "spawn_if_missing" function is looking for task that is in "scheduled" state during the dynflow startup. If no "CreateRssNotifications" is in "schdeduled" state, then create one. I suspect that the task was not in "scheduled" state previously before Dynflow got terminated and cause Dynflow to create a duplicate task during startup. /usr/share/foreman/app/jobs/application_job.rb def self.spawn_if_missing(world) return if (Foreman.in_rake? && !Foreman.in_rake?('dynflow:executor')) || Rails.env.test? pending_jobs = world.persistence.find_execution_plans(filters: { :state => 'scheduled' }) scheduled_job = pending_jobs.select do |job| delayed_plan = world.persistence.load_delayed_plan job.id next unless delayed_plan.present? arg = delayed_plan.to_hash[:serialized_args].first arg.is_a?(Hash) && arg['job_class'] == self.to_s end # Schedule the job only if it doesn't exit yet self.perform_later if scheduled_job.blank? end