Bug 1654347 - Satellite may create duplicate CreateRssNotifications tasks after restarting foreman tasks
Summary: Satellite may create duplicate CreateRssNotifications tasks after restarting ...
Keywords:
Status: NEW
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Tasks Plugin
Version: 6.4
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Peter Ondrejka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-28 15:00 UTC by Hao Chang Yu
Modified: 2020-02-22 04:24 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Foreman Issue Tracker 25723 Normal New Foreman may create duplicate CreateRssNotifications tasks after restarting foreman tasks 2020-02-18 02:24:57 UTC

Description Hao Chang Yu 2018-11-28 15:00:01 UTC
Description of problem:
Satellite may create duplicate CreateRssNotifications, CreatePulpDiskSpaceNotifications or SendExpireSoonNotifications when restart foreman-tasks.

# su - postgres -c "psql -d foreman -c 'select label,count(label),state from foreman_tasks_tasks where state <> '\''stopped'\'' group by label,state;'"
                 label                  | count |   state   
----------------------------------------+-------+-----------
 CreateRssNotifications                 |     2 | scheduled
 CreatePulpDiskSpaceNotifications       |     1 | scheduled
 SendExpireSoonNotifications            |     1 | scheduled


The "spawn_if_missing" function is looking for task that is in "scheduled" state during the dynflow startup. If no "CreateRssNotifications" is in "schdeduled" state, then create one. I suspect that the task was not in "scheduled" state previously before Dynflow got terminated and cause Dynflow to create a duplicate task during startup.

/usr/share/foreman/app/jobs/application_job.rb

  def self.spawn_if_missing(world)
    return if (Foreman.in_rake? && !Foreman.in_rake?('dynflow:executor')) || Rails.env.test?

    pending_jobs = world.persistence.find_execution_plans(filters: { :state => 'scheduled' })
    scheduled_job = pending_jobs.select do |job|
      delayed_plan = world.persistence.load_delayed_plan job.id
      next unless delayed_plan.present?
      arg = delayed_plan.to_hash[:serialized_args].first
      arg.is_a?(Hash) && arg['job_class'] == self.to_s
    end

    # Schedule the job only if it doesn't exit yet
    self.perform_later if scheduled_job.blank?
  end

Comment 3 Adam Ruzicka 2018-11-29 12:09:08 UTC
I think I can see how this might happen, but I need to know many executors they have. 

Running "grep EXECUTORS_COUNT /etc/sysconfig/{dynflowd,foreman-tasks}" should show that.

Comment 5 Adam Ruzicka 2018-12-18 12:04:32 UTC
Created redmine issue http://projects.theforeman.org/issues/25723 from this bug


Note You need to log in before you can comment on or make changes to this bug.