Bug 1378913 - [upgrades] Auto-execute run in db:seed causes upgrade failures
Summary: [upgrades] Auto-execute run in db:seed causes upgrade failures
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Tasks Plugin
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
medium
high vote
Target Milestone: Unspecified
Assignee: Ivan Necas
QA Contact: Lukas Pramuk
URL:
Whiteboard:
: 1374912 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-23 13:53 UTC by Ivan Necas
Modified: 2020-01-17 15:57 UTC (History)
8 users (show)

Fixed In Version: rubygem-foreman-tasks-0.7.14.9-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-26 12:26:54 UTC
Target Upstream Version:


Attachments (Terms of Use)
foreman tasks stopped (151.33 KB, image/png)
2016-10-13 14:00 UTC, jcallaha
no flags Details
foreman tasks started (152.22 KB, image/png)
2016-10-13 14:00 UTC, jcallaha
no flags Details
sync tasks completed (156.76 KB, image/png)
2016-10-13 14:00 UTC, jcallaha
no flags Details


Links
System ID Priority Status Summary Last Updated
Foreman Issue Tracker 16666 None None None 2016-09-23 13:53:18 UTC
Red Hat Product Errata RHBA-2016:2108 normal SHIPPED_LIVE Satellite 6.2.3 Async Bug Release 2016-10-26 16:21:52 UTC

Description Ivan Necas 2016-09-23 13:53:16 UTC
The auto-execute feature is meant to be run once the permanent
executor is starting to pick up tasks that were queed while no
executor was avaiable. However, we run executor also in db:seed (to
not be dependent on external executor at this phase). This can lead to
auto-execute to be run there as well, with errors like this:

Sequel::UniqueConstraintViolation: PG::Error: ERROR:  duplicate key value
violates unique constraint "dynflow_coordinator_records_pkey"
DETAIL:  Key (id, class)=(auto-execute,
Dynflow::Coordinator::AutoExecuteLock) already exists.

The solution should be not trying to auto-execute inside rake tasks

Comment 1 Ivan Necas 2016-09-23 13:53:21 UTC
Created from redmine issue http://projects.theforeman.org/issues/16666

Comment 2 Ivan Necas 2016-09-23 13:53:26 UTC
Upstream bug assigned to inecas@redhat.com

Comment 4 Bryan Kearney 2016-10-03 20:19:12 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/16666 has been resolved.

Comment 7 Ivan Necas 2016-10-06 10:45:19 UTC
*** Bug 1374912 has been marked as a duplicate of this bug. ***

Comment 18 Ivan Necas 2016-10-13 08:07:11 UTC
Steps to verify (it doesn't have to be during upgrade, latest sat 6.2. is enough)

1. stop foreman-tasks
2. run some tasks, such as repo sync
3. see the task in pending state
4. run `foreman-rake db:seed`

Current result

the pending task switches to running/stopped during the db:seed

Expected result

the task is not touched when db:seed is started

the tasks starts being processed when foreman-tasks service is started again.

Comment 19 jcallaha 2016-10-13 13:59:11 UTC
Following the steps provided in #18, I had no issues with the db:seed operation as well as automatically resuming tasks to completion. See below and attached screenshots.

[root@rhsm-qe-1 workstation]# service foreman-tasks stop
Redirecting to /bin/systemctl stop  foreman-tasks.service
[root@rhsm-qe-1 workstation]# foreman-rake db:seed
Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb
Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb
Seeding /usr/share/foreman/db/seeds.d/03-roles.rb
Seeding /usr/share/foreman/db/seeds.d/04-admin.rb
Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb
Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb
Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb
Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb
Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/101-locations.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/102-organizations.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/103-provisioning_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/104-proxy.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/106-mail_notifications.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/107-enable_dynflow.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/108-ensure_sync_notification.rb
Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb
Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb
Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb
Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.9/db/seeds.d/20-foreman_tasks_permissions.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/redhat_access-1.0.13/db/seeds.d/200-update-insights-roles.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0.3/db/seeds.d/50-bootdisk_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-5.0.0.9/db/seeds.d/50_discovery_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.9/db/seeds.d/60-dynflow_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-0.3.0.12/db/seeds.d/60-ssh_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-5.0.0.9/db/seeds.d/60_discovery_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.9/db/seeds.d/61-foreman_tasks_bookmarks.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-0.3.0.12/db/seeds.d/70-job_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/db/seeds.d/75-job_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-0.3.0.12/db/seeds.d/90-bookmarks.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.30/db/seeds.d/990 - provisioning_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_openscap-0.5.3.18/db/seeds.d/openscap_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_openscap-0.5.3.18/db/seeds.d/openscap_policy_notification.rb
All seed files executed
[root@rhsm-qe-1 workstation]# service foreman-tasks start
Redirecting to /bin/systemctl start  foreman-tasks.service

Comment 20 jcallaha 2016-10-13 14:00:07 UTC
Created attachment 1210127 [details]
foreman tasks stopped

Comment 21 jcallaha 2016-10-13 14:00:31 UTC
Created attachment 1210128 [details]
foreman tasks started

Comment 22 jcallaha 2016-10-13 14:00:51 UTC
Created attachment 1210129 [details]
sync tasks completed

Comment 23 Lukas Pramuk 2016-10-13 14:19:50 UTC
VERIFIED.

@satellite-6.2.3-1.0.el7sat.noarch
tfm-rubygem-foreman-tasks-0.7.14.9-1.el7sat.noarch

by using manual reproducer described in comment#18:

...
4. Sync task remained pending after db:seed finished. 

After starting foreman-tasks again Sync task got running and finished successfully

Comment 25 errata-xmlrpc 2016-10-26 12:26:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:2108


Note You need to log in before you can comment on or make changes to this bug.