Bug 1263650

Summary: hammer import repository-enable publishes CVs async though --no-async is specified
Product: Red Hat Satellite Reporter: Lukas Pramuk <lpramuk>
Component: TransitionsAssignee: Adam Price <adprice>
Status: CLOSED WONTFIX QA Contact: Lukas Pramuk <lpramuk>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: UnspecifiedCC: bbuckingham, bkearney, lpramuk, rplevka
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-13 20:48:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
production.log none

Description Lukas Pramuk 2015-09-16 10:58:54 UTC 
Description of problem:
hammer import repository-enable publishes CVs async though --no-async is specified.
If you specify --no-async option for Red Hat repository enablement, only repository syncing is done synchronously and following CV publishing is still done asynchronously.

So, successive hammer import repository-enable runs fail with 500 ISE and you cannot simply prevent the conflict errors by using --no-async option.


Version-Release number of selected component (if applicable):
rubygem-hammer_cli_import-0.10.20-1.el7sat.noarch

How reproducible:
100%

Steps to Reproduce:

1. Import an organization and provide it with a manifest:

# cat users.csv 
organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active
1,Imported Organization,1,admin,Admin,Admin,,root@localhost,,,,enabled

# hammer import organization --csv-file users.csv 
Summary
  Created 1 organization.

# hammer subscription upload --file $manifest --organization 'Imported Organization'

2. Run no-async import repository enablement preferably for a big/base channel

# cat channels.csv
channel_id,channel_label,channel_name,number_of_packages,org_id
103,rhel-x86_64-server-7,Red Hat Enterprise Linux Server (v. 7 for 64-bit x86_64),0,

# hammer import repository-enable --csv-file channels.csv --synchronize --wait --no-async
Summary
  Mapped 1 redhat repository.
  Created 1 redhat content view.

3. Run no-async  rebository enablement instantly once again 
# hammer import repository-enable --csv-file channels.csv --synchronize --wait --no-async
Publishing of redhat content view [36] failed with RestClient::InternalServerError: 500 Internal Server Error
Summary
  No action taken.

500 ISE is simply a conflict:
Required lock is already taken by other running tasks.
Please inspect their state, fix their errors and resume them.

Required lock: read
Conflicts with tasks:
- https://<SAT>/foreman_tasks/tasks/95b1e319-85ea-425c-ac02-4efd57f82567

Actual results:
hammer import repository-enable --no-async run ends, however CV publishing is still ongoing (causing conflicts)

Expected results:
when hammer import repository-enable --no-async run ends, CV publishing has been finished. (preventing possible conflicts)
Comment 1 Lukas Pramuk 2015-09-16 11:25:15 UTC 
I should have used verbose mode to see what was going on:

# hammer import repository-enable --verbose --csv-file channels.csv --synchronize --no-async
Importing from channels.csv
Repository 'Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server' already enabled as 134.
Redhat content view [1103->36] already imported.
Summary
  Found 1 redhat content view.
  No action taken.

<----1st run has started publishing and ended

# hammer import repository-enable --verbose --csv-file channels.csv --synchronize --no-async
Importing from channels.csv
Repository 'Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server' already enabled as 134.
Redhat content view [1103->36] already imported.
Publishing of redhat content view [36] failed with RestClient::InternalServerError: 500 Internal Server Error
Summary
  Found 1 redhat content view.
  No action taken.
Comment 2 Lukas Pramuk 2015-09-16 11:28:32 UTC 
# tail /root/import.log

I, [2015-09-16T05:32:06.829620 #13932]  INFO -- : Importing from users.csv
I, [2015-09-16T05:32:06.830373 #13932]  INFO -- : Creating new organization: Imported Organization
I, [2015-09-16T05:32:09.796299 #13932]  INFO -- : Summary
I, [2015-09-16T05:32:09.796479 #13932]  INFO -- :   Created 1 organization.
I, [2015-09-16T06:14:48.258260 #16844]  INFO -- : Importing from channels.csv
I, [2015-09-16T06:16:09.085306 #16844]  INFO -- : Enabling /content/dist/rhel/server/7/7Server/x86_64/os/Packages for channel rhel-x86_64-server-7 in org 8
I, [2015-09-16T06:16:23.539577 #16844]  INFO -- : Mapping redhat repository [1103->134].
I, [2015-09-16T06:16:26.802383 #16844]  INFO -- : Waiting for uuids (non async): ["3ccf0544-9593-4114-b0b7-de61c20feec7"].
I, [2015-09-16T06:31:37.270348 #16844]  INFO -- : Creating new redhat content view: Red Hat Enterprise Linux Server (v. 7 for 64-bit x86_64)
I, [2015-09-16T06:32:38.740051 #16844]  INFO -- : Summary
I, [2015-09-16T06:32:38.740290 #16844]  INFO -- :   Mapped 1 redhat repository.
I, [2015-09-16T06:32:38.740387 #16844]  INFO -- :   Created 1 redhat content view.
I, [2015-09-16T06:33:20.405388 #21073]  INFO -- : Importing from channels.csv
I, [2015-09-16T06:34:48.416819 #21073]  INFO -- : Repository 'Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server' already enabled as 134.
I, [2015-09-16T06:34:49.241886 #21073]  INFO -- : Redhat content view [1103->36] already imported.
W, [2015-09-16T06:34:49.979998 #21073]  WARN -- : Publishing of redhat content view [36] failed with RestClient::InternalServerError: 500 Internal Server Error
I, [2015-09-16T06:35:30.169471 #21073]  INFO -- : Summary
I, [2015-09-16T06:35:30.169684 #21073]  INFO -- :   Found 1 redhat content view.
I, [2015-09-16T06:35:30.169792 #21073]  INFO -- :   No action taken.
Comment 3 Lukas Pramuk 2015-09-16 11:55:37 UTC 
# hammer -d ...

Redhat content view [1103->36] already imported.
[ INFO 2015-09-16 07:49:24 API] POST /katello/api/content_views/36/publish
[DEBUG 2015-09-16 07:49:24 API] Params: {}
[DEBUG 2015-09-16 07:49:24 API] Headers: {}
[DEBUG 2015-09-16 07:49:25 API] 500 Internal Server Error
{
            "errors" => [
        [0] "Required lock is already taken by other running tasks.\nPlease inspect their state, fix their errors and resume them.\n\nRequired lock: read\nConflicts with tasks:\n- https://<SAT>/foreman_tasks/tasks/ff5257c9-b0e1-4911-a004-89abe16a89f2"
    ],
    "displayMessage" => "Required lock is already taken by other running tasks.\nPlease inspect their state, fix their errors and resume them.\n\nRequired lock: read\nConflicts with tasks:\n- https://<SAT>/foreman_tasks/tasks/ff5257c9-b0e1-4911-a004-89abe16a89f2"
}
Publishing of redhat content view [36] failed with RestClient::InternalServerError: 500 Internal Server Error
Comment 4 Brad Buckingham 2015-10-14 19:44:07 UTC 
Lukas, can we get a foreman-debug that includes the failure scenario?  It appears that there is a locked task; therefore, may have been a failure prior.
Comment 5 Roman Plevka 2015-10-27 09:15:54 UTC 
Created attachment 1086759 [details]
production.log

the interesting part of production log - 
capturing 
 -importing of 3 organizations (ids 201-203)
 -importing of 3 repositories
 -importing of 3 content views
 -importing of 3 content hosts
Comment 7 Bryan Kearney 2016-07-26 19:11:18 UTC 
Moving 6.2 bugs out to sat-backlog.
Comment 10 Bryan Kearney 2017-01-13 20:48:33 UTC 
This is an older bug which I do not envision being addressed in the near term. I am closing this out. If you believe doing so is an issue, please feel free to re-open and provide additional business information. Thank you.