Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1430879 - Support verify content sync on bulk products
Summary: Support verify content sync on bulk products
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: 6.3.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Sachin Ghai
URL:
Whiteboard:
Depends On: 1223023 1427618
Blocks: 1530485 1686971
TreeView+ depends on / blocked
 
Reported: 2017-03-09 18:48 UTC by Justin Sherrill
Modified: 2019-04-01 20:27 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1427618
: 1530485 (view as bug list)
Environment:
Last Closed: 2018-02-21 16:54:37 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
advanced sync with 3 sync options on product page (97.93 KB, image/png)
2017-12-28 14:14 UTC, Sachin Ghai
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 18844 0 High Closed Support verify content sync on bulk products 2021-02-10 13:44:24 UTC

Comment 1 Justin Sherrill 2017-03-09 18:49:52 UTC
This is for adding UI/API/CLI for bulk product force and verify syncs

Comment 3 Satellite Program 2017-03-09 19:10:32 UTC
Upstream bug assigned to jsherril

Comment 4 Satellite Program 2017-03-16 20:11:01 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/18844 has been resolved.

Comment 5 Sachin Ghai 2017-12-28 14:13:22 UTC
Verified with sat6.3 snap30

I can see the option to verify content sync on bulk products using UI. 3 new options have been added on selecting "Advanced Sync" from bulk action drop down.

a) optimized sync b) complete sync c) validate content sync


Please see the attachment

Comment 6 Sachin Ghai 2017-12-28 14:14:31 UTC
Created attachment 1373244 [details]
advanced sync with 3 sync options on product page

Comment 7 Sachin Ghai 2017-12-28 14:16:52 UTC
Running "validate content sync" on on_demand sync policy yum repos throws error:

 RuntimeError: No syncable repositories found for selected products and options. 

This option is valid only for "immediate" and backround sync policy yum repos.

Comment 8 Sachin Ghai 2017-12-28 14:23:20 UTC
From UI, sync on bulk product is working fine though I feel we can improve on two things:

a) error message as mentioned in comment7 for "validate content sync" for on_demand sync policy yum repos is a bit misleading. I think we should update it like:

Validate content sync operation is not permitted for on_demand sync policy yum repos

I'll file another bug for it.

b) when we run sync on bulk products, as soon as we submit the form, UI should navigate from product page to sync status page or atleast unselect all selected repos. ( will file seperate issue)


I'll test the same through cli and api as well.

Comment 9 Sachin Ghai 2017-12-29 11:15:20 UTC
I can see following api call only (katello/api/products/40/sync) for bulk product sync.

 ~]# curl -i -k -u admin:changeme  -H "Content-Type: application/json" -X POST 'https://dell-pet410-04.khw.lab.eng.bos.redhat.com/katello/api/products/40/sync'
HTTP/1.1 202 Accepted
 {"id":"19acab7a-9847-45d5-85a4-1e45135e7904","label":"Actions::BulkAction","pending":true,"username":"admin","started_at":"2017-12-29 11:09:12 UTC","ended_at":null,"state":"planned","result":"pending","progress":0.0,"input":{"action_class":"Actions::Katello::Repository::Sync","target_ids":[5],"target_class":"Katello::Repository","args":[],"current_user_id":3},"output":{},"humanized":{"action":"Bulk action","input":null,"output":null,"errors":[]},"cli_example":null}


There is no hammer command for bulk product sync.

Subcommands:
 create                        Create a product
 delete                        Destroy a product
 info                          Show a product
 list                          List products
 remove-sync-plan              Delete assignment sync plan and product.
 set-sync-plan                 Assign sync plan to product.
 synchronize                   Sync all repositories for a product
 update                        Updates a product



@Justin: I can see UI changes around bulk products sync as stated above but no such cmds in hammer ?

bz description we should have support from CLI and api as well. whats expected here ? please suggest.

Comment 10 Justin Sherrill 2017-12-29 15:44:13 UTC
Sachin, 

The bulk product api is actually:

PUT /products/bulk/sync/

with a body of:

{
  "ids': [1,2,3],
  "validate_contents": true
}

The api you mentioned is the single product sync api.


To answer your question about the cli, you are correct, this is not in the cli currently.  In fact there are no bulk actions available via the cli for products at all (and i don't think they ever existed).  

My suggestion would be to open a new bug/clone this bug to create hammer commands for all bulk product actions (sync, delete, update sync plans).  We could fail_qa this bug, but i don't see this being done for 6.3 (but maybe the triage team will disagree).

Comment 11 Sachin Ghai 2018-01-02 11:18:01 UTC
Thank you Justin for your reply. I see the bulk product api and can sync multiple products w/ validate_content option.


 ~]#  curl -i -k -u admin:changeme  -H "Content-Type: application/json" -X PUT -d '{"ids": [40,41], "validate_contents": "true"}' 'https://SAT_SERVER/katello/api/products/bulk/sync'
HTTP/1.1 202 Accepted
Status: 202 Accepted
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8

  {"id":"7153215c-f441-4470-8823-e4683c97e6ed","label":"Actions::BulkAction","pending":true,"username":"admin","started_at":"2018-01-02 11:06:18 UTC","ended_at":null,"state":"planned","result":"pending","progress":0.0,"input":{"action_class":"Actions::Katello::Repository::Sync","target_ids":[5,6],"target_class":"Katello::Repository","args":[null,{"skip_metadata_check":false,"validate_contents":true}],"current_user_id":3},"output":{},"humanized":{"action":"Bulk action","input":null,"output":null,"errors":[]},"cli_example":null}



I'll open a new bug to add hammer commands for bulk product actions. thanks.

Comment 12 Sachin Ghai 2018-01-03 06:33:29 UTC
following bz is opened to add support in hammer for bulk product actions:

https://bugzilla.redhat.com/show_bug.cgi?id=1530485


For issue b) in comment8: https://bugzilla.redhat.com/show_bug.cgi?id=1529677

Comment 13 Sachin Ghai 2018-01-03 06:37:55 UTC
Bulk product sync through api using flag skip_metadata_check:

# curl -i -k -u admin:changeme  -H "Content-Type: application/json" -X PUT -d '{"ids": [40,41], "skip_metadata_check": "true"}' 'https://SAT_SERVER/katello/api/products/bulk/sync'
HTTP/1.1 202 Accepted
Date: Wed, 03 Jan 2018 06:30:07 GMT
----
Status: 202 Accepted
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8

  {"id":"3366646a-535d-4751-806c-29fffa68dab8","label":"Actions::BulkAction","pending":true,"username":"admin","started_at":"2018-01-03 06:30:07 UTC","ended_at":null,"state":"planned","result":"pending","progress":0.0,"input":{"action_class":"Actions::Katello::Repository::Sync","target_ids":[5,6],"target_class":"Katello::Repository","args":[null,{"skip_metadata_check":true,"validate_contents":false}],"current_user_id":3},"output":{},"humanized":{"action":"Bulk action","input":null,"output":null,"errors":[]},"cli_example":null}


from production.log:
==================

 [I] Started POST "/katello/api/v2/repositories/sync_complete?token=2SWS4odpCFAqfYmMgtaaBmVMP77NwWLK" for 10.16.185.162 at 2018-01-03 01:31:41 -0500
2018-01-03 01:31:41 3bf54494 [app] [I] Processing by Katello::Api::V2::RepositoriesController#sync_complete as */*
2018-01-03 01:31:41 3bf54494 [app] [I]   Parameters: {"call_report"=>"[FILTERED]", "event_type"=>"repo.sync.finish", "payload"=>{"importer_id"=>"yum_importer", "exception"=>nil, "repo_id"=>"21b3569b-cd00-4831-a102-3cd869d313c6", "traceback"=>nil, "started"=>"2018-01-03T06:30:15Z", "_ns"=>"repo_sync_results", "completed"=>"2018-01-03T06:31:39Z", "importer_type_id"=>"yum_importer", "error_message"=>nil, "summary"=>{"content"=>{"state"=>"FINISHED"}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"state"=>"FINISHED"}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "added_count"=>0, "result"=>"success", "updated_count"=>2, "details"=>{"content"=>{"size_total"=>0, "items_left"=>0, "items_total"=>0, "state"=>"FINISHED", "size_left"=>0, "details"=>{"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0}, "error_details"=>[]}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "id"=>"5a4c78cbef0f7a33c29812a1", "removed_count"=>0}, "token"=>"2SWS4odpCFAqfYmMgtaaBmVMP77NwWLK", "api_version"=>"v2", "repository"=>{}}

Comment 14 Sachin Ghai 2018-01-03 09:19:28 UTC
I corrupted a package and later resynced the repo from UI by selecting option "validate content sync" and package was resycned. I confirmed it through md5sum


[root@dell-pet410-04 fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8]# ll
total 64
-rw-r--r--. 1 apache apache 61552 Dec 22 07:29 libgudev1-147-2.46.el6.x86_64.rpm
[root@dell-pet410-04 fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8]# pwd
/var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8


Checked the md5sum before corrupting the package:
=====================================================
[root@dell-pet410-04 fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8]# md5sum /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm 
d0323018ad8675c16dd94232dcf719b2  /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm
 

corrupted the package
=======================
echo "FOO" >> /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm

md5sum changed after corrupting the package
===========================================

md5sum /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm
5fde6817f5ac9c40deba6fe385e436e3  /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm


resync the repo using validate content sync option and checked the md5sum and it shows the same md5sum which was set originally.

# md5sum /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm
d0323018ad8675c16dd94232dcf719b2  /var/lib/pulp/content/units/rpm/fe/fe1f2946d57d9b26c9ea16e0b777ceb47287bd440e974909db071350ccd8e8/libgudev1-147-2.46.el6.x86_64.rpm

Comment 15 Satellite Program 2018-02-21 16:54:37 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> > 
> > For information on the advisory, and where to find the updated files, follow the link below.
> > 
> > If the solution does not work for you, open a new bug report.
> > 
> > https://access.redhat.com/errata/RHSA-2018:0336


Note You need to log in before you can comment on or make changes to this bug.