Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1175493 - pulp puppet directory import does not delete missing files
Summary: pulp puppet directory import does not delete missing files
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: Unspecified
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: jcallaha
URL:
Whiteboard:
Depends On: 1131146 1145719
Blocks: sat6-pulp-future 1122832 1131148 1175448
TreeView+ depends on / blocked
 
Reported: 2014-12-17 20:36 UTC by Michael Hrivnak
Modified: 2021-04-06 18:03 UTC (History)
17 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 1131146
Environment:
Last Closed: 2016-07-27 09:12:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 645 0 High CLOSED - CURRENTRELEASE pulp puppet directory import does not delete missing files Never
Red Hat Product Errata RHBA-2016:1501 0 normal SHIPPED_LIVE Red Hat Satellite 6.2 Capsule and Server 2016-07-27 12:28:58 UTC

Comment 1 Dennis Kliban 2015-01-08 18:54:09 UTC
I was not able to reproduce the bug.  Here are the approaches I took:

I added the following to the puppet_importer.json

{
 "remove_missing": "True"
}

Then I ran 3 commands:

pulp-admin puppet repo create --repo-id=repo1 --description="Mirror of Puppet Forge" --display-name="Repo 1" --feed=http://forge.puppetlabs.com

pulp-admin puppet repo update --repo-id=repo1 --queries=libvirt

pulp-admin puppet repo sync run --repo-id=repo1

The three commands above created a repository with 8 modules.  Then I ran the following commands:

pulp-admin puppet repo update --repo-id=repo1 --queries=libvirttttttt
pulp-admin puppet repo sync run --repo-id=repo1

This produced an empty repository because nothing matched the query.  I didn't see any tracebacks in the logs. 

I then attempted to reproduce this bug without modifying the --querries attribute of the repo.  

pulp-admin puppet repo create --repo-id=repo1 --description="Mirror of Puppet Forge" --display-name="Repo 1" --feed=http://forge.puppetlabs.com

pulp-admin puppet repo update --repo-id=repo1 --queries=libvirt

pulp-admin puppet repo sync run --repo-id=repo1

pulp-admin puppet repo create --repo-id=repo1 --description="Mirror of repo1" --display-name="Repo 2" --feed=http://localhost/pulp/puppet/repo1

pulp-admin puppet repo sync run --repo-id=repo2

After running the above commands I had 2 repositories with 8 modules. Then I ran the following 3 commands:

pulp-admin puppet repo update --repo-id=repo1 --queries=libvirttttttt
pulp-admin puppet repo sync run --repo-id=repo1
pulp-admin puppet repo sync run --repo-id=repo2

This produced two repos that were empty.  Once again - I didn't see any tracebacks.

I also tried manually removing modules from a repository and then synced from it without seeing this bug.

Comment 2 Preethi Thomas 2015-01-30 16:37:36 UTC
verified

[root@cloud-qe-4 modules]# rpm -qa pulp-server
pulp-server-2.6.0-0.5.beta.el7.noarch
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# rpm -qa |grep puppet
pulp-puppet-admin-extensions-2.6.0-0.5.beta.el7.noarch
pulp-puppet-tools-2.6.0-0.5.beta.el7.noarch
python-pulp-puppet-common-2.6.0-0.5.beta.el7.noarch
pulp-puppet-plugins-2.6.0-0.5.beta.el7.noarch
puppet-3.6.2-3.el7.noarch
[root@cloud-qe-4 modules]# 

Here are the steps I followed

1. Enable remove_missing in puppet_importer.json
$ sudo mkdir -p /opt/puppet/modules
$ sudo chmod -R 777 /opt/puppet
$ cd /opt/puppet
$ pulp-puppet-module-builder --url=https://github.com/puppetlabs/puppetlabs-xinetd -o ../modules

$ pulp-puppet-module-builder --working-dir=/opt/puppet --output-dir=/opt/puppet/modules --clean --url=https://github.com/puppetlabs/puppetlabs-mcollective.git -t 2.0.0

$pulp-admin puppet repo sync run --repo-id=puppet-builds

Now I removed the puppetlabs-mcollective-2.0.0.tar.gz

$ rm -rf puppetlabs-mcollective-2.0.0.tar.gz

Also removed it from the PULP_MANIFEST

Now ran pulp-admin puppet repo sync run --repo-id=puppet-builds

listing puppet repo shows that the module is removed from the repo





[root@cloud-qe-4 modules]# ls
PULP_MANIFEST  puppetlabs-inifile-1.1.3.tar.gz  puppetlabs-xinetd-1.4.0.tar.gz  reidmv-mco_profile-0.1.0.tar.gz
[root@cloud-qe-4 modules]# vi PULP_MANIFEST 
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# rm -rf reidmv-mco_profile-0.1.0.tar.gz
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]# pulp-admin puppet repo sync run --repo-id=puppet-builds
+----------------------------------------------------------------------+
                Synchronizing Repository [puppet-builds]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[==================================================] 100%
Metadata Query: 1/1 items
... completed

Downloading new modules...
[==================================================] 100%
Module: 0/0 items
... completed


Task Succeeded



Publishing modules...
[==================================================] 100%
Module: 2/2 items
... completed

Generating repository metadata...
[-]
... completed

Publishing repository over HTTP...
... completed

Publishing repository over HTTPS...
... skipped

Task Succeeded


[root@cloud-qe-4 modules]# pulp-admin puppet repo list
+----------------------------------------------------------------------+
                          Puppet Repositories
+----------------------------------------------------------------------+

Id:                  repo1
Display Name:        Repo 1
Description:         Mirror of Puppet Forge
Content Unit Counts: 
  Puppet Module: 1835

Id:                  puppet-builds
Display Name:        puppet-builds
Description:         None
Content Unit Counts: 
  Puppet Module: 2


[root@cloud-qe-4 modules]# rpm -qa pulp-server
pulp-server-2.6.0-0.5.beta.el7.noarch
[root@cloud-qe-4 modules]# 
[root@cloud-qe-4 modules]#

Comment 3 Brian Bouterse 2015-02-28 23:16:55 UTC
Moved to https://pulp.plan.io/issues/645

Comment 4 RHEL Program Management 2015-03-04 11:24:00 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 6 pulp-infra@redhat.com 2015-04-23 16:39:55 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 10 jcallaha 2016-05-20 19:41:02 UTC
Verified in Satellite 6.2 Beta Snap 12.

This functionality is now working correctly when manually managing uploaded puppet modules in a repo. Additionally, I feel it would be very beneficial to be able to filter puppet repositories in Satellite like you can with pulp-admin.

pulp-admin puppet repo update --repo-id=repo1 --queries=libvirt

Comment 12 errata-xmlrpc 2016-07-27 09:12:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501


Note You need to log in before you can comment on or make changes to this bug.