Hide Forgot
+++ This bug was initially created as a clone of Bug #1123445 +++ Description of problem: When using puppet module sync on a puppet repo created using pulp-puppet-module-builder, puppet modules are imported with a name of 'author-name' instead of just 'name' The same issue does not occur when using hammer or the webUI to upload the modules Version-Release number of selected component (if applicable): pulp-server-2.4.0-0.23.beta.el6sat.noarch How reproducible: Always Steps to Reproduce: 1. Download a puppet module such as puppetlabs-concat from puppet forge 2. Extract the module in a directory /modules 3. cd /modules && pulp-puppet-module-builder 4. chmod 755 /modules 5. Create a puppet repo with url of file:///modules 6. Sync the repo Actual results: The puppet module is imported with a name of 'puppetlabs-concat'. The module is viewable via content search or the 'content view puppet modules' page Expected results: The puppet module is imported with a name of 'concat' Additional info: --- Additional comment from Justin Sherrill on 2014-07-25 12:22:43 EDT --- patch from a customer to fix the issue. (Note it was sent in email, so spacing is likely off) diff -urN /usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers.old/directory.py /usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/directory.py --- /usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers.old/directory.py 2014-06-16 13:00:58.706453008 +0000 +++ /usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/directory.py 2014-06-16 13:01:10.871179975 +0000 @@ -271,6 +271,7 @@ if self.canceled: return [] puppet_manifest = self._extract_metadata(module_path) + puppet_manifest["name"]=puppet_manifest["name"].replace(puppet_manifest["author"]+"-","") module = Module.from_dict(puppet_manifest) if inventory.already_associated(module): continue
See also this commit in pulp_puppet that fixes this for manual uploaded modules. But the directory.py importer is not updated to use the new function: https://github.com/pulp/pulp_puppet/commit/58c075b0009eda30ab285e66a4fa34e095ff8e65
The following patch looks like to work also: --- directory.py.old 2014-07-28 13:15:58.205905361 +0000 +++ directory.py 2014-07-28 14:09:41.284902425 +0000 @@ -271,7 +271,7 @@ if self.canceled: return [] puppet_manifest = self._extract_metadata(module_path) - module = Module.from_dict(puppet_manifest) + module = Module.from_json(puppet_manifest) if inventory.already_associated(module): continue _LOG.info(IMPORT_MODULE % dict(mod=module_path))
https://github.com/pulp/pulp_puppet/pull/127
The above pull request also fixes the inability to import modules using the old Puppet naming style (authorname/modulename rather than authorname-modulename), which is still legal. To verify: 1. Download the following puppet modules. The first uses the new name style, and the second uses the old naming style: https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz https://forgeapi.puppetlabs.com/v3/files/jcline-test-0.1.0.tar.gz 2. Extract the modules into a directory <module-dir> 3. cd <module-dir> && pulp-puppet-module-builder 4. chmod 755 <module-dir> 5. Create a puppet repo with url of file://<absolute-path-to-module-dir> 6. Sync the repo 7. List the Puppet modules in the repo using 'pulp-admin puppet repo modules --repo-id <repo-name>' 8. The first module should have the name 'concat' and the author 'puppetlabs' 9. The second module should have the name 'test' and the author 'jcline'
This was fixed in the 2.4.1-0.1.alpha build.
verified [root@cloud-qe-1-vm-2 module]# rpm -qa pulp-server pulp-server-2.4.1-0.1.alpha.el7.noarch [root@cloud-qe-1-vm-2 module]# [root@cloud-qe-1-vm-2 ~]# mkdir /tmp/module/ [root@cloud-qe-1-vm-2 ~]# [root@cloud-qe-1-vm-2 ~]# [root@cloud-qe-1-vm-2 ~]# [root@cloud-qe-1-vm-2 ~]# cd /tmp/module/ [root@cloud-qe-1-vm-2 module]# wget https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz --2014-08-18 09:49:27-- https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz Resolving forgeapi.puppetlabs.com (forgeapi.puppetlabs.com)... 23.236.57.230 Connecting to forgeapi.puppetlabs.com (forgeapi.puppetlabs.com)|23.236.57.230|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 17588 (17K) [application/octet-stream] Saving to: ‘puppetlabs-concat-1.0.4.tar.gz’ 100%[======================================>] 17,588 --.-K/s in 0s 2014-08-18 09:49:27 (124 MB/s) - ‘puppetlabs-concat-1.0.4.tar.gz’ saved [17588/17588] [root@cloud-qe-1-vm-2 module]# wget https://forgeapi.puppetlabs.com/v3/files/jcline-test-0.1.0.tar.gz --2014-08-18 09:49:35-- https://forgeapi.puppetlabs.com/v3/files/jcline-test-0.1.0.tar.gz Resolving forgeapi.puppetlabs.com (forgeapi.puppetlabs.com)... 23.236.57.230 Connecting to forgeapi.puppetlabs.com (forgeapi.puppetlabs.com)|23.236.57.230|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1869 (1.8K) [application/octet-stream] Saving to: ‘jcline-test-0.1.0.tar.gz’ 100%[======================================>] 1,869 --.-K/s in 0s 2014-08-18 09:49:36 (295 MB/s) - ‘jcline-test-0.1.0.tar.gz’ saved [1869/1869] [root@cloud-qe-1-vm-2 module]# [root@cloud-qe-1-vm-2 module]# [root@cloud-qe-1-vm-2 module]# pulp-puppet-module-builder cd /tmp/module git status find . -name init.pp cd /tmp/module cd /tmp/module cd /tmp/module [root@cloud-qe-1-vm-2 module]# pulp-admin puppet repo create --repo-id puppet --feed file://tmp/module/ Successfully created repository [puppet] [root@cloud-qe-1-vm-2 module]# [root@cloud-qe-1-vm-2 module]# [root@cloud-qe-1-vm-2 module]# pulp-admin puppet repo sync run --repo-id puppet +----------------------------------------------------------------------+ Synchronizing Repository [puppet] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [==================================================] 100% Metadata Query: 1/1 items ... completed Downloading new modules... [ ] 0% Module: 0/2 items Task Succeeded Publishing modules... [==================================================] 100% Module: 2/2 items ... completed Generating repository metadata... [-] ... completed Publishing repository over HTTP... ... completed Publishing repository over HTTPS... ... skipped Task Succeeded [root@cloud-qe-1-vm-2 module]# pulp-admin puppet repo modules --repo-id puppet Name: concat Version: 1.0.4 Author: puppetlabs Checksum: None Checksum Type: sha256 Dependencies: Description: Concat module License: Apache 2.0 Project Page: http://github.com/puppetlabs/puppetlabs-concat Source: git://github.com/puppetlabs/puppetlabs-concat.git Summary: Concat module Tag List: None Name: test Version: 0.1.0 Author: jcline Checksum: None Checksum Type: sha256 Dependencies: Description: UNKNOWN License: Apache License, Version 2.0 Project Page: UNKNOWN Source: UNKNOWN Summary: UNKNOWN Tag List: None [root@cloud-qe-1-vm-2 module]#
We are rebasing pulp-2.4.1, and so all of these bugs can no longer be considered VERIFIED. We will need to recheck them against the rebased build to make sure the cherry picking strategy did not introduce any regressions.
I have cherry picked the commit that fixed this bug[0] into the 2.4.1-dev branch. The commit applied cleanly, and all unit tests pass. [0] https://github.com/pulp/pulp_puppet/commit/891709e5898a26d9f4f02828ff122f7600f6e35c#diff-d41d8cd98f00b204e9800998ecf8427e
This was fixed in 2.4.1-0.2.alpha, available in Pulp's testing repository.
verified root@qe-blade-08 module]# rpm -qa pulp-server pulp-server-2.4.1-0.2.alpha.el6.noarch [root@qe-blade-13 module]# rpm -qa pulp-server pulp-server-2.4.1-0.2.alpha.el7.noarch [root@qe-blade-13 module]# [root@qe-blade-08 module]# [root@qe-blade-08 module]# wget https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz --no-check-certificate --2014-08-20 22:00:46-- https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz Resolving forgeapi.puppetlabs.com... 23.236.57.230 Connecting to forgeapi.puppetlabs.com|23.236.57.230|:443... connected. WARNING: certificate common name “forge.puppetlabs.com” doesn’t match requested host name “forgeapi.puppetlabs.com”. HTTP request sent, awaiting response... 200 OK Length: 17588 (17K) [application/octet-stream] Saving to: “puppetlabs-concat-1.0.4.tar.gz” 100%[======================================>] 17,588 --.-K/s in 0.07s 2014-08-20 22:00:47 (255 KB/s) - “puppetlabs-concat-1.0.4.tar.gz” saved [17588/17588] [root@qe-blade-08 module]# wget https://forgeapi.puppetlabs.com/v3/files/jcline-test-0.1.0.tar.gz --no-check-certificate --2014-08-20 22:00:55-- https://forgeapi.puppetlabs.com/v3/files/jcline-test-0.1.0.tar.gz Resolving forgeapi.puppetlabs.com... 23.236.57.230 Connecting to forgeapi.puppetlabs.com|23.236.57.230|:443... connected. WARNING: certificate common name “forge.puppetlabs.com” doesn’t match requested host name “forgeapi.puppetlabs.com”. HTTP request sent, awaiting response... 200 OK Length: 1869 (1.8K) [application/octet-stream] Saving to: “jcline-test-0.1.0.tar.gz” 100%[======================================>] 1,869 --.-K/s in 0s 2014-08-20 22:00:55 (327 MB/s) - “jcline-test-0.1.0.tar.gz” saved [1869/1869] [root@qe-blade-08 module]# ls jcline-test-0.1.0.tar.gz puppetlabs-concat-1.0.4.tar.gz [root@qe-blade-08 module]# yum install pulp-puppet-tools Loaded plugins: product-id, pulp-profile-update, security, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package pulp-puppet-tools.noarch 0:2.4.1-0.2.alpha.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: pulp-puppet-tools noarch 2.4.1-0.2.alpha.el6 pulp-v2-testing 31 k Transaction Summary ================================================================================ Install 1 Package(s) Total download size: 31 k Installed size: 42 k Is this ok [y/N]: y Downloading Packages: pulp-puppet-tools-2.4.1-0.2.alpha.el6.noarch.rpm | 31 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : pulp-puppet-tools-2.4.1-0.2.alpha.el6.noarch 1/1 Verifying : pulp-puppet-tools-2.4.1-0.2.alpha.el6.noarch 1/1 Installed: pulp-puppet-tools.noarch 0:2.4.1-0.2.alpha.el6 Complete! [root@qe-blade-08 module]# [root@qe-blade-08 module]# [root@qe-blade-08 module]# ls jcline-test-0.1.0.tar.gz puppetlabs-concat-1.0.4.tar.gz [root@qe-blade-08 module]# pulp-puppet-module-builder cd /tmp/module git status find . -name init.pp cd /tmp/module cd /tmp/module cd /tmp/module [root@qe-blade-08 module]# [root@qe-blade-08 module]# pulp-admin puppet repo create --repo-id puppet --feed file://tmp/module/ Successfully created repository [puppet] [root@qe-blade-08 module]# pulp-admin puppet repo sync run --repo-id puppet +----------------------------------------------------------------------+ Synchronizing Repository [puppet] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [==================================================] 100% Metadata Query: 1/1 items ... completed Downloading new modules... [ ] 0% Module: 0/2 items Task Succeeded Publishing modules... [==================================================] 100% Module: 2/2 items ... completed Generating repository metadata... [-] ... completed Publishing repository over HTTP... ... completed Publishing repository over HTTPS... ... skipped Task Succeeded [root@qe-blade-08 module]# pulp-admin puppet repo modules --repo-id puppet Name: concat Version: 1.0.4 Author: puppetlabs Checksum: None Checksum Type: sha256 Dependencies: Description: Concat module License: Apache 2.0 Project Page: http://github.com/puppetlabs/puppetlabs-concat Source: git://github.com/puppetlabs/puppetlabs-concat.git Summary: Concat module Tag List: None Name: test Version: 0.1.0 Author: jcline Checksum: None Checksum Type: sha256 Dependencies: Description: UNKNOWN License: Apache License, Version 2.0 Project Page: UNKNOWN Source: UNKNOWN Summary: UNKNOWN Tag List: None [root@qe-blade-08 module]#
Bug can be closed, confirmed that it works in Sat6 GA - 6.0.4
This is fixed in Pulp-2.4.1-1.