Bug 946966

Summary: Pulp doesn't support puppet module versioning puppet does.
Product: [Retired] Pulp Reporter: rush.ru
Component: puppet-supportAssignee: Michael Hrivnak <mhrivnak>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: high Docs Contact:
Priority: medium    
Version: 2.1 BetaCC: chrungoo.abhay, james.defelice, jason.dobies, rbarlow
Target Milestone: ---Keywords: Triaged
Target Release: 2.3.0   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-09 14:30:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description rush.ru 2013-04-01 04:47:55 UTC
Description of problem:

According to puppet labs (http://docs.puppetlabs.com/puppet/2.7/reference/modules_publishing.html#write-a-modulefile ), puppet modules are using semantic version ( http://semver.org/spec/v1.0.0.html ). This specification allows modules to use dash in version number (i.e. 1.0.1-build22). When one tries to upload module with such version number to pulp repository (pulp-admin puppet repo uploads upload --repo-id puppet -f /root/author-mod-8.1.0-SNAPSHOT.tar.gz ), error is occured:
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/client/extensions/core.py", line 478, in run
exit_code = Cli.run(self, args)
File "/usr/lib/python2.6/site-packages/okaara/cli.py", line 966, in run
exit_code = command_or_section.execute(self.prompt, remaining_args)
File "/usr/lib/python2.6/site-packages/pulp/client/extensions/extensions.py", line 224, in execute
return self.method(*arg_list, **clean_kwargs)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/upload.py", line 163, in run
unit_key, unit_metadata = self.generate_unit_key_and_metadata(filename, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/upload.py", line 279, in generate_unit_key_and_metadata
unit_key = self.generate_unit_key(filename, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp_puppet/extensions/admin/repo/upload.py", line 45, in generate_unit_key
author, name, version = root_filename.split('-')
ValueError: too many values to unpack

That's because dash is used as separator to find author, name and version from filename without above in mind.

Comment 1 Michael Hrivnak 2013-04-01 13:52:31 UTC
It should be noted that the OP graciously submitted a pull request for this: https://github.com/pulp/pulp_puppet/pull/47

It will require a bit of testing before being merged, but should be a fairly simple enhancement.

Comment 2 Michael Hrivnak 2013-05-15 21:04:36 UTC
Solving this with a slightly different change than the OP suggested, plus tests to confirm.

https://github.com/pulp/pulp_puppet/pull/52

Comment 3 Jeff Ortel 2013-06-06 16:20:03 UTC
build: 2.2.0-0.2.beta.

Some of these were likely included in an earlier build but found them this time with a better BZ query.

Comment 4 Preethi Thomas 2013-06-07 21:07:27 UTC
verified

[root@hp-dl360g6-01 ~]# rpm -q pulp-server
pulp-server-2.2.0-0.2.beta.fc18.noarch
[root@hp-dl360g6-01 ~]# 

[root@hp-dl360g6-01 ~]# pulp-admin puppet repo uploads upload  --repo-id bz946966 --file puppetlabs-apache-0.5.0-rc1.tar.gz -v
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: puppetlabs-apache-0.5.0-rc1.tar.gz
... completed

Files to be uploaded:
  puppetlabs-apache-0.5.0-rc1.tar.gz

Creating upload requests on the server...
[==================================================] 100%
Initializing: puppetlabs-apache-0.5.0-rc1.tar.gz
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: puppetlabs-apache-0.5.0-rc1.tar.gz
[==================================================] 100%
28866/28866 bytes
... completed

Importing into the repository...
... completed

Deleting the upload request...
... completed

[root@hp-dl360g6-01 ~]# pulp-admin puppet repo modules --repo-id bz946966 
Name:         apache
Version:      0.5.0-rc1
Author:       puppetlabs
Dependencies: 
  Name:                puppetlabs/firewall
  Version Requirement: >= 0.0.4
  Name:                puppetlabs/stdlib
  Version Requirement: >= 2.2.1
Description:  Module for Apache configuration
License:      Apache 2.0
Project Page: https://github.com/puppetlabs/puppetlabs-apache
Source:       git://github.com/puppetlabs/puppetlabs-apache.git
Summary:      Puppet module for Apache
Tag List:     None
Types:        
  Doc:        Manage Apache 2 modules
  Name:       a2mod
  Parameters: 
    Doc:  The name of the module to be managed
    Name: name
    Doc:  The name of the .so library to be loaded
    Name: lib
    Doc:  Module identifier string used by LoadModule. Default:
          module-name_module
    Name: identifier
  Properties: 
    Doc:  The basic property that the resource should be in.  Valid values are
          `present`, `absent`.
    Name: ensure
  Providers:  
    Doc:  Manage Apache 2 modules on Debian and Ubuntu  Required binaries:
          `a2enmod`, `a2dismod`.  Default for `operatingsystem` == `debian,
          ubuntu`.
    Name: a2mod
    Doc:  Manage Apache 2 modules on Gentoo  Default for `operatingsystem` ==
          `gentoo`.
    Name: gentoo
    Doc:  Dummy provider for A2mod.      Fake nil resources when there is no
          crontab binary available. Allows     puppetd to run on a bootstrapped
          machine before a Cron package has been     installed. Workaround for:
          http://projects.puppetlabs.com/issues/2384     
    Name: modfix
    Doc:  Manage Apache 2 modules on RedHat family OSs  Default for `osfamily`
          == `redhat`.
    Name: redhat

Comment 5 Abhay Chrungoo 2013-08-24 01:11:02 UTC
While upload works, download is still an issue

[root@puppetrepo modules]# rpm -q pulp-server
pulp-server-2.2.0-0.25.beta.el6.noarch

[root@puppet-master modules]# puppet module install server-oasis
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Error: Could not execute operation for 'server/oasis'
  The server being queried was http://.:puppetforge@puppetrepo.server.com
  The HTTP response we received was '500 Internal Server Error'
    Check the author and module names are correct.
[root@puppet-master modules]# puppet module install server/oasis -v 0.0.3-20130823150537
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Notice: Installing -- do not interrupt ...
/etc/puppet/modules
âââ server-oasis (v0.0.3-20130823150537)
[root@puppet-master modules]# puppet module upgrade server/oasis
Notice: Preparing to upgrade 'server-oasis' ...
Notice: Found 'server-oasis' (v0.0.3-20130823150537) in /etc/puppet/modules ...
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Error: Could not upgrade module 'server-oasis' (v0.0.3-20130823150537 -> latest)
  Module 'server-oasis' does not exist on http://.:puppetforge@puppetrepo.server.com
[root@puppet-master modules]# puppet module upgrade server/oasis -v 0.0.5-20130823172126
Error: puppet module upgrade takes 1 argument, but you gave 2
Error: Try 'puppet help module upgrade' for usage
[root@puppet-master modules]# puppet module install  server/oasis -v 0.0.5-20130823172126
Notice: Preparing to install into /etc/puppet/modules ...
Error: Could not install module 'server-oasis' (v0.0.5-20130823172126)
  Module 'server-oasis' (v0.0.3-20130823150537) is already installed
    Use `puppet module upgrade` to install a different version
    Use `puppet module install --force` to re-install only this module
[root@puppet-master modules]# puppet module install  server/oasis -v 0.0.5-20130823172126 --force
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Notice: Installing -- do not interrupt ...
/etc/puppet/modules
âââ server-oasis (v0.0.5-20130823172126)

Comment 6 James DeFelice 2013-08-28 15:22:10 UTC
Resolved problem from comment 5, created pull request:

https://github.com/pulp/pulp_puppet/pull/64

[root@project2 modules]# puppet module install companyname-project --module_repository http://.:project-snapshots-puppet@node251.mgmt.dev
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from http://.:project-snapshots-puppet@node251.mgmt.dev ...
Notice: Installing -- do not interrupt ...
/etc/puppet/modules
└── companyname-project (v0.0.6-20130827151152)

Comment 7 Abhay Chrungoo 2013-08-28 19:46:30 UTC
Dang! you're fast. ;) 
https://github.com/pulp/pulp_puppet/pull/65

PUBLISH puppet-test-0.0.2-rc1

[root@puppet-master2 tmp]# puppet module install puppet/test --target-dir=/tmp/mods
Notice: Preparing to install into /tmp/mods ...
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Notice: Installing -- do not interrupt ...
/tmp/mods
âââ puppet-test (v0.0.2-rc1)

PUBLISH puppet-test-0.0.2-rc1-11

[root@puppet-master2 tmp]# rm -rf /tmp/mods
[root@puppet-master2 tmp]# puppet module install puppet/test --target-dir=/tmp/mods
Notice: Preparing to install into /tmp/mods ...
Notice: Created target directory /tmp/mods
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Notice: Installing -- do not interrupt ...
/tmp/mods
âââ puppet-test (v0.0.2-rc1-11)

PUBLISH puppet-test-0.0.2

[root@puppet-master2 tmp]# rm -rf /tmp/mods
[root@puppet-master2 tmp]# puppet module install puppet/test --target-dir=/tmp/mods
Notice: Preparing to install into /tmp/mods ...
Notice: Created target directory /tmp/mods
Notice: Downloading from http://.:puppetforge@puppetrepo.server.com ...
Notice: Installing -- do not interrupt ...
/tmp/mods
âââ puppet-test (v0.0.2)

Comment 8 Randy Barlow 2013-08-30 14:38:48 UTC
Thanks for the pull requests! I'm marking this back to assigned so one of us can take a look at them and get it merged. Unfortunately, we are a little too close to the 2.2 release for us to merge this fix for that one, so I'm also marking this ticket for 2.3.

Comment 9 Michael Hrivnak 2013-09-05 21:02:32 UTC
https://github.com/pulp/pulp_puppet/pull/68

Comment 10 Jeff Ortel 2013-09-12 22:35:36 UTC
build: 2.3.0-0.10.alpha

Comment 11 Preethi Thomas 2013-10-08 15:42:59 UTC
moving to verified

puppet module install puppetlabs-apache --version 0.5.0-rc1  --module_repository http://pulp-client:.@pulp-v2-server.usersys.redhat.com --ignore-dependencies
Notice: Preparing to install into /etc/puppet/modules ...
Notice: Downloading from http://pulp-client:.@pulp-v2-server.usersys.redhat.co

Comment 12 Preethi Thomas 2013-12-09 14:30:16 UTC
Pulp 2.3 released.