Bug 1103311 - [RFE] Have pulp parse the unit_key data from the module metadata if it's not passed in for uploaded puppet modules
[RFE] Have pulp parse the unit_key data from the module metadata if it's not ...
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: puppet-support (Show other bugs)
2.4 Beta
Unspecified Unspecified
medium Severity unspecified
: ---
: 2.4.0
Assigned To: David Davis
Preethi Thomas
: Triaged
Depends On:
Blocks: 950743 1097209
  Show dependency treegraph
 
Reported: 2014-05-30 13:18 EDT by David Davis
Modified: 2014-08-09 02:56 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-08-09 02:56:37 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description David Davis 2014-05-30 13:18:57 EDT
Katello (the webserver part) basically passes the upload bits calls from the CLI straight through to Pulp so we never see the full file. We could also store a copy of the file but it would probably just be easier to have pulp use the metadata from the module to populate the unit_key data. This is what we're doing in Katello already. Having this small code change would simplify content uploads in Katello a lot.
Comment 1 David Davis 2014-05-30 13:19:24 EDT
Here's the patch I am proposing:

https://github.com/daviddavis/pulp_puppet/commit/f06d1186c88df3f6ea9e27965827467b432293e0
Comment 2 David Davis 2014-05-30 13:25:02 EDT
https://github.com/pulp/pulp_puppet/pull/126
Comment 3 Michael Hrivnak 2014-06-02 09:10:02 EDT
This is worth doing in pulp. Having glanced at the existing code, the workflow is a bit odd. I think this RFE can be accomplished with relatively little change, as the above PR demonstrates.

The only reservation I have about the above PR is that it allows instantiation of a model object with no data. I think we can re-work the process slightly to avoid that.
Comment 4 David Davis 2014-06-05 08:12:29 EDT
https://github.com/pulp/pulp_puppet/pull/126
Comment 5 David Davis 2014-06-05 08:13:33 EDT
Commit info:

58c075b0009eda30ab285e66a4fa34e095ff8e65

1103311 - Extract any unit_key fields from metadata
If no unit_key metadata is passed in for uploaded puppet modules, just use the
module metadata to populate name, author, and version.

https://bugzilla.redhat.com/show_bug.cgi?id=1103311
Comment 6 Michael Hrivnak 2014-06-05 09:03:04 EDT
https://github.com/pulp/pulp_puppet/pull/126
Comment 7 Jeff Ortel 2014-06-10 21:33:07 EDT
build: 2.4.0-0.20.beta
Comment 8 Preethi Thomas 2014-06-30 13:49:06 EDT
could you please include the steps on how to verify this bug
Comment 9 David Davis 2014-07-10 14:37:32 EDT
This might be really hard to test without scripting something as it requires:

1) Directly hitting the API
2) Reading in a file and sending its contents

That said, Katello is currently using this feature and it's been tested as part of https://bugzilla.redhat.com/show_bug.cgi?id=1097209.
Comment 10 Preethi Thomas 2014-07-11 11:11:12 EDT
verified

[root@yttrium ~]# rpm -qa pulp-server
pulp-server-2.4.0-0.23.beta.el6.noarch

[root@yttrium ~]#  curl -H "Accept: application/json" -X PUT -k -u admin:admin  --data-binary @puppetlabs-java-1.1.1.tar.gz  'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/content/uploads/41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/0/'
null[root@yttrium ~]# 


[root@yttrium ~]# curl -H "Accept: application/json" -X POST -k -u admin:admin -d '{"unit_type_id": "puppet_module", "upload_id": "41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/", "unit_key": {"version": "1.1.1", "name": "java", "author": "puppetlabs"}, "unit_metadata": {}}' 'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/repositories/repo/actions/import_upload/'
{"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/", "task_id": "0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d"}], "result": null, "error": null}[root@yttrium ~]# 
[root@yttrium ~]# 


[root@yttrium ~]# curl -H "Accept: application/json" -X GET -k -u admin:admin -d '{"upload_id":"41a2ecd6-ca45-4ba5-97a1-3f287e7b751a","unit_type_id":"puppet_module","unit_key":{"author":"puppetlabs","name":"java","version":"1.1.1"}}' 'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/' |python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
128   749  107   749    0   150   5714   1144 --:--:-- --:--:-- --:--:--  7216
{
    "_href": "/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/", 
    "_id": {
        "$oid": "53bff3d35e162a0c2626fab1"
    }, 
    "_ns": "task_status", 
    "error": null, 
    "exception": null, 
    "finish_time": "2014-07-11T14:25:23Z", 
    "id": "53bff3d37bcdb34d0750c025", 
    "progress_report": {}, 
    "queue": "reserved_resource_worker-3@yttrium.idm.lab.bos.redhat.com.dq", 
    "result": {
        "details": {}, 
        "success_flag": false, 
        "summary": "/var/lib/pulp/uploads/41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/"
    }, 
    "spawned_tasks": [], 
    "start_time": "2014-07-11T14:25:23Z", 
    "state": "finished", 
    "tags": [
        "pulp:repository:repo", 
        "pulp:action:import_upload"
    ], 
    "task_id": "0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d", 
    "task_type": "pulp.server.managers.content.upload.import_uploaded_unit", 
    "traceback": null
}
Comment 11 Randy Barlow 2014-08-09 02:56:37 EDT
This has been fixed in Pulp 2.4.0-1.

Note You need to log in before you can comment on or make changes to this bug.