Bug 1103311 - [RFE] Have pulp parse the unit_key data from the module metadata if it's not passed in for uploaded puppet modules
Summary: [RFE] Have pulp parse the unit_key data from the module metadata if it's not ...
Alias: None
Product: Pulp
Classification: Retired
Component: puppet-support   
(Show other bugs)
Version: 2.4 Beta
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 2.4.0
Assignee: David Davis
QA Contact: Preethi Thomas
Keywords: Triaged
Depends On:
Blocks: 950743 1097209
TreeView+ depends on / blocked
Reported: 2014-05-30 17:18 UTC by David Davis
Modified: 2014-08-09 06:56 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-08-09 06:56:37 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description David Davis 2014-05-30 17:18:57 UTC
Katello (the webserver part) basically passes the upload bits calls from the CLI straight through to Pulp so we never see the full file. We could also store a copy of the file but it would probably just be easier to have pulp use the metadata from the module to populate the unit_key data. This is what we're doing in Katello already. Having this small code change would simplify content uploads in Katello a lot.

Comment 1 David Davis 2014-05-30 17:19:24 UTC
Here's the patch I am proposing:


Comment 2 David Davis 2014-05-30 17:25:02 UTC

Comment 3 Michael Hrivnak 2014-06-02 13:10:02 UTC
This is worth doing in pulp. Having glanced at the existing code, the workflow is a bit odd. I think this RFE can be accomplished with relatively little change, as the above PR demonstrates.

The only reservation I have about the above PR is that it allows instantiation of a model object with no data. I think we can re-work the process slightly to avoid that.

Comment 4 David Davis 2014-06-05 12:12:29 UTC

Comment 5 David Davis 2014-06-05 12:13:33 UTC
Commit info:


1103311 - Extract any unit_key fields from metadata
If no unit_key metadata is passed in for uploaded puppet modules, just use the
module metadata to populate name, author, and version.


Comment 6 Michael Hrivnak 2014-06-05 13:03:04 UTC

Comment 7 Jeff Ortel 2014-06-11 01:33:07 UTC
build: 2.4.0-0.20.beta

Comment 8 Preethi Thomas 2014-06-30 17:49:06 UTC
could you please include the steps on how to verify this bug

Comment 9 David Davis 2014-07-10 18:37:32 UTC
This might be really hard to test without scripting something as it requires:

1) Directly hitting the API
2) Reading in a file and sending its contents

That said, Katello is currently using this feature and it's been tested as part of https://bugzilla.redhat.com/show_bug.cgi?id=1097209.

Comment 10 Preethi Thomas 2014-07-11 15:11:12 UTC

[root@yttrium ~]# rpm -qa pulp-server

[root@yttrium ~]#  curl -H "Accept: application/json" -X PUT -k -u admin:admin  --data-binary @puppetlabs-java-1.1.1.tar.gz  'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/content/uploads/41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/0/'
null[root@yttrium ~]# 

[root@yttrium ~]# curl -H "Accept: application/json" -X POST -k -u admin:admin -d '{"unit_type_id": "puppet_module", "upload_id": "41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/", "unit_key": {"version": "1.1.1", "name": "java", "author": "puppetlabs"}, "unit_metadata": {}}' 'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/repositories/repo/actions/import_upload/'
{"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/", "task_id": "0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d"}], "result": null, "error": null}[root@yttrium ~]# 
[root@yttrium ~]# 

[root@yttrium ~]# curl -H "Accept: application/json" -X GET -k -u admin:admin -d '{"upload_id":"41a2ecd6-ca45-4ba5-97a1-3f287e7b751a","unit_type_id":"puppet_module","unit_key":{"author":"puppetlabs","name":"java","version":"1.1.1"}}' 'https://yttrium.idm.lab.bos.redhat.com/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/' |python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
128   749  107   749    0   150   5714   1144 --:--:-- --:--:-- --:--:--  7216
    "_href": "/pulp/api/v2/tasks/0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d/", 
    "_id": {
        "$oid": "53bff3d35e162a0c2626fab1"
    "_ns": "task_status", 
    "error": null, 
    "exception": null, 
    "finish_time": "2014-07-11T14:25:23Z", 
    "id": "53bff3d37bcdb34d0750c025", 
    "progress_report": {}, 
    "queue": "reserved_resource_worker-3@yttrium.idm.lab.bos.redhat.com.dq", 
    "result": {
        "details": {}, 
        "success_flag": false, 
        "summary": "/var/lib/pulp/uploads/41a2ecd6-ca45-4ba5-97a1-3f287e7b751a/"
    "spawned_tasks": [], 
    "start_time": "2014-07-11T14:25:23Z", 
    "state": "finished", 
    "tags": [
    "task_id": "0cb1bf50-4c2b-4f5e-881f-fb3e5c11cf4d", 
    "task_type": "pulp.server.managers.content.upload.import_uploaded_unit", 
    "traceback": null

Comment 11 Randy Barlow 2014-08-09 06:56:37 UTC
This has been fixed in Pulp 2.4.0-1.

Note You need to log in before you can comment on or make changes to this bug.