Bug 1128274
| Summary: | Puppet module sync progress incorrect for PULP_MANIFEST style repos | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Justin Sherrill <jsherril> |
| Component: | API | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Katello QA List <katello-qa-list> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.0.0 | CC: | bbuckingham, jcline, pthomas, skarmark |
| Target Milestone: | Unspecified | ||
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-11-24 21:33:42 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 950746 | ||
We need to make both of them consistent. https://github.com/pulp/pulp_puppet/pull/131 To verify: 1. Download https://forgeapi.puppetlabs.com/v3/files/puppetlabs-concat-1.0.4.tar.gz 2. Extract the above module to a directory <dir> 3. cd <dir> && pulp-puppet-module-builder 4. pulp-admin puppet repo create --repo-id dir_test --feed file:///path/to/<dir> 5. pulp-admin puppet repo sync run --repo-id dir_test 6. Check ~/.pulp/server_calls.log or curl the task to check the progress report. It should be in the state 'success', with a total count of 1 and a finished count of 1. 7. Repeat step 6. 8. Consult ~/.pulp/server_calls.log again. The total count should be 0, and the finished count is 0. You could also add some more modules are rerun pulp-puppet-module-builder to make sure the total count only includes new modules. build: 2.5.0-0.6.beta verified
[root@cloud-qe-15 ~]# rpm -qa pulp-server
pulp-server-2.5.0-0.6.beta.el7.noarch
[root@cloud-qe-15 ~]#
"progress_report": {
"puppet_importer": {
"modules": {
"error_message": null,
"execution_time": 0.007193088531494141,
"total_count": 1,
"traceback": null,
"individual_errors": [],
"state": "success",
"error_count": 0,
"error": "None",
"finished_count": 1
},
"metadata": {
"query_finished_count": 1,
"traceback": null,
"execution_time": 0.000804901123046875,
"query_total_count": 1,
"error_message": null,
"state": "success",
"error": "None",
"current_query": null
}
}
},
"queue": "reserved_resource_worker-1.lab.eng.bos.redhat.com.dq",
"state": "finished",
"result": {
"result": "success",
"importer_id": "puppet_importer",
"exception": null,
"repo_id": "puppet-builds",
"traceback": null,
"started": "2014-10-02T14:10:12Z",
"_ns": "repo_sync_results",
"completed": "2014-10-02T14:10:12Z",
"importer_type_id": "puppet_importer",
"error_message": null,
"summary": {
"total_execution_time": 0.007997989654541016
},
"added_count": 0,
"removed_count": 0,
"updated_count": 1,
"id": "542d5cc4dba9cb738fd58ac8",
"details": {
"finished_count": 1,
"total_count": 1,
"error_count": 0
}
},
"error": null,
"_id": {
"$oid": "542d5cc47ccf31edbac08bae"
},
"id": "542d5cc4dba9cb6e3014ee69"
}
2014-10-02 10:10:14,989 - INFO - GET request to /pulp/api/v2/tasks/6d9337f2-2015-442a-89c1-93619565fbd3/ with parameters None
2014-10-02 10:10:14,989 - INFO - Response status : 200
2014-10-02 10:10:14,990 - INFO - Response body :
{
"exception": null,
"task_type": "pulp.server.managers.repo.publish.publish",
"_href": "/pulp/api/v2/tasks/6d9337f2-2015-442a-89c1-93619565fbd3/",
"task_id": "6d9337f2-2015-442a-89c1-93619565fbd3",
"tags": [
"pulp:repository:puppet-builds",
"pulp:action:publish"
],
"finish_time": "2014-10-02T14:10:13Z",
"_ns": "task_status",
"start_time": "2014-10-02T14:10:13Z",
"traceback": null,
"spawned_tasks": [],
"progress_report": {
"puppet_distributor": {
"modules": {
"error_message": null,
"execution_time": 0,
"total_count": 1,
"traceback": null,
"individual_errors": null,
"state": "success",
"error_count": 0,
"error": "None",
"finished_count": 1
},
"publishing": {
"http": "success",
"https": "skipped"
},
"metadata": {
"execution_time": 0,
"state": "success",
"error_message": null,
"error": "None",
"traceback": null
}
}
},
"queue": "reserved_resource_worker-1.lab.eng.bos.redhat.com.dq",
"state": "finished",
"result": {
"result": "success",
"exception": null,
"repo_id": "puppet-builds",
"started": "2014-10-02T14:10:13Z",
"_ns": "repo_publish_results",
"completed": "2014-10-02T14:10:13Z",
"traceback": null,
"distributor_type_id": "puppet_distributor",
"summary": {
"total_execution_time": 0
},
"error_message": null,
"distributor_id": "puppet_distributor",
"id": "542d5cc5dba9cb738fd58acb",
"details": {}
},
"error": null,
"_id": {
"$oid": "542d5cc57ccf31edbac08baf"
},
"id": "542d5cc5dba9cb738fd58ac9"
}
Resync
"progress_report": {
"puppet_importer": {
"modules": {
"error_message": null,
"execution_time": 0.004343986511230469,
"total_count": 0,
"traceback": null,
"individual_errors": [],
"state": "success",
"error_count": 0,
"error": "None",
"finished_count": 0
},
"metadata": {
"query_finished_count": 1,
"traceback": null,
"execution_time": 0.0008900165557861328,
"query_total_count": 1,
"error_message": null,
"state": "success",
"error": "None",
"current_query": null
}
}
},
"queue": "reserved_resource_worker-1.lab.eng.bos.redhat.com.dq",
"state": "finished",
"result": {
"result": "success",
"importer_id": "puppet_importer",
"exception": null,
"repo_id": "puppet-builds",
"traceback": null,
"started": "2014-10-02T14:11:25Z",
"_ns": "repo_sync_results",
"completed": "2014-10-02T14:11:25Z",
"importer_type_id": "puppet_importer",
"error_message": null,
"summary": {
"total_execution_time": 0.0052340030670166016
},
"added_count": 0,
"removed_count": 0,
"updated_count": 0,
"id": "542d5d0ddba9cb738fd58acd",
"details": {
"finished_count": 0,
"total_count": 0,
"error_count": 0
}
},
"error": null,
"_id": {
"$oid": "542d5d0d7ccf31edbac08bb0"
},
"id": "542d5d0ddba9cb6e3014ee6b"
}
2014-10-02 10:11:27,143 - INFO - GET request to /pulp/api/v2/tasks/846dbd34-a4ce-4667-a978-3793c7c86290/ with parameters None
2014-10-02 10:11:27,143 - INFO - Response status : 200
2014-10-02 10:11:27,143 - INFO - Response body :
{
"exception": null,
"task_type": "pulp.server.managers.repo.publish.publish",
"_href": "/pulp/api/v2/tasks/846dbd34-a4ce-4667-a978-3793c7c86290/",
"task_id": "846dbd34-a4ce-4667-a978-3793c7c86290",
"tags": [
"pulp:repository:puppet-builds",
"pulp:action:publish"
],
"finish_time": "2014-10-02T14:11:25Z",
"_ns": "task_status",
"start_time": "2014-10-02T14:11:25Z",
"traceback": null,
"spawned_tasks": [],
"progress_report": {
"puppet_distributor": {
"modules": {
"error_message": null,
"execution_time": 0,
"total_count": 1,
"traceback": null,
"individual_errors": null,
"state": "success",
"error_count": 0,
"error": "None",
"finished_count": 1
},
"publishing": {
"http": "success",
"https": "skipped"
},
"metadata": {
"execution_time": 0,
"state": "success",
"error_message": null,
"error": "None",
"traceback": null
}
}
},
"queue": "reserved_resource_worker-1.lab.eng.bos.redhat.com.dq",
"state": "finished",
"result": {
"result": "success",
"exception": null,
"repo_id": "puppet-builds",
"started": "2014-10-02T14:11:25Z",
"_ns": "repo_publish_results",
"completed": "2014-10-02T14:11:25Z",
"traceback": null,
"distributor_type_id": "puppet_distributor",
"summary": {
"total_execution_time": 0
},
"error_message": null,
"distributor_id": "puppet_distributor",
"id": "542d5d0ddba9cb738fd58ad0",
"details": {}
},
"error": null,
"_id": {
"$oid": "542d5d0d7ccf31edbac08bb1"
},
"id": "542d5d0ddba9cb738fd58ace"
}
|
Description of problem: Currently when syncing puppet forge-style repos, the progress report looks something like: On first sync: modules: traceback: execution_time: 1 total_count: 2136 error_message: finished_count: 2000 state: success error_count: 136 error: None On next sync: modules: traceback: execution_time: 1 total_count: 136 error_message: finished_count: 0 state: success error_count: 136 error: None This looks correct. But puppet repos that use a pulp manifest file are incorrect: On first sync: modules: traceback: execution_time: total_count: 4 error_message: finished_count: 0 state: running error_count: 0 error: None individual_errors: [] On next sync: modules: traceback: execution_time: total_count: 4 error_message: finished_count: 0 state: running error_count: 0 error: None individual_errors: [] There are 3 things wrong here: * State stays 'running'. Even though the task is finished * "finished_count" is 0 even though 4 modules were downloaded * The 2nd sync still shows a total of 4, even though all 4 modules were already synced the first time. Puppet forge syncing doesn't appear to show any of these issues Version-Release number of selected component (if applicable): pulp-server-2.4.0-0.29.beta.el6.noarch How reproducible: Always Steps to Reproduce: 1. Sync a pulp_manifest style puppet repo 2. Examine progress report Actual results: Progress report is not correct (3 issues above) Expected results: Progress report is correct Additional info: