Bug 765930

Summary: ProgressTracker: Issue when re-downloading a failed file, "Error Negative delat_bytes"
Product: [Retired] Pulp Reporter: John Matthews <jmatthew>
Component: user-experienceAssignee: John Matthews <jmatthew>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: skarmark
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-24 20:11:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description John Matthews 2011-12-09 17:42:31 UTC 
Description of problem:

When grinder fails a file and does a re-fetch we are not cleaning up the ProgressTracker correctly.

In BaseFetch look at when verifyChecksum fails, we re-fetch the file but we don't clear out previous progress.  Progress should be reset and we should start from scratch in regard to this file.


2011-12-09 11:44:14,421 26213:140597163718400: grinder.BaseFetch:ERROR: activeobject:160 Retrying fetch of: boot.iso with 1 retry attempts left.
2011-12-09 11:44:14,422 26213:140597163718400: grinder.BaseFetch:INFO: activeobject:160 Fetching Unknown bytes: boot.iso from https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso
2011-12-09 11:44:16,245 26213:140597163718400: grinder.ProgressTracker:ERROR: ProgressTracker:206 Negative delta_bytes <-262128030.0>. download_total=<262144000.0>, downloaded=<15970.0>, prev_remaining_bytes=<0.0>, remaining_bytes=<262128030.0>, total_size_bytes=<262144000.0>, https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso
2011-12-09 11:44:16,246 26213:140597163718400: grinder.ProgressTracker:ERROR: ProgressTracker:206 Negative delta_bytes <-262124381.0>. download_total=<262144000.0>, downloaded=<19619.0>, prev_remaining_bytes=<0.0>, remaining_bytes=<262124381.0>, total_size_bytes=<262144000.0>, https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso


QE:

Test Scenario is to sync a repo that has a treeinfo specifying a file with a bad checksum.

We want to see grinder:
 1) Download file and fail it on a checksum
 2) Attempt re-download, will still fail (this is OK)

Success for this BZ is that grinder logs do not say:  "ERROR: negative delta_bytes".

Below is an example of CLI output that looks correct for this context.
Our goal is to look at the grinder.log and ensure we removed the case of delta_bytes becoming negative.

pulp-admin repo create --id rhel6_1_x86_64_preserve --feed https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os/ --feed_ca ./cdn.redhat.com-chain.crt --feed_cert ./rcm-debug.pem --preserve_metadata


$ time sudo pulp-admin repo sync --id rhel6_1_x86_64_preserve -F
Sync for repository rhel6_1_x86_64_preserve started
Sync: Finished
5611/5612 new items downloaded
1/5612 existing items processed

Item Details: 
Tree Files: 5/6
RPMs: 5606/5606
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso


real	63m2.202s
user	1m27.437s
sys	0m7.562s
Comment 1 John Matthews 2011-12-15 14:05:28 UTC 
Added ability to reset the progress tracker before we refetch a file.

http://git.fedorahosted.org/git/?p=grinder.git;a=commitdiff;h=58b1f43b761b626ad6649af8d5c3c2a9c3054b89
Comment 2 Jeff Ortel 2011-12-15 20:18:05 UTC 
build: 0.255
Comment 3 Preethi Thomas 2012-01-03 19:38:38 UTC 
[root@preethi CDN]# rpm -q pulp
pulp-0.0.255-1.fc15.noarch


[root@preethi CDN]# pulp-admin repo sync --id=rhel6_x86_64_preserve -F
Sync for repository rhel6_x86_64_preserve started
Sync: Finished
3396/3397 new items downloaded
1/3397 existing items processed

Item Details: 
Tree Files: 5/6
RPMs: 3391/3391
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on https://cdn.redhat.com/content/dist/rhel/rhui/server-6/releases/6Server/x86_64/os/images/boot.iso

[root@preethi CDN]# pulp-admin repo sync --id=test_iso_repo -FSync for repository test_iso_repo started
Sync: Finished
1/2 new items downloaded
1/2 existing items processed

Item Details: 
Tree Files: 1/2
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on http://jmatthews.fedorapeople.org/repo_with_iso_bad_checksum//images/file2.img
Comment 4 Preethi Thomas 2012-02-24 20:11:04 UTC 
Pulp v1.0 is released
Closed Current Release.