Bug 765930

Summary:	ProgressTracker: Issue when re-downloading a failed file, "Error Negative delat_bytes"
Product:	[Retired] Pulp	Reporter:	John Matthews <jmatthew>
Component:	user-experience	Assignee:	John Matthews <jmatthew>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	1.0.0	CC:	skarmark
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-02-24 20:11:04 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description John Matthews 2011-12-09 17:42:31 UTC

Description of problem:

When grinder fails a file and does a re-fetch we are not cleaning up the ProgressTracker correctly.

In BaseFetch look at when verifyChecksum fails, we re-fetch the file but we don't clear out previous progress.  Progress should be reset and we should start from scratch in regard to this file.


2011-12-09 11:44:14,421 26213:140597163718400: grinder.BaseFetch:ERROR: activeobject:160 Retrying fetch of: boot.iso with 1 retry attempts left.
2011-12-09 11:44:14,422 26213:140597163718400: grinder.BaseFetch:INFO: activeobject:160 Fetching Unknown bytes: boot.iso from https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso
2011-12-09 11:44:16,245 26213:140597163718400: grinder.ProgressTracker:ERROR: ProgressTracker:206 Negative delta_bytes <-262128030.0>. download_total=<262144000.0>, downloaded=<15970.0>, prev_remaining_bytes=<0.0>, remaining_bytes=<262128030.0>, total_size_bytes=<262144000.0>, https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso
2011-12-09 11:44:16,246 26213:140597163718400: grinder.ProgressTracker:ERROR: ProgressTracker:206 Negative delta_bytes <-262124381.0>. download_total=<262144000.0>, downloaded=<19619.0>, prev_remaining_bytes=<0.0>, remaining_bytes=<262124381.0>, total_size_bytes=<262144000.0>, https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso


QE:

Test Scenario is to sync a repo that has a treeinfo specifying a file with a bad checksum.

We want to see grinder:
 1) Download file and fail it on a checksum
 2) Attempt re-download, will still fail (this is OK)

Success for this BZ is that grinder logs do not say:  "ERROR: negative delta_bytes".

Below is an example of CLI output that looks correct for this context.
Our goal is to look at the grinder.log and ensure we removed the case of delta_bytes becoming negative.

pulp-admin repo create --id rhel6_1_x86_64_preserve --feed https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os/ --feed_ca ./cdn.redhat.com-chain.crt --feed_cert ./rcm-debug.pem --preserve_metadata


$ time sudo pulp-admin repo sync --id rhel6_1_x86_64_preserve -F
Sync for repository rhel6_1_x86_64_preserve started
Sync: Finished
5611/5612 new items downloaded
1/5612 existing items processed

Item Details: 
Tree Files: 5/6
RPMs: 5606/5606
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os//images/boot.iso


real	63m2.202s
user	1m27.437s
sys	0m7.562s

Comment 1 John Matthews 2011-12-15 14:05:28 UTC

Added ability to reset the progress tracker before we refetch a file.

http://git.fedorahosted.org/git/?p=grinder.git;a=commitdiff;h=58b1f43b761b626ad6649af8d5c3c2a9c3054b89

Comment 2 Jeff Ortel 2011-12-15 20:18:05 UTC

build: 0.255

Comment 3 Preethi Thomas 2012-01-03 19:38:38 UTC

[root@preethi CDN]# rpm -q pulp
pulp-0.0.255-1.fc15.noarch


[root@preethi CDN]# pulp-admin repo sync --id=rhel6_x86_64_preserve -F
Sync for repository rhel6_x86_64_preserve started
Sync: Finished
3396/3397 new items downloaded
1/3397 existing items processed

Item Details: 
Tree Files: 5/6
RPMs: 3391/3391
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on https://cdn.redhat.com/content/dist/rhel/rhui/server-6/releases/6Server/x86_64/os/images/boot.iso

[root@preethi CDN]# pulp-admin repo sync --id=test_iso_repo -FSync for repository test_iso_repo started
Sync: Finished
1/2 new items downloaded
1/2 existing items processed

Item Details: 
Tree Files: 1/2
Warning: 1 errors occurred
1 Tree_File Error(s):
	md5_missmatch on http://jmatthews.fedorapeople.org/repo_with_iso_bad_checksum//images/file2.img

Comment 4 Preethi Thomas 2012-02-24 20:11:04 UTC

Pulp v1.0 is released
Closed Current Release.