Bug 1440177

Summary:	[3.6] Image pruning may incorrectly delete an image if it has both weak and strong references
Product:	OpenShift Container Platform	Reporter:	Andy Goldstein <agoldste>
Component:	Image Registry	Assignee:	Maciej Szulik <maszulik>
Status:	CLOSED ERRATA	QA Contact:	ge liu <geliu>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.6.0	CC:	aos-bugs, eparis, geliu, haowang, maszulik, pweil, tdawson, yinzhou
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Logic error in how weak and strong references are identified when searching images eligible for pruning. Consequence: Some images having both strong and weak references in pruning graph could be removed during pruning. Fix: Fix logic responsible for finding which images have strong references. Result: Pruning correctly recognizes and prunes images.	Story Points:	---
Clone Of:	1439926	Environment:
Last Closed:	2017-08-10 05:20:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1439926, 1440197, 1440199, 1440201, 1440202
Bug Blocks:

Comment 1 Andy Goldstein 2017-04-07 13:42:33 UTC

Description of problem: Image pruning may incorrectly delete an image if it has both weak and strong references

Version-Release number of selected component (if applicable): 3.3.1.9

How reproducible: easily with a unit test in a loop

Steps to Reproduce:
1. Push an image A to an image stream with tag "aaa"
2. oc tag that image to another tag "bbb"
3. Push other images B, C, D, E, F, G to the tag "aaa" so that image A is lower than #5 in the history list for tag "aaa"
4. run 'oadm prune images --keep-tag-revisions=5'

Actual results: sometimes image A is pruned, sometimes it isn't

Expected results: image A is never pruned

Additional info:

This is happening due to a flaw in the logic for how we determine if an image is prunable. We were checking for the absence of a weak image reference from the image stream to the image. A weak image reference is one where we know there is a link from the image stream to the image, such as being in revision #6 in a tag's history, and if there are no other "strong" references, this image may be pruned. A strong reference is one where the image's position in the tag's history is less than --keep-tag-revisions.

It is possible for an image stream to have both weak and strong references to an image. It all depends on the order in which we process the image stream's tags, and this order is nondeterministic. Using the example above, if we process tag "bbb" first, that creates a strong reference (revision=1), and we'll never add a weak reference if we encounter one. If, on the other hand, we process tag "aaa" first, we'll add a weak reference (revision=6). When we next process the tag "bbb", we'll add a strong reference. When the logic runs to determine if the image is prunable, if it looks for the absence of a weak reference, one of two things could happen in this scenario. If there are both weak and strong references, then we would incorrectly consider the image prunable. If there is only a strong reference, then we would correctly not prune the image.

The fix is to change the logic to look for strong references explicitly, since that is what keeps us from pruning.

Comment 2 Andy Goldstein 2017-04-07 13:43:10 UTC

https://github.com/openshift/origin/pull/13671

Comment 3 Troy Dawson 2017-04-12 12:59:03 UTC

This has been merged into ocp and is in OCP v3.6.28 or newer.

Comment 7 errata-xmlrpc 2017-08-10 05:20:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1716