Bug 1439926

Summary: [3.3] Image pruning may incorrectly delete an image if it has both weak and strong references
Product: OpenShift Container Platform Reporter: Andy Goldstein <agoldste>
Component: Image RegistryAssignee: Maciej Szulik <maszulik>
Status: CLOSED ERRATA QA Contact: ge liu <geliu>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.3.0CC: aos-bugs, byount, clichybi, eparis, maszulik, pweil, tdawson, yinzhou
Target Milestone: ---   
Target Release: 3.3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Logic error in how weak and strong references are identified when searching images eligible for pruning. Consequence: Some images having both strong and weak references in pruning graph could be removed during pruning. Fix: Fix logic responsible for finding which images have strong references. Result: Pruning correctly recognizes and prunes images.
 Story Points: ---
Clone Of:
: 1440177 1440197 1440199 1440201 1440202 (view as bug list) Environment:
Last Closed: 2017-04-19 19:43:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1440177, 1440197, 1440199, 1440201, 1440202    

Description Andy Goldstein 2017-04-06 21:19:20 UTC 
Description of problem: Image pruning may incorrectly delete an image if it has both weak and strong references


Version-Release number of selected component (if applicable): 3.3.1.9


How reproducible: easily with a unit test in a loop


Steps to Reproduce:
1. Push an image A to an image stream with tag "aaa"
2. oc tag that image to another tag "bbb"
3. Push other images B, C, D, E, F, G to the tag "aaa" so that image A is lower than #5 in the history list for tag "aaa"
4. run 'oadm prune images --keep-tag-revisions=5'

Actual results: sometimes image A is pruned, sometimes it isn't


Expected results: image A is never pruned


Additional info:

This is happening due to a flaw in the logic for how we determine if an image is prunable. We were checking for the absence of a weak image reference from the image stream to the image. A weak image reference is one where we know there is a link from the image stream to the image, such as being in revision #6 in a tag's history, and if there are no other "strong" references, this image may be pruned. A strong reference is one where the image's position in the tag's history is less than --keep-tag-revisions.

It is possible for an image stream to have both weak and strong references to an image. It all depends on the order in which we process the image stream's tags, and this order is nondeterministic. Using the example above, if we process tag "bbb" first, that creates a strong reference (revision=1), and we'll never add a weak reference if we encounter one. If, on the other hand, we process tag "aaa" first, we'll add a weak reference (revision=6). When we next process the tag "bbb", we'll add a strong reference. When the logic runs to determine if the image is prunable, if it looks for the absence of a weak reference, one of two things could happen in this scenario. If there are both weak and strong references, then we would incorrectly consider the image prunable. If there is only a strong reference, then we would correctly not prune the image.

The fix is to change the logic to look for strong references explicitly, since that is what keeps us from pruning.
Comment 10 errata-xmlrpc 2017-04-19 19:43:21 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0989