Bug 1439926 - [3.3] Image pruning may incorrectly delete an image if it has both weak and strong references
Summary: [3.3] Image pruning may incorrectly delete an image if it has both weak and s...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.3.1
Assignee: Maciej Szulik
QA Contact: ge liu
URL:
Whiteboard:
Depends On:
Blocks: 1440177 1440197 1440199 1440201 1440202
TreeView+ depends on / blocked
 
Reported: 2017-04-06 21:19 UTC by Andy Goldstein
Modified: 2021-08-30 13:18 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Logic error in how weak and strong references are identified when searching images eligible for pruning. Consequence: Some images having both strong and weak references in pruning graph could be removed during pruning. Fix: Fix logic responsible for finding which images have strong references. Result: Pruning correctly recognizes and prunes images.
Clone Of:
: 1440177 1440197 1440199 1440201 1440202 (view as bug list)
Environment:
Last Closed: 2017-04-19 19:43:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2995561 0 None None None 2017-04-11 20:17:58 UTC
Red Hat Product Errata RHBA-2017:0989 0 normal SHIPPED_LIVE OpenShift Container Platform 3.4, 3.3, 3.2, and 3.1 bug fix update 2017-04-19 23:42:19 UTC

Description Andy Goldstein 2017-04-06 21:19:20 UTC
Description of problem: Image pruning may incorrectly delete an image if it has both weak and strong references


Version-Release number of selected component (if applicable): 3.3.1.9


How reproducible: easily with a unit test in a loop


Steps to Reproduce:
1. Push an image A to an image stream with tag "aaa"
2. oc tag that image to another tag "bbb"
3. Push other images B, C, D, E, F, G to the tag "aaa" so that image A is lower than #5 in the history list for tag "aaa"
4. run 'oadm prune images --keep-tag-revisions=5'

Actual results: sometimes image A is pruned, sometimes it isn't


Expected results: image A is never pruned


Additional info:

This is happening due to a flaw in the logic for how we determine if an image is prunable. We were checking for the absence of a weak image reference from the image stream to the image. A weak image reference is one where we know there is a link from the image stream to the image, such as being in revision #6 in a tag's history, and if there are no other "strong" references, this image may be pruned. A strong reference is one where the image's position in the tag's history is less than --keep-tag-revisions.

It is possible for an image stream to have both weak and strong references to an image. It all depends on the order in which we process the image stream's tags, and this order is nondeterministic. Using the example above, if we process tag "bbb" first, that creates a strong reference (revision=1), and we'll never add a weak reference if we encounter one. If, on the other hand, we process tag "aaa" first, we'll add a weak reference (revision=6). When we next process the tag "bbb", we'll add a strong reference. When the logic runs to determine if the image is prunable, if it looks for the absence of a weak reference, one of two things could happen in this scenario. If there are both weak and strong references, then we would incorrectly consider the image prunable. If there is only a strong reference, then we would correctly not prune the image.

The fix is to change the logic to look for strong references explicitly, since that is what keeps us from pruning.

Comment 10 errata-xmlrpc 2017-04-19 19:43:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0989


Note You need to log in before you can comment on or make changes to this bug.