Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1440177 - [3.6] Image pruning may incorrectly delete an image if it has both weak and strong references
[3.6] Image pruning may incorrectly delete an image if it has both weak and s...
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry (Show other bugs)
3.6.0
Unspecified Unspecified
urgent Severity urgent
: ---
: ---
Assigned To: Maciej Szulik
ge liu
:
Depends On: 1439926 1440197 1440199 1440201 1440202
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-07 09:41 EDT by Andy Goldstein
Modified: 2017-08-16 15 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Logic error in how weak and strong references are identified when searching images eligible for pruning. Consequence: Some images having both strong and weak references in pruning graph could be removed during pruning. Fix: Fix logic responsible for finding which images have strong references. Result: Pruning correctly recognizes and prunes images.
Story Points: ---
Clone Of: 1439926
Environment:
Last Closed: 2017-08-10 01:20:02 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1716 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.6 RPM Release Advisory 2017-08-10 05:02:50 EDT

  None (edit)
Comment 1 Andy Goldstein 2017-04-07 09:42:33 EDT
Description of problem: Image pruning may incorrectly delete an image if it has both weak and strong references


Version-Release number of selected component (if applicable): 3.3.1.9


How reproducible: easily with a unit test in a loop


Steps to Reproduce:
1. Push an image A to an image stream with tag "aaa"
2. oc tag that image to another tag "bbb"
3. Push other images B, C, D, E, F, G to the tag "aaa" so that image A is lower than #5 in the history list for tag "aaa"
4. run 'oadm prune images --keep-tag-revisions=5'

Actual results: sometimes image A is pruned, sometimes it isn't


Expected results: image A is never pruned


Additional info:

This is happening due to a flaw in the logic for how we determine if an image is prunable. We were checking for the absence of a weak image reference from the image stream to the image. A weak image reference is one where we know there is a link from the image stream to the image, such as being in revision #6 in a tag's history, and if there are no other "strong" references, this image may be pruned. A strong reference is one where the image's position in the tag's history is less than --keep-tag-revisions.

It is possible for an image stream to have both weak and strong references to an image. It all depends on the order in which we process the image stream's tags, and this order is nondeterministic. Using the example above, if we process tag "bbb" first, that creates a strong reference (revision=1), and we'll never add a weak reference if we encounter one. If, on the other hand, we process tag "aaa" first, we'll add a weak reference (revision=6). When we next process the tag "bbb", we'll add a strong reference. When the logic runs to determine if the image is prunable, if it looks for the absence of a weak reference, one of two things could happen in this scenario. If there are both weak and strong references, then we would incorrectly consider the image prunable. If there is only a strong reference, then we would correctly not prune the image.

The fix is to change the logic to look for strong references explicitly, since that is what keeps us from pruning.
Comment 2 Andy Goldstein 2017-04-07 09:43:10 EDT
https://github.com/openshift/origin/pull/13671
Comment 3 Troy Dawson 2017-04-12 08:59:03 EDT
This has been merged into ocp and is in OCP v3.6.28 or newer.
Comment 7 errata-xmlrpc 2017-08-10 01:20:02 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1716

Note You need to log in before you can comment on or make changes to this bug.