Bug 1371507

Summary: [RFE] Provide a way to run image pruning automatically/on a schedule
Product: OpenShift Container Platform Reporter: Alexander Koksharov <akokshar>
Component: RFEAssignee: Ben Parees <bparees>
Status: CLOSED DEFERRED QA Contact: Xiaoli Tian <xtian>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.2.0CC: aos-bugs, bparees, ccoleman, erich, jokerman, mfojtik, mmccomas, pweil, ssadeghi, sspeiche, vwalek
Target Milestone: ---Keywords: Reopened
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-16 21:18:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Alexander Koksharov 2016-08-30 11:40:04 UTC
Description of problem:
Build start failing due to error updating imagestream. It looks like imageStream json object size reaches some maximum value and it is not possible to push it to etcd any more.
We have retrieved the image-stream sizes on the two prod environments, where the build are not working anymore due to the given issue..

environment1:
        oc get is -o json | grep dockerImageReference | wc -l
        -> 3187
        oc get is -o json | wc --bytes
        -> 1467191


environment2:
        oc get is -o json | grep dockerImageReference | wc -l
         -> 3188
        oc get is -o json | wc --bytes
         -> 1467595


Version-Release number of selected component (if applicable):
3.2

How reproducible:
automatically trigger build ('oc start-build test-app') periodically. It it will stop working after ~3188 builds.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
there should be no limits

Additional info:

Comment 1 Clayton Coleman 2016-08-30 13:26:23 UTC
Pruning should be run occasionally.  We can also consider whether to allow users to opt-out of automatic old tag cleanup.

Comment 2 Michal Fojtik 2016-10-10 09:33:17 UTC
I would also suggest to do pruning occasionally when triggering 3188 builds.

Mid term, we will probably look into off-loading image manifests into an external storage, but that won't happen for 3.4.

Comment 4 Eric Rich 2016-12-01 22:46:55 UTC
(In reply to Michal Fojtik from comment #2)
> I would also suggest to do pruning occasionally when triggering 3188 builds.
> 
> Mid term, we will probably look into off-loading image manifests into an
> external storage, but that won't happen for 3.4.

This is already being requested in https://bugzilla.redhat.com/show_bug.cgi?id=1378180 

(In reply to Clayton Coleman from comment #1)
> Pruning should be run occasionally.  We can also consider whether to allow
> users to opt-out of automatic old tag cleanup.

I think what you are referring to is https://trello.com/c/WsnlyxUf/805-use-scheduledjobs-to-prune-images

Comment 6 Michal Fojtik 2017-03-27 12:16:31 UTC
Added Trello tracker.

Comment 8 Ben Parees 2018-12-20 17:26:02 UTC
*** Bug 1661212 has been marked as a duplicate of this bug. ***