Bug 1658386 - Container runtime should limit container storage
Summary: Container runtime should limit container storage
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
: 3.11.z
Assignee: Russell Teague
QA Contact: Johnny Liu
URL:
Whiteboard:
: 1658385 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-11 22:46 UTC by brad.williams
Modified: 2019-10-15 15:21 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-15 15:21:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description brad.williams 2018-12-11 22:46:20 UTC
Description of problem:

Observed on starter-ca-central-1 cluster. Our pre-upgrade diagnostics reported disk pressure on 10 compute nodes.  Upon further investigation, all 10 nodes had had their container storage fully consumed by a single rogue pod (tropospheric) that continuously creates a core dump on the container storage volume. 

Version-Release number of selected component (if applicable):

openshift v3.11.44
cri-o://1.11.9

How reproducible:
100%

Steps to Reproduce:
1. Create a project that continuously writes to the container storage volume.  

Actual results:

Eventually, this project will consume the entire disk and ultimately criple the container runtime.

Expected results:

Ideally, a container would not be able to consume all of the shared resources that the container runtime is providing.

Additional info:

Comment 6 Mrunal Patel 2019-01-14 21:48:36 UTC
*** Bug 1658385 has been marked as a duplicate of this bug. ***

Comment 7 Daniel Walsh 2019-03-08 20:36:41 UTC
Do we expose this in the OpenShift Operator for 4.0?

Comment 8 Mrunal Patel 2019-03-08 21:44:38 UTC
Yes, we did but Justin wanted this open for adding this to openshift-ansible.

Comment 9 Seth Jennings 2019-07-30 15:12:34 UTC
Sending to Installer to eval change for 3.x.

For 4.x, I opened https://jira.coreos.com/browse/NODE-163 to verify we can set `overlay.size` in /etc/containers/storage via the ContainerRuntimeConfigs CRD.

Comment 10 Scott Dodson 2019-10-15 15:21:47 UTC
Online no longer needs this and there's no customer case attached so marking as deferred.


Note You need to log in before you can comment on or make changes to this bug.