Bug 1591190 - Installer should prevent from installing dockergc if docker storage driver is devicemapper
Summary: Installer should prevent from installing dockergc if docker storage driver is...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.10.z
Assignee: Scott Dodson
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On: 1568772
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-14 08:48 UTC by Gan Huang
Modified: 2018-11-19 20:12 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1568772
Environment:
Last Closed: 2018-11-19 20:12:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Gan Huang 2018-06-14 08:48:21 UTC
+++ This bug was initially created as a clone of Bug #1568772 +++

Description of problem:
dockergc pods would get crashed if docker storage driver is devicemapper.

Please see Bug 1511400.

Installer/OpenShift health check should stop from moving the installation forward.

Version-Release number of the following components:
openshift-ansible-3.9.22-1.git.7.92620c6.el7.noarch.rpm

How reproducible:
always

Steps to Reproduce:
1. trigger installation with devicemapper docker storage driver 
# cat inventory
<--snip-->
openshift_crio_use_rpm=true
openshift_crio_docker_gc_node_selector={"role": "node"}
<--snip-->

Actual results:
Installation succeeded, but dockergc pods got crashed:

# oc get po
NAME                       READY     STATUS             RESTARTS   AGE
docker-registry-1-xb8wj    1/1       Running            0          1h
dockergc-rrxsd             0/1       CrashLoopBackOff   24         1h
dockergc-xj6tv             0/1       CrashLoopBackOff   23         1h
registry-console-1-nsw8q   1/1       Running            0          1h
router-1-tp2vq             1/1       Running            0          1h

# oc logs dockergc-rrxsd
I0418 08:17:17.462417       1 dockergc.go:242] docker build garbage collection daemon
I0418 08:17:17.462537       1 dockergc.go:246] MinimumGCAge: {1h0m0s}, ImageGCHighThresholdPercent: 80, ImageGCLowThresholdPercent: 60


Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Vadim Rutkovsky 2018-06-14 10:39:08 UTC
I don't think it possible to implement this in openshift-ansible.

Docker GC is being setup as a daemonset, thus its cluster-wide. Docker driver config can be different per host, so its possible to have some hosts on overlay driver and some on devicemapper - docker gc would work fine in this case. Its also possible to deploy docker gc on a cluster with devicemapper nodes and scale up new hosts with overlayfs.

The issue with restarting docker gc pods should be solved by proper node labeling and nodeselector.

Scott, WDYT?

Comment 2 Scott Dodson 2018-06-14 13:11:27 UTC
My understanding is that when deploying cri-o the dockergc is created with the selector specified in openshift_crio_docker_gc_node_selector.

Need to verify that the install works in scenarios when cri-o is in use but runtime=cri-o is not applied to any nodes.

Comment 3 Russell Teague 2018-11-19 20:12:59 UTC
There appear to be no active cases related to this bug. As such we're closing this bug in order to focus on bugs that are still tied to active customer cases. Please re-open this bug if you feel it was closed in error or a new active case is attached.


Note You need to log in before you can comment on or make changes to this bug.