Bug 1669555

Summary:	Sync Pod script does not copy volume-config.yaml to node
Product:	OpenShift Container Platform	Reporter:	Ryan Howe <rhowe>
Component:	Installer	Assignee:	Patrick Dillon <padillon>
Status:	CLOSED ERRATA	QA Contact:	Gaoyun Pei <gpei>
Severity:	high	Docs Contact:
Priority:	high
Version:	3.11.0	CC:	aos-bugs, chris.liles, fshaikh, gpei, jokerman, mmccomas, padillon, vrutkovs
Target Milestone:	---
Target Release:	3.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: volume-config.yaml is not copied to /etc/origin/node/ Consequence: volume quotas were not being observed so local storage size (emptyDir) was not limited Fix: copy volume-config.yaml to /etc/origin/node/ and check for updates in sync pod Result: volume quotas are observed and local storage size is limited by setting openshift_node_local_quota_per_fsgroup in inventory	Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-02-20 14:11:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ryan Howe 2019-01-25 16:05:48 UTC

Description of problem:

Trying to set quotas and via installer but nothing gets set due to sync pod never moving the  volume-config.yaml from configmap to /etc/origin/node/ of the node. 


Version-Release number of the following components:
 rpm -q openshift-ansible
openshift-ansible-3.11.59-1.git.0.ba8e948.el7.noarch



How reproducible:

Steps to Reproduce:
1. Install with openshift_node_local_quota_per_fsgroup

Actual results:
Nothing is created on the nodes 


Expected results:
Sync pod to copy over the volume-config.yaml and possibly check for changes like it does with the node-config.yaml 

  # ls /etc/origin/node/volume-config.yaml
  volume-config.yaml


Additional info:

The piece are here to create the configmap 

   openshift_node_local_quota_per_fsgroup=1Gi

https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/openshift_node_group/tasks/create_config.yml#L43
https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/openshift_node_group/templates/volume-config.yaml.j2


Docs say it does this:
https://docs.openshift.com/container-platform/3.11/install_config/master_node_configuration.html#node-config-volume-config

Only node-config.yaml is moved 
https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_node_group/files/sync.yaml#L128

Volume emptyDir limits bug https://bugzilla.redhat.com/show_bug.cgi?id=1579305
And PR https://github.com/openshift/origin/pull/19533



---------------------- 
Work around now is just to set manually on the node. 

# cat > /etc/origin/node/volume-config.yaml << EOF
apiVersion: kubelet.config.openshift.io/v1
kind: VolumeConfig
localQuota:
  perFSGroupInGiB: 10
EOF

# systemctl restart atomic-openshift-node

Comment 1 Scott Dodson 2019-01-25 16:38:32 UTC

https://github.com/openshift/openshift-ansible/commit/ef259f710bf95c479acc37add40a9bc98b3bf1a4 is the commit that broke this by changing the path that it's extracted from /etc/origin/node to /etc/origin/node/tmp then further mutating things

Comment 2 Scott Dodson 2019-01-25 18:41:58 UTC

According to the pod team we need to trigger a restart when this file changes too.

Comment 3 chris.liles 2019-02-05 01:33:52 UTC

Fixed in https://github.com/openshift/openshift-ansible/pull/11125

Comment 7 chris.liles 2019-02-07 20:11:41 UTC

PR sponsored by RedHat -> https://github.com/openshift/openshift-ansible/pull/11131 supersedes 11125.

Comment 9 Gaoyun Pei 2019-02-11 06:16:12 UTC

Verify this bug with openshift-ansible-3.11.82-1.git.0.f29227a.el7.noarch.rpm

With openshift_node_local_quota_per_fsgroup=1Gi set in ansible inventory file, after installation, we could see volume-config.yaml created under /etc/origin/node/ as following:

[root@qe-gpei-311mrre-1 ~]# cat /etc/origin/node/volume-config.yaml 
apiVersion: kubelet.config.openshift.io/v1
kind: VolumeConfig
localQuota:
  perFSGroup: 1Gi

Comment 11 errata-xmlrpc 2019-02-20 14:11:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0326