Bug 1669555 - Sync Pod script does not copy volume-config.yaml to node
Summary: Sync Pod script does not copy volume-config.yaml to node
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.11.z
Assignee: Patrick Dillon
QA Contact: Gaoyun Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-25 16:05 UTC by Ryan Howe
Modified: 2019-02-20 14:11 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: volume-config.yaml is not copied to /etc/origin/node/ Consequence: volume quotas were not being observed so local storage size (emptyDir) was not limited Fix: copy volume-config.yaml to /etc/origin/node/ and check for updates in sync pod Result: volume quotas are observed and local storage size is limited by setting openshift_node_local_quota_per_fsgroup in inventory
Clone Of:
Environment:
Last Closed: 2019-02-20 14:11:02 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0326 0 None None None 2019-02-20 14:11:07 UTC

Description Ryan Howe 2019-01-25 16:05:48 UTC
Description of problem:

Trying to set quotas and via installer but nothing gets set due to sync pod never moving the  volume-config.yaml from configmap to /etc/origin/node/ of the node. 


Version-Release number of the following components:
 rpm -q openshift-ansible
openshift-ansible-3.11.59-1.git.0.ba8e948.el7.noarch



How reproducible:

Steps to Reproduce:
1. Install with openshift_node_local_quota_per_fsgroup

Actual results:
Nothing is created on the nodes 


Expected results:
Sync pod to copy over the volume-config.yaml and possibly check for changes like it does with the node-config.yaml 

  # ls /etc/origin/node/volume-config.yaml
  volume-config.yaml


Additional info:

The piece are here to create the configmap 

   openshift_node_local_quota_per_fsgroup=1Gi

https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/openshift_node_group/tasks/create_config.yml#L43
https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/openshift_node_group/templates/volume-config.yaml.j2


Docs say it does this:
https://docs.openshift.com/container-platform/3.11/install_config/master_node_configuration.html#node-config-volume-config

Only node-config.yaml is moved 
https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_node_group/files/sync.yaml#L128

Volume emptyDir limits bug https://bugzilla.redhat.com/show_bug.cgi?id=1579305
And PR https://github.com/openshift/origin/pull/19533



---------------------- 
Work around now is just to set manually on the node. 

# cat > /etc/origin/node/volume-config.yaml << EOF
apiVersion: kubelet.config.openshift.io/v1
kind: VolumeConfig
localQuota:
  perFSGroupInGiB: 10
EOF

# systemctl restart atomic-openshift-node

Comment 1 Scott Dodson 2019-01-25 16:38:32 UTC
https://github.com/openshift/openshift-ansible/commit/ef259f710bf95c479acc37add40a9bc98b3bf1a4 is the commit that broke this by changing the path that it's extracted from /etc/origin/node to /etc/origin/node/tmp then further mutating things

Comment 2 Scott Dodson 2019-01-25 18:41:58 UTC
According to the pod team we need to trigger a restart when this file changes too.

Comment 3 chris.liles 2019-02-05 01:33:52 UTC
Fixed in https://github.com/openshift/openshift-ansible/pull/11125

Comment 7 chris.liles 2019-02-07 20:11:41 UTC
PR sponsored by RedHat -> https://github.com/openshift/openshift-ansible/pull/11131 supersedes 11125.

Comment 9 Gaoyun Pei 2019-02-11 06:16:12 UTC
Verify this bug with openshift-ansible-3.11.82-1.git.0.f29227a.el7.noarch.rpm

With openshift_node_local_quota_per_fsgroup=1Gi set in ansible inventory file, after installation, we could see volume-config.yaml created under /etc/origin/node/ as following:

[root@qe-gpei-311mrre-1 ~]# cat /etc/origin/node/volume-config.yaml 
apiVersion: kubelet.config.openshift.io/v1
kind: VolumeConfig
localQuota:
  perFSGroup: 1Gi

Comment 11 errata-xmlrpc 2019-02-20 14:11:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0326


Note You need to log in before you can comment on or make changes to this bug.