Description of problem:

If you specify perFSGroup in your node-config.yaml before the first node startup, the volume directory does not yet exist at the correct time and checks for the quota plugin cannot run, the node will not start.

Ops is planning to use openshift-ansible to manage node configs, which will soon support setting perFSGroup quota. Thus any time a new node is created it will fail to start.


Version-Release number of selected component (if applicable):

Problem is present in origin master as of 1ae5b25eaf86357306ce9f0c2fca4bba7e81a6da


How reproducible:

Should be 100% if you create a new node, and have a way to set the perFSGroup quota *before* the node is first launched.

Easiest way to do this is probably once the openshift-ansible fix is in to enable setting this setting. (will link to PR as soon as it's available) 


Steps to Reproduce:

1. Ensure node-config.yaml has a perFSGroup setting before the node is first started:

volumeConfig:
  localQuota:
    perFSGroup: "512Mi"

Actual results:

Node service will refuse to start and show errors in journalctl similar to:

Mar 22 08:06:09 node3.aos.example.com atomic-openshift-node[4638]: I0322 08:06:09.004776    4638 node_config.go:179] Replacing empty-dir volume plugin with quota wrapper
Mar 22 08:06:09 node3.aos.example.com atomic-openshift-node[4638]: F0322 08:06:09.008850    4638 start_node.go:124] unable to check filesystem type for emptydir volume /var/lib/origin/openshift.local.volumes: exit status 1

You will see that /var/lib/origin/openshift.local.volumes does not yet exist.

Expected results:

The checks should only be performed *after* the volumeDirectory is known to exist.


Additional info: