Bug 1640119

Description of problem:
++++++++++++++++++++++++

See details  -https://bugzilla.redhat.com/show_bug.cgi?id=1638988#c11


As per Prasanna 

""
As of OCS-3.10 we already install a file at /var/lib/gluster-block/ called gluster-block-caps.info.

At OCS-3.11 we started maintaining the status file gb_upgrade.status at /var/lib/gluster-block/ as part of upgrade activities, and we need this directory to be persistent on pod restarts.

If we don't have /var/lib/gluster-block/ persistent, then on every pod restart, the upgrade activities will be triggered, which is not a good idea.

""

As seen, this folder is only available only on glusterfs pods, but not on the gluster-nodes. Hence the contents changed on this folder are not persistent across gluster pod restarts.


From node
===========
[root@apu-v311-app-cns-1 ~]# ls /var/lib/gluster-block
ls: cannot access /var/lib/gluster-block: No such file or directory
[root@apu-v311-app-cns-1 ~]# ls /var/lib|grep gluster
glusterd


As seen from the glusterfs pod
=================================
sh-4.2# ls -ltrh /var/log/gluster-block/
total 16K
-rw-r--r--. 1 root root  168 Oct 17 11:19 gluster-blockd.log
-rw-------. 1 root root 8.9K Oct 17 11:19 gluster-block-gfapi.log
sh-4.2# 







Version-Release number of selected component (if applicable):

# oc describe ds glusterfs-cns |grep -i image
    Image:      ocs/rhgs-server-rhel7:v3.11
[root@apu-v311-master-0 ~]# 



How reproducible:
=====

Always

Steps to Reproduce:
1. Install OCS 3.11
2. Check for the persistence of /var/lib/gluster-block folder.

Actual results:
=======


Expected results:


Additional info:
===========

The directories currently bind-mounted to pod are
=========================

terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /var/lib/heketi
            name: glusterfs-heketi
          - mountPath: /run
            name: glusterfs-run
          - mountPath: /run/lvm
            name: glusterfs-lvm
          - mountPath: /etc/glusterfs
            name: glusterfs-etc
          - mountPath: /var/log/glusterfs
            name: glusterfs-logs
          - mountPath: /var/lib/glusterd
            name: glusterfs-config
          - mountPath: /dev
            name: glusterfs-dev
          - mountPath: /var/lib/misc/glusterfsd
            name: glusterfs-misc
          - mountPath: /sys/fs/cgroup
            name: glusterfs-cgroup
            readOnly: true
          - mountPath: /etc/ssl
            name: glusterfs-ssl
            readOnly: true
          - mountPath: /usr/lib/modules
            name: kernel-modules
            readOnly: true
          - mountPath: /etc/target
            name: glusterfs-target