Bug 1917659

Summary: Gluster bricks lack SELinux labels post gluster deployment
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: SATHEESARAN <sasundar>
Component: rhhiAssignee: Gobinda Das <godas>
Status: CLOSED CURRENTRELEASE QA Contact: SATHEESARAN <sasundar>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhhiv-1.8CC: rcyriac, rhs-bugs
Target Milestone: ---Keywords: ZStream
Target Release: RHHI-V 1.8.z Batch Update 4   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Gluster bricks lacks the required SELinux label on gluster bricks as the SELinux labels were applied on the incorrect brick names Consequence: Gluster bricks lack the requried SELinux label Fix: gluster-ansible task is fixed to apply the SELinux label on the correct names of brick directories Result: gluster bricks are applied with correct SELinux label
 Story Points: ---
Clone Of:
: 1917661 (view as bug list) Environment:
rhhiv
Last Closed: 2021-04-29 05:49:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1917661    
Bug Blocks:    

Description SATHEESARAN 2021-01-19 05:15:27 UTC 
Description of problem:
-------------------------
The gluster bricks doesn't have the required SELinux labels on them post deployment. 

Version-Release number of selected component (if applicable):
--------------------------------------------------------------
RHHI-V 1.8.3 ( RHV 4.4.3 )
As well as in RHHI-V 1.8.2 ( RHV 4.4.2 )
This was not checked with RHHI-V 1.8

How reproducible:
------------------
Always

Steps to Reproduce:
---------------------
1. Start the RHHI-V deployment from cockpit or CLI
2. After gluster deployment, check for SELinux labels set on the brick

Actual results:
----------------
Gluster brick mounts lack proper SELinux labels on them

Expected results:
-----------------
Gluster brick mounts should have the correct SELinux labels on them
Comment 1 SATHEESARAN 2021-01-19 05:24:41 UTC 
This is the ansible playbook that sets the SELinux labels:

<snip>
- name: Set Gluster specific SeLinux context on the bricks
  sefcontext:
     target: "{{ (item.path | realpath | regex_escape()) + '(/.*)?' }}"
     setype: glusterd_brick_t
     state: present
  with_items: "{{ gluster_infra_mount_devices }}"
  when: gluster_set_selinux_labels| default(false)| bool == true
</snip>

Console log while executing this task:
---------------------------------------
<snip>
TASK [gluster.infra/roles/backend_setup : Set Gluster specific SeLinux context on the bricks] ***
changed: [rhsqa-grafton10.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/engine', 'lvname': 'gluster_lv_engine', 'vgname': 'gluster_vg_sdb'})
changed: [rhsqa-grafton11.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/engine', 'lvname': 'gluster_lv_engine', 'vgname': 'gluster_vg_sdb'})
changed: [rhsqa-grafton12.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/engine', 'lvname': 'gluster_lv_engine', 'vgname': 'gluster_vg_sdb'})
changed: [rhsqa-grafton10.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/data', 'lvname': 'gluster_lv_data', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton11.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/data', 'lvname': 'gluster_lv_data', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton12.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/data', 'lvname': 'gluster_lv_data', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton11.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/vmstore', 'lvname': 'gluster_lv_vmstore', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton10.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/vmstore', 'lvname': 'gluster_lv_vmstore', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton12.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/vmstore', 'lvname': 'gluster_lv_vmstore', 'vgname': 'gluster_vg_sdc'})
changed: [rhsqa-grafton10.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/testvol', 'lvname': 'gluster_lv_testvol', 'vgname': 'gluster_vg_sdd'})
changed: [rhsqa-grafton11.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/testvol', 'lvname': 'gluster_lv_testvol', 'vgname': 'gluster_vg_sdd'})
changed: [rhsqa-grafton12.lab.eng.blr.redhat.com] => (item={'path': '/gluster_bricks/testvol', 'lvname': 'gluster_lv_testvol', 'vgname': 'gluster_vg_sdd'})
</snip>

Checking for SELinux labels on the mount:
[root@ ]# ls -lsahZd /gluster_bricks/testvol
0 drwxr-xr-x. 3 root root system_u:object_r:default_t:s0 21 Jan 12 07:48 /gluster_bricks/testvol

[root@ ]# semanage fcontext -E /gluster_bricks/testvol
fcontext -a -f a -t glusterd_brick_t -r 's0' '\/gluster_bricks\/data(/.*)?'
fcontext -a -f a -t glusterd_brick_t -r 's0' '\/gluster_bricks\/engine(/.*)?'
fcontext -a -f a -t glusterd_brick_t -r 's0' '\/gluster_bricks\/testvol(/.*)?'
fcontext -a -f a -t glusterd_brick_t -r 's0' '\/gluster_bricks\/vmstore(/.*)?'
Comment 2 SATHEESARAN 2021-02-25 11:38:03 UTC 
The issue is due to the usage of regex.escape() in the playbook.
When this is removed, I could observe that the bricks have the right set of selinux labels on them.


Existing playbook code:
------------------------
<snip>
- name: Set Gluster specific SeLinux context on the bricks
  sefcontext:
     target: "{{ (item.path | realpath | regex_escape()) + '(/.*)?' }}"   <-- usage of regex_escape
     setype: glusterd_brick_t
     state: present
  with_items: "{{ gluster_infra_mount_devices }}"
  when: gluster_set_selinux_labels| default(false)| bool == true
</snip>


Proposed change
----------------
<snip>
- name: Set Gluster specific SeLinux context on the bricks
  sefcontext:
     target: "{{ (item.path | realpath) + '(/.*)?' }}"
     setype: glusterd_brick_t
     state: present
  with_items: "{{ gluster_infra_mount_devices }}"
  when: gluster_set_selinux_labels| default(false)| bool == true
</snip>

I have performed all the tests with this modified code and everything looks good.
Comment 4 Gobinda Das 2021-03-01 05:25:13 UTC 
Upstream PR: https://github.com/gluster/gluster-ansible-infra/pull/118
Comment 5 SATHEESARAN 2021-03-01 13:19:16 UTC 
Verified with gluster-ansible-infra-1.0.19.el8rhgs

Post gluster deployment, gluster bricks contain the right SELinux label - glusterd_brick_t

[root@ ~]# ls -lZd /gluster_bricks/engine/engine/
drwxr-xr-x. 3 vdsm kvm system_u:object_r:glusterd_brick_t:s0 24 Mar  1 13:05 /gluster_bricks/engine/engine/

But the package is right now not available in RHVH 4.4.5
Once this version gluster-ansible-infra is included with RHVH 4.4.5, this bug will be verified
Comment 6 SATHEESARAN 2021-03-04 07:32:47 UTC 
Tested with RHV 4.4.5 latest build - 4.4.5.7-0.1.el8ev

When creating the brick from RHV Manager UI, it fails

Snippet from logfile
---------------------
2021-03-04 07:24:42 UTC - TASK [gluster-brick-create : Create volume groups] *****************************
2021-03-04 07:24:42 UTC - 
2021-03-04 07:24:42 UTC - fatal: [rhsqa-grafton10-nic2.lab.eng.blr.redhat.com]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/libexec/platform-python"}, "changed": fal
se, "msg": "argument pvs is of type <class 'dict'> and we were unable to convert to list: <class 'dict'> cannot be converted to a list"}
2021-03-04 07:24:42 UTC - {
  "status" : "OK",
  "msg" : "",
  "data" : {
    "uuid" : "4ce7c9ac-4444-4f17-8942-b56bfd32328f",
    "counter" : 53,
    "stdout" : "fatal: [rhsqa-grafton10-nic2.lab.eng.blr.redhat.com]: FAILED! => {\"ansible_facts\": {\"discovered_interpreter_python\": \"/usr/libexec/platform-python\"}, \"changed\": false
, \"msg\": \"argument pvs is of type <class 'dict'> and we were unable to convert to list: <class 'dict'> cannot be converted to a list\"}",
    "start_line" : 52,
    "end_line" : 53,
    "runner_ident" : "8fb26200-7cba-11eb-ab5a-004855204901",
    "event" : "runner_on_failed",
    "pid" : 50108,
    "created" : "2021-03-04T07:24:42.022945",
    "parent_uuid" : "00485520-4901-513a-1b3e-000000000011",
    "event_data" : {
      "playbook" : "create-brick.yml",
      "playbook_uuid" : "a9810415-ca55-437c-9082-b07c14e0420a",
      "play" : "all",
      "play_uuid" : "00485520-4901-513a-1b3e-000000000008",
      "play_pattern" : "all",
      "task" : "Create volume groups",
      "task_uuid" : "00485520-4901-513a-1b3e-000000000011",
      "task_action" : "lvg",
      "task_args" : "",
      "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/gluster-brick-create/tasks/main.yml:36",
      "role" : "gluster-brick-create",
      "host" : "rhsqa-grafton10-nic2.lab.eng.blr.redhat.com",
      "remote_addr" : "rhsqa-grafton10-nic2.lab.eng.blr.redhat.com",
      "res" : {
        "msg" : "argument pvs is of type <class 'dict'> and we were unable to convert to list: <class 'dict'> cannot be converted to a list",
        "invocation" : {
          "module_args" : {
            "state" : "present",
            "vg" : "RHGS_vg_apollo",
            "pvs" : {
              "elements" : [ {
                "textValue" : "/dev/sde",
                "binaryValue" : "/dev/sde",
Comment 7 SATHEESARAN 2021-03-04 07:33:57 UTC 
(In reply to SATHEESARAN from comment #6)
> Tested with RHV 4.4.5 latest build - 4.4.5.7-0.1.el8ev
> 
> When creating the brick from RHV Manager UI, it fails
> 
> Snippet from logfile
> ---------------------
> 2021-03-04 07:24:42 UTC - TASK [gluster-brick-create : Create volume groups]
> *****************************
> 2021-03-04 07:24:42 UTC - 
> 2021-03-04 07:24:42 UTC - fatal:
> [rhsqa-grafton10-nic2.lab.eng.blr.redhat.com]: FAILED! => {"ansible_facts":
> {"discovered_interpreter_python": "/usr/libexec/platform-python"},
> "changed": fal
> se, "msg": "argument pvs is of type <class 'dict'> and we were unable to
> convert to list: <class 'dict'> cannot be converted to a list"}
> 2021-03-04 07:24:42 UTC - {
>   "status" : "OK",
>   "msg" : "",
>   "data" : {
>     "uuid" : "4ce7c9ac-4444-4f17-8942-b56bfd32328f",
>     "counter" : 53,
>     "stdout" : "fatal: [rhsqa-grafton10-nic2.lab.eng.blr.redhat.com]:
> FAILED! => {\"ansible_facts\": {\"discovered_interpreter_python\":
> \"/usr/libexec/platform-python\"}, \"changed\": false
> , \"msg\": \"argument pvs is of type <class 'dict'> and we were unable to
> convert to list: <class 'dict'> cannot be converted to a list\"}",
>     "start_line" : 52,
>     "end_line" : 53,
>     "runner_ident" : "8fb26200-7cba-11eb-ab5a-004855204901",
>     "event" : "runner_on_failed",
>     "pid" : 50108,
>     "created" : "2021-03-04T07:24:42.022945",
>     "parent_uuid" : "00485520-4901-513a-1b3e-000000000011",
>     "event_data" : {
>       "playbook" : "create-brick.yml",
>       "playbook_uuid" : "a9810415-ca55-437c-9082-b07c14e0420a",
>       "play" : "all",
>       "play_uuid" : "00485520-4901-513a-1b3e-000000000008",
>       "play_pattern" : "all",
>       "task" : "Create volume groups",
>       "task_uuid" : "00485520-4901-513a-1b3e-000000000011",
>       "task_action" : "lvg",
>       "task_args" : "",
>       "task_path" :
> "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/
> gluster-brick-create/tasks/main.yml:36",
>       "role" : "gluster-brick-create",
>       "host" : "rhsqa-grafton10-nic2.lab.eng.blr.redhat.com",
>       "remote_addr" : "rhsqa-grafton10-nic2.lab.eng.blr.redhat.com",
>       "res" : {
>         "msg" : "argument pvs is of type <class 'dict'> and we were unable
> to convert to list: <class 'dict'> cannot be converted to a list",
>         "invocation" : {
>           "module_args" : {
>             "state" : "present",
>             "vg" : "RHGS_vg_apollo",
>             "pvs" : {
>               "elements" : [ {
>                 "textValue" : "/dev/sde",
>                 "binaryValue" : "/dev/sde",

Apologies that I have updated the incorrect bug.
Comment 8 SATHEESARAN 2021-04-05 17:50:32 UTC 
(In reply to SATHEESARAN from comment #5)
> Verified with gluster-ansible-infra-1.0.19.el8rhgs
> 
> Post gluster deployment, gluster bricks contain the right SELinux label -
> glusterd_brick_t
> 
> [root@ ~]# ls -lZd /gluster_bricks/engine/engine/
> drwxr-xr-x. 3 vdsm kvm system_u:object_r:glusterd_brick_t:s0 24 Mar  1 13:05
> /gluster_bricks/engine/engine/
> 
> But the package is right now not available in RHVH 4.4.5
> Once this version gluster-ansible-infra is included with RHVH 4.4.5, this
> bug will be verified

Tested with gluster-ansible-infra-1.0.4-19.el8rhgs.noarch and this is included with 
RHVH 4.4.5 ISO now

[root@ ~]# ls -lZ /gluster_bricks/
total 0
drwxr-xr-x. 3 root root system_u:object_r:glusterd_brick_t:s0 18 Apr  5 17:40 data
drwxr-xr-x. 3 root root system_u:object_r:glusterd_brick_t:s0 20 Apr  5 17:40 engine
drwxr-xr-x. 3 root root system_u:object_r:glusterd_brick_t:s0 18 Apr  5 17:40 test
drwxr-xr-x. 3 root root system_u:object_r:glusterd_brick_t:s0 21 Apr  5 17:40 vmstore