Description of problem: ======================= While creating a geo-rep session , 'gluster system:: execute gsec_create' fails to create the required permissions/files required hence geo-rep fails to start on a SELINUX enabled setup (which is by default enabled) Logs - WITH SELINUX ============= [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub [root@dhcp47-160 geo-replication]# ls common_secret.pem.pub gsyncd_template.conf WITH SELINUX SET TO PERMISSIVE =============================== [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub [root@dhcp47-160 geo-replication]# ls common_secret.pem.pub gsyncd_template.conf Version-Release number of selected component (if applicable): ============================================================= glusterfs-6.0-30.el8rhgs.x86_64 How reproducible: ================= 2/2 Steps to Reproduce: 1.Create a master and a slave setup ,enable shared storage on master 2.disable performance.quick-read on SLAVE volume 3.set password less ssh from primary master node to primary slave node 4.gluster system:: execute gsec_create Actual results: =============== when selinux is enabled it fails to create some files/permissions [root@dhcp46-200 geo-replication]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 31 [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub [root@dhcp47-160 geo-replication]# ls common_secret.pem.pub gsyncd_template.conf [root@dhcp47-160 geo-replication]# because of which when i start geo-rep it is going into faulty state [root@dhcp47-160 geo-replication]# gluster volume geo-replication replica-vol 10.70.47.89::replica-vol status MASTER NODE MASTER VOL MASTER BRICK SLAVE USER SLAVE SLAVE NODE STATUS CRAWL STATUS LAST_SYNCED ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ dhcp47-160.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v1/rep1 root 10.70.47.89::replica-vol N/A Faulty N/A N/A dhcp46-200.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep2 root 10.70.47.89::replica-vol N/A Created N/A N/A dhcp47-29.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep3 root 10.70.47.89::replica-vol N/A Created N/A N/A Expected results: ================= when selinux is disabled this is working fine On Master ======== [root@dhcp47-160 geo-replication]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: permissive Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 31 [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub [root@dhcp47-160 geo-replication]# ls common_secret.pem.pub gsyncd_template.conf secret.pem secret.pem.pub tar_ssh.pem tar_ssh.pem.pub [root@dhcp47-160 geo-replication]# gluster volume geo-replication replica-vol 10.70.47.89::replica-vol status MASTER NODE MASTER VOL MASTER BRICK SLAVE USER SLAVE SLAVE NODE STATUS CRAWL STATUS LAST_SYNCED ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- dhcp47-160.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v1/rep1 root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com Active Changelog Crawl 2020-03-16 18:28:58 dhcp47-29.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep3 root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com Passive N/A N/A dhcp46-200.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep2 root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com Passive N/A N/A Additional info: ================ [root@dhcp47-160 geo-replication]# gluster v status Status of volume: gluster_shared_storage Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick dhcp47-29.lab.eng.blr.redhat.com:/var /lib/glusterd/ss_brick 49154 0 Y 29648 Brick dhcp46-200.lab.eng.blr.redhat.com:/va r/lib/glusterd/ss_brick 49154 0 Y 11582 Brick dhcp47-160.lab.eng.blr.redhat.com:/va r/lib/glusterd/ss_brick 49153 0 Y 29344 Self-heal Daemon on localhost N/A N/A Y 30450 Self-heal Daemon on dhcp47-29.lab.eng.blr.r edhat.com N/A N/A Y 30324 Self-heal Daemon on dhcp46-200.lab.eng.blr. redhat.com N/A N/A Y 12244 Task Status of Volume gluster_shared_storage ------------------------------------------------------------------------------ There are no active volume tasks Status of volume: replica-vol Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick dhcp47-160.lab.eng.blr.redhat.com:/mn t/bricks/v1/rep1 49152 0 Y 29164 Brick dhcp46-200.lab.eng.blr.redhat.com:/mn t/bricks/v2/rep2 49152 0 Y 11415 Brick dhcp47-29.lab.eng.blr.redhat.com:/mnt /bricks/v2/rep3 49152 0 Y 29481 Self-heal Daemon on localhost N/A N/A Y 30450 Self-heal Daemon on dhcp46-200.lab.eng.blr. redhat.com N/A N/A Y 12244 Self-heal Daemon on dhcp47-29.lab.eng.blr.r edhat.com N/A N/A Y 30324 Task Status of Volume replica-vol ------------------------------------------------------------------------------ There are no active volume tasks [root@dhcp47-160 geo-replication]# gluster v info Volume Name: gluster_shared_storage Type: Replicate Volume ID: eefc97be-793b-4fd6-9f33-6bfaa5f996a3 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: dhcp47-29.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick Brick2: dhcp46-200.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick Brick3: dhcp47-160.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick Options Reconfigured: performance.client-io-threads: off nfs.disable: on storage.fips-mode-rchecksum: on transport.address-family: inet cluster.enable-shared-storage: enable Volume Name: replica-vol Type: Replicate Volume ID: f06998c0-30a6-4174-8fb7-d2faed1c62b0 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: dhcp47-160.lab.eng.blr.redhat.com:/mnt/bricks/v1/rep1 Brick2: dhcp46-200.lab.eng.blr.redhat.com:/mnt/bricks/v2/rep2 Brick3: dhcp47-29.lab.eng.blr.redhat.com:/mnt/bricks/v2/rep3 Options Reconfigured: performance.client-io-threads: off nfs.disable: on storage.fips-mode-rchecksum: on transport.address-family: inet geo-replication.indexing: on geo-replication.ignore-pid-check: on changelog.changelog: on cluster.enable-shared-storage: enable [root@dhcp47-160 geo-replication]# Thanks a lot kotresh for debugging this.
(In reply to Upasana from comment #0) > Description of problem: > ======================= > While creating a geo-rep session , 'gluster system:: execute gsec_create' > fails to create the required permissions/files required hence geo-rep fails > to start > on a SELINUX enabled setup (which is by default enabled) > > Logs - > WITH SELINUX > ============= > [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create > Common secret pub file present at > /var/lib/glusterd/geo-replication/common_secret.pem.pub > [root@dhcp47-160 geo-replication]# ls > common_secret.pem.pub gsyncd_template.conf > > WITH SELINUX SET TO PERMISSIVE > =============================== [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub [root@dhcp47-160 geo-replication]# ls common_secret.pem.pub gsyncd_template.conf secret.pem secret.pem.pub tar_ssh.pem tar_ssh.pem.pub Correcting the logs here > > > Version-Release number of selected component (if applicable): > ============================================================= > glusterfs-6.0-30.el8rhgs.x86_64 > > How reproducible: > ================= > 2/2 > > Steps to Reproduce: > 1.Create a master and a slave setup ,enable shared storage on master > 2.disable performance.quick-read on SLAVE volume > 3.set password less ssh from primary master node to primary slave node > 4.gluster system:: execute gsec_create > > Actual results: > =============== > when selinux is enabled it fails to create some files/permissions > > [root@dhcp46-200 geo-replication]# sestatus > SELinux status: enabled > SELinuxfs mount: /sys/fs/selinux > SELinux root directory: /etc/selinux > Loaded policy name: targeted > Current mode: enforcing > Mode from config file: enforcing > Policy MLS status: enabled > Policy deny_unknown status: allowed > Memory protection checking: actual (secure) > Max kernel policy version: 31 > > > [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create > Common secret pub file present at > /var/lib/glusterd/geo-replication/common_secret.pem.pub > > [root@dhcp47-160 geo-replication]# ls > common_secret.pem.pub gsyncd_template.conf > [root@dhcp47-160 geo-replication]# > > > because of which when i start geo-rep it is going into faulty state > > [root@dhcp47-160 geo-replication]# gluster volume geo-replication > replica-vol 10.70.47.89::replica-vol status > > MASTER NODE MASTER VOL MASTER BRICK > SLAVE USER SLAVE SLAVE NODE STATUS CRAWL > STATUS LAST_SYNCED > ----------------------------------------------------------------------------- > ----------------------------------------------------------------------------- > -------------- > dhcp47-160.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v1/rep1 > root 10.70.47.89::replica-vol N/A Faulty N/A > N/A > dhcp46-200.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep2 > root 10.70.47.89::replica-vol N/A Created N/A > N/A > dhcp47-29.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep3 > root 10.70.47.89::replica-vol N/A Created N/A > N/A > > > Expected results: > ================= > when selinux is disabled this is working fine > > On Master > ======== > [root@dhcp47-160 geo-replication]# sestatus > SELinux status: enabled > SELinuxfs mount: /sys/fs/selinux > SELinux root directory: /etc/selinux > Loaded policy name: targeted > Current mode: permissive > Mode from config file: permissive > Policy MLS status: enabled > Policy deny_unknown status: allowed > Memory protection checking: actual (secure) > Max kernel policy version: 31 > > [root@dhcp47-160 geo-replication]# gluster system:: execute gsec_create > Common secret pub file present at > /var/lib/glusterd/geo-replication/common_secret.pem.pub > > > [root@dhcp47-160 geo-replication]# ls > common_secret.pem.pub gsyncd_template.conf secret.pem secret.pem.pub > tar_ssh.pem tar_ssh.pem.pub > > [root@dhcp47-160 geo-replication]# gluster volume geo-replication > replica-vol 10.70.47.89::replica-vol status > > MASTER NODE MASTER VOL MASTER BRICK > SLAVE USER SLAVE SLAVE NODE > STATUS CRAWL STATUS LAST_SYNCED > ----------------------------------------------------------------------------- > ----------------------------------------------------------------------------- > ------------------------------------------------ > dhcp47-160.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v1/rep1 > root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com > Active Changelog Crawl 2020-03-16 18:28:58 > dhcp47-29.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep3 > root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com > Passive N/A N/A > dhcp46-200.lab.eng.blr.redhat.com replica-vol /mnt/bricks/v2/rep2 > root 10.70.47.89::replica-vol dhcp47-175.lab.eng.blr.redhat.com > Passive N/A N/A > > > > Additional info: > ================ > [root@dhcp47-160 geo-replication]# gluster v status > Status of volume: gluster_shared_storage > Gluster process TCP Port RDMA Port Online Pid > ----------------------------------------------------------------------------- > - > Brick dhcp47-29.lab.eng.blr.redhat.com:/var > /lib/glusterd/ss_brick 49154 0 Y > 29648 > Brick dhcp46-200.lab.eng.blr.redhat.com:/va > r/lib/glusterd/ss_brick 49154 0 Y > 11582 > Brick dhcp47-160.lab.eng.blr.redhat.com:/va > r/lib/glusterd/ss_brick 49153 0 Y > 29344 > Self-heal Daemon on localhost N/A N/A Y > 30450 > Self-heal Daemon on dhcp47-29.lab.eng.blr.r > edhat.com N/A N/A Y > 30324 > Self-heal Daemon on dhcp46-200.lab.eng.blr. > redhat.com N/A N/A Y > 12244 > > Task Status of Volume gluster_shared_storage > ----------------------------------------------------------------------------- > - > There are no active volume tasks > > Status of volume: replica-vol > Gluster process TCP Port RDMA Port Online Pid > ----------------------------------------------------------------------------- > - > Brick dhcp47-160.lab.eng.blr.redhat.com:/mn > t/bricks/v1/rep1 49152 0 Y > 29164 > Brick dhcp46-200.lab.eng.blr.redhat.com:/mn > t/bricks/v2/rep2 49152 0 Y > 11415 > Brick dhcp47-29.lab.eng.blr.redhat.com:/mnt > /bricks/v2/rep3 49152 0 Y > 29481 > Self-heal Daemon on localhost N/A N/A Y > 30450 > Self-heal Daemon on dhcp46-200.lab.eng.blr. > redhat.com N/A N/A Y > 12244 > Self-heal Daemon on dhcp47-29.lab.eng.blr.r > edhat.com N/A N/A Y > 30324 > > Task Status of Volume replica-vol > ----------------------------------------------------------------------------- > - > There are no active volume tasks > > [root@dhcp47-160 geo-replication]# gluster v info > > Volume Name: gluster_shared_storage > Type: Replicate > Volume ID: eefc97be-793b-4fd6-9f33-6bfaa5f996a3 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x 3 = 3 > Transport-type: tcp > Bricks: > Brick1: dhcp47-29.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick > Brick2: dhcp46-200.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick > Brick3: dhcp47-160.lab.eng.blr.redhat.com:/var/lib/glusterd/ss_brick > Options Reconfigured: > performance.client-io-threads: off > nfs.disable: on > storage.fips-mode-rchecksum: on > transport.address-family: inet > cluster.enable-shared-storage: enable > > Volume Name: replica-vol > Type: Replicate > Volume ID: f06998c0-30a6-4174-8fb7-d2faed1c62b0 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x 3 = 3 > Transport-type: tcp > Bricks: > Brick1: dhcp47-160.lab.eng.blr.redhat.com:/mnt/bricks/v1/rep1 > Brick2: dhcp46-200.lab.eng.blr.redhat.com:/mnt/bricks/v2/rep2 > Brick3: dhcp47-29.lab.eng.blr.redhat.com:/mnt/bricks/v2/rep3 > Options Reconfigured: > performance.client-io-threads: off > nfs.disable: on > storage.fips-mode-rchecksum: on > transport.address-family: inet > geo-replication.indexing: on > geo-replication.ignore-pid-check: on > changelog.changelog: on > cluster.enable-shared-storage: enable > [root@dhcp47-160 geo-replication]# > > > > Thanks a lot kotresh for debugging this.
Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/24433/
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2572