Bug 1264334 - DHT:3.1.1 Gluster fails to create special files on Fuse mount
DHT:3.1.1 Gluster fails to create special files on Fuse mount
Status: CLOSED NOTABUG
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute (Show other bugs)
3.1
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Bug Updates Notification Mailing List
storage-qa-internal@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-18 04:42 EDT by Amit Chaurasia
Modified: 2015-10-27 20:10 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-18 05:33:08 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Amit Chaurasia 2015-09-18 04:42:20 EDT
Document URL: 

Section Number and Name: 

Describe the issue: 

Creation of special files like device files or named pipes fails on Fuse mount while the same file can be created on the backend brick.

[root@dht-rhs-24 test_blk_char_dev]# cd /mnt/glusterfs/blk_char_dev/
[root@dht-rhs-24 blk_char_dev]# mknod MYFIFO p
mknod: ‘MYFIFO’: Permission denied
[root@dht-rhs-24 blk_char_dev]# mkfifo a=rw myfifo
mkfifo: cannot create fifo ‘a=rw’: Permission denied
mkfifo: cannot create fifo ‘myfifo’: Permission denied
[root@dht-rhs-24 blk_char_dev]# 
[root@dht-rhs-24 blk_char_dev]# pwd
/mnt/glusterfs/blk_char_dev
[root@dht-rhs-24 blk_char_dev]# 

[root@dht-rhs-24 blk_char_dev]# mount | grep gluster
localhost:testvol on /run/gluster/testvol type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.47.114:/testvol on /mnt/glusterfs type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
[root@dht-rhs-24 blk_char_dev]# 

=== 

[root@dht-rhs-24 test_blk_char_dev]# mkfifo a=rw myfifo
[root@dht-rhs-24 test_blk_char_dev]# ls -lthr
total 0
brwxrwxrwx. 1 root root 13, 42 Sep 18 18:15 block_200674_glusterfs
crwxrwxrwx. 1 root root 14, 43 Sep 18 18:15 char_200674_glusterfs
prw-r--r--. 1 root root      0 Sep 18 19:06 MYFIFO
prw-r--r--. 1 root root      0 Sep 18 19:06 myfifo
prw-r--r--. 1 root root      0 Sep 18 19:06 a=rw
[root@dht-rhs-24 test_blk_char_dev]# ls -lthr
total 0
brwxrwxrwx. 1 root root 13, 42 Sep 18 18:15 block_200674_glusterfs
crwxrwxrwx. 1 root root 14, 43 Sep 18 18:15 char_200674_glusterfs
prw-r--r--. 1 root root      0 Sep 18 19:06 MYFIFO
prw-r--r--. 1 root root      0 Sep 18 19:06 myfifo
prw-r--r--. 1 root root      0 Sep 18 19:06 a=rw
[root@dht-rhs-24 test_blk_char_dev]# 
[root@dht-rhs-24 test_blk_char_dev]# ls -l
total 0
prw-r--r--. 1 root root      0 Sep 18 19:06 a=rw
brwxrwxrwx. 1 root root 13, 42 Sep 18 18:15 block_200674_glusterfs
crwxrwxrwx. 1 root root 14, 43 Sep 18 18:15 char_200674_glusterfs
prw-r--r--. 1 root root      0 Sep 18 19:06 myfifo
prw-r--r--. 1 root root      0 Sep 18 19:06 MYFIFO
[root@dht-rhs-24 test_blk_char_dev]# pwd
/root/test_blk_char_dev

=============

Messages in the log files and the attempt to create special files in the backend on the same brick:


[root@dht-rhs-24 bricks]# grep -i mknod *
bricks-brick2-testvol.log:[2015-09-18 12:40:18.761987] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:18.762078] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62963: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:21.467720] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62971: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:21.467653] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:44:55.901827] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:44:55.901946] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62980: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
[root@dht-rhs-24 bricks]# 

[root@dht-rhs-24 bricks]# pwd
/var/log/glusterfs/bricks

[root@dht-rhs-24 bricks]# cd /bricks/brick2/testvol/blk_char_dev/

[root@dht-rhs-24 blk_char_dev]# #mknod char_200674_glusterfs c 14 43

[root@dht-rhs-24 blk_char_dev]# pwd
/bricks/brick2/testvol/blk_char_dev

[root@dht-rhs-24 blk_char_dev]# 

[root@dht-rhs-24 blk_char_dev]# mknod char_200674_glusterfs c 14 43

[root@dht-rhs-24 blk_char_dev]# ls -ltrh
total 0
crw-r--r--. 1 root root 14, 43 Sep 18 18:22 char_200674_glusterfs

[root@dht-rhs-24 blk_char_dev]# mknod block_200674_glusterfs b 13 42

[root@dht-rhs-24 blk_char_dev]# ls -ltrh
total 0
crw-r--r--. 1 root root 14, 43 Sep 18 18:22 char_200674_glusterfs
brw-r--r--. 1 root root 13, 42 Sep 18 18:23 block_200674_glusterfs

[root@dht-rhs-24 blk_char_dev]# cd -
/var/log/glusterfs/bricks

[root@dht-rhs-24 bricks]# pwd
/var/log/glusterfs/bricks

[root@dht-rhs-24 bricks]# grep -i mknod *
bricks-brick2-testvol.log:[2015-09-18 12:40:18.761987] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:18.762078] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62963: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:21.467720] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62971: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:40:21.467653] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:44:55.901827] E [MSGID: 113022] [posix.c:1165:posix_mknod] 0-testvol-posix: mknod on /bricks/brick2/testvol/blk_char_dev/block_200674_glusterfs failed [Permission denied]
bricks-brick2-testvol.log:[2015-09-18 12:44:55.901946] E [MSGID: 115057] [server-rpc-fops.c:557:server_mknod_cbk] 0-testvol-server: 62980: MKNOD /blk_char_dev/block_200674_glusterfs (e3b98157-3106-4a9d-a15d-dd1f760c46e4/block_200674_glusterfs) ==> (Permission denied) [Permission denied]
[root@dht-rhs-24 bricks]# 

[root@dht-rhs-24 blk_char_dev]# rpm -qa  |grep -i gluster
glusterfs-rdma-3.7.1-15.el7rhgs.x86_64
python-gluster-3.7.1-15.el7rhgs.x86_64
vdsm-gluster-4.16.20-1.3.el7rhgs.noarch
glusterfs-fuse-3.7.1-15.el7rhgs.x86_64
glusterfs-server-3.7.1-15.el7rhgs.x86_64
glusterfs-client-xlators-3.7.1-15.el7rhgs.x86_64
glusterfs-geo-replication-3.7.1-15.el7rhgs.x86_64
gluster-nagios-addons-0.2.5-1.el7rhgs.x86_64
glusterfs-3.7.1-15.el7rhgs.x86_64
glusterfs-api-3.7.1-15.el7rhgs.x86_64
glusterfs-libs-3.7.1-15.el7rhgs.x86_64
glusterfs-cli-3.7.1-15.el7rhgs.x86_64
gluster-nagios-common-0.2.2-1.el7rhgs.noarch
[root@dht-rhs-24 blk_char_dev]# 




[root@dht-rhs-24 blk_char_dev]# gluster v info
 
Volume Name: testvol
Type: Distributed-Replicate
Volume ID: 1f9b8551-5a26-4566-9f1b-4725b8c93efc
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.47.114:/bricks/brick0/testvol
Brick2: 10.70.47.174:/bricks/brick0/testvol
Brick3: 10.70.47.114:/bricks/brick1/testvol
Brick4: 10.70.47.174:/bricks/brick1/testvol
Brick5: 10.70.47.114:/bricks/brick2/testvol
Brick6: 10.70.47.174:/bricks/brick2/testvol
Options Reconfigured:
performance.quick-read: on
cluster.lookup-optimize: on
cluster.min-free-disk: 90%
features.quota-deem-statfs: on
features.inode-quota: on
features.quota: on
performance.readdir-ahead: on
[root@dht-rhs-24 blk_char_dev]# 


[root@dht-rhs-24 blk_char_dev]# gluster volume status testvol 
Status of volume: testvol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.47.114:/bricks/brick0/testvol   49152     0          Y       14376
Brick 10.70.47.174:/bricks/brick0/testvol   49152     0          Y       14362
Brick 10.70.47.114:/bricks/brick1/testvol   49153     0          Y       14603
Brick 10.70.47.174:/bricks/brick1/testvol   49153     0          Y       14524
Brick 10.70.47.114:/bricks/brick2/testvol   49154     0          Y       14722
Brick 10.70.47.174:/bricks/brick2/testvol   49154     0          Y       14625
NFS Server on localhost                     2049      0          Y       14644
Self-heal Daemon on localhost               N/A       N/A        Y       14652
Quota Daemon on localhost                   N/A       N/A        Y       14661
NFS Server on 10.70.47.114                  2049      0          Y       14747
Self-heal Daemon on 10.70.47.114            N/A       N/A        Y       14755
Quota Daemon on 10.70.47.114                N/A       N/A        Y       14765
 
Task Status of Volume testvol
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@dht-rhs-24 blk_char_dev]# 


Cluster nodes:
10.70.47.114
10.70.47.174

Login:
root/<Usual password>
Comment 2 Amit Chaurasia 2015-09-18 05:05:55 EDT
Strace info:

[root@dht-rhs-24 blk_char_dev]# strace mknod MYFIFO p
execve("/usr/bin/mknod", ["mknod", "MYFIFO", "p"], [/* 30 vars */]) = 0
brk(0)                                  = 0x13e5000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1bb000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=35047, ...}) = 0
mmap(NULL, 35047, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdb9b1b2000
close(3)                                = 0
open("/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240d\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=147096, ...}) = 0
mmap(NULL, 2246808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9ad76000
mprotect(0x7fdb9ad97000, 2097152, PROT_NONE) = 0
mmap(0x7fdb9af97000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21000) = 0x7fdb9af97000
mmap(0x7fdb9af99000, 6296, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdb9af99000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\34\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2107760, ...}) = 0
mmap(NULL, 3932736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9a9b5000
mprotect(0x7fdb9ab6b000, 2097152, PROT_NONE) = 0
mmap(0x7fdb9ad6b000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7fdb9ad6b000
mmap(0x7fdb9ad71000, 16960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdb9ad71000
close(3)                                = 0
open("/lib64/libpcre.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\25\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=398272, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1b1000
mmap(NULL, 2490888, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9a754000
mprotect(0x7fdb9a7b3000, 2097152, PROT_NONE) = 0
mmap(0x7fdb9a9b3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5f000) = 0x7fdb9a9b3000
close(3)                                = 0
open("/lib64/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000/\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=153184, ...}) = 0
mmap(NULL, 2245240, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9a52f000
mprotect(0x7fdb9a553000, 2093056, PROT_NONE) = 0
mmap(0x7fdb9a752000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x23000) = 0x7fdb9a752000
close(3)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=19512, ...}) = 0
mmap(NULL, 2109744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9a32b000
mprotect(0x7fdb9a32e000, 2093056, PROT_NONE) = 0
mmap(0x7fdb9a52d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fdb9a52d000
close(3)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240l\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=141616, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1b0000
mmap(NULL, 2208864, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdb9a10f000
mprotect(0x7fdb9a125000, 2097152, PROT_NONE) = 0
mmap(0x7fdb9a325000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7fdb9a325000
mmap(0x7fdb9a327000, 13408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdb9a327000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1af000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1ad000
arch_prctl(ARCH_SET_FS, 0x7fdb9b1ad800) = 0
mprotect(0x7fdb9ad6b000, 16384, PROT_READ) = 0
mprotect(0x7fdb9a325000, 4096, PROT_READ) = 0
mprotect(0x7fdb9a52d000, 4096, PROT_READ) = 0
mprotect(0x7fdb9a752000, 4096, PROT_READ) = 0
mprotect(0x7fdb9a9b3000, 4096, PROT_READ) = 0
mprotect(0x7fdb9af97000, 4096, PROT_READ) = 0
mprotect(0x60e000, 4096, PROT_READ)     = 0
mprotect(0x7fdb9b1bc000, 4096, PROT_READ) = 0
munmap(0x7fdb9b1b2000, 35047)           = 0
set_tid_address(0x7fdb9b1adad0)         = 29730
set_robust_list(0x7fdb9b1adae0, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fdb9a115780, [], SA_RESTORER|SA_SIGINFO, 0x7fdb9a11e130}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fdb9a115810, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fdb9a11e130}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", {f_type=0xf97cff8c, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
statfs("/sys/fs/selinux", {f_type=0xf97cff8c, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
stat("/sys/fs/selinux", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
brk(0)                                  = 0x13e5000
brk(0x1406000)                          = 0x1406000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=106065056, ...}) = 0
mmap(NULL, 106065056, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdb93be8000
close(3)                                = 0
mknod("MYFIFO", S_IFIFO|0666)           = -1 EACCES (Permission denied)
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2502, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdb9b1ba000
read(3, "# Locale name alias data base.\n#"..., 4096) = 2502
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7fdb9b1ba000, 4096)            = 0
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/charset.alias", O_RDONLY|O_NOFOLLOW) = -1 ENOENT (No such file or directory)
write(2, "mknod: ", 7mknod: )                  = 7
write(2, "\342\200\230MYFIFO\342\200\231", 12‘MYFIFO’) = 12
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": Permission denied", 19: Permission denied)     = 19
write(2, "\n", 1
)                       = 1
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++
[root@dht-rhs-24 blk_char_dev]# 

Saving logs and sosreports @http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1264334
Comment 3 Amit Chaurasia 2015-09-18 05:19:41 EDT
I tried this on another cluster and there it worked fine.

Looks like this is some SELinux permission issue as I see avc failures in the log files while running this test on beaker.

Here is a snippet of one error message:

Info: Searching AVC errors produced since 1442399189.96 (Wed Sep 16 06:26:29 2015)
Searching logs...
Running '/usr/bin/env LC_ALL=en_US.UTF-8 /sbin/ausearch -m AVC -m USER_AVC -m SELINUX_ERR -ts 09/16/2015 06:26:29 < /dev/null >/mnt/testarea/tmp.rhts-db-submit-result.wVDnmS 2>&1'
----
time->Wed Sep 16 06:26:55 2015
type=SYSCALL msg=audit(1442399215.522:14331): arch=c000003e syscall=133 success=no exit=-13 a0=7f3214edb770 a1=61a4 a2=d2a a3=7f3229092ac0 items=0 ppid=1 pid=17394 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="glusterfsd" exe="/usr/sbin/glusterfsd" subj=system_u:system_r:glusterd_t:s0 key=(null)
type=AVC msg=audit(1442399215.522:14331): avc:  denied  { create } for  pid=17394 comm="glusterfsd" name="block_200674_nfs" scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=blk_file
Fail: AVC messages found.
Checking for errors...
Using stronger AVC checks.
	Define empty RHTS_OPTION_STRONGER_AVC parameter if this causes any problems.
Running 'cat /mnt/testarea/tmp.rhts-db-submit-result.wVDnmS | /sbin/ausearch -m AVC -m SELINUX_ERR'
Fail: AVC messages found.
Running 'cat %s | /sbin/ausearch -m USER_AVC >/mnt/testarea/tmp.rhts-db-submit-result.c3G2a3 2>&1'
Info: No AVC messages found.
/bin/grep 'avc: ' /mnt/testarea/dmesg.log | /bin/grep --invert-match TESTOUT.log
No AVC messages found in dmesg
Running '/usr/sbin/sestatus'
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28
Running 'rpm -q selinux-policy || true'
selinux-policy-3.13.1-23.el7_1.18.noarch


The job link of the test on beaker:

https://beaker.engineering.redhat.com/jobs/1083777
Comment 4 Amit Chaurasia 2015-09-18 05:33:08 EDT
This is essentially a SELinux policy issue.
I changed the context and it worked. Closing the bug. The test scripts need to be modified to setup the right SE Linux context.

Note You need to log in before you can comment on or make changes to this bug.