Description of problem: ======================== While trying to create the hardlink of a file from the fuse mount, it fails to create with reason "no such file and directory". But with the retrial of the link creation it is successful. For Example: ============ 1. Failed to create the hardlink in the first attempt: [root@wingo vol0]# ls host.conf hosts hosts.allow hosts.deny [root@wingo vol0]# [root@wingo vol0]# ln host.conf a ln: creating hard link `a' => `host.conf': No such file or directory [root@wingo vol0]# 2. But Subsequent attempts are successful. [root@wingo vol0]# ln host.conf a [root@wingo vol0]# [root@wingo vol0]# ls a host.conf hosts hosts.allow hosts.deny [root@wingo vol0]# Version-Release number of selected component (if applicable): ============================================================= glusterfs-3.4.1.snap.jan15.2014git-1.el6.x86_64 How reproducible: ================= Always on my configuration Steps to Reproduce: =================== 1. Create a hardlink of a file Actual results: =============== Failed to create hard links from the fuse mount for first time and subsequent attempts are successful Expected results: ================= Hardlink should be successful. Additional info: ================ Absolutely no logs were logged in any of the gluster logs or dmesg. Strace of one such attempt is as follows: [root@wingo vol0]# strace ln hosts.deny r execve("/bin/ln", ["ln", "hosts.deny", "r"], [/* 36 vars */]) = 0 brk(0) = 0x12bf000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc729dcc000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=40754, ...}) = 0 mmap(NULL, 40754, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc729dc2000 close(3) = 0 open("/lib64/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\241~3\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1922152, ...}) = 0 mmap(0x337ea00000, 3745960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x337ea00000 mprotect(0x337eb8a000, 2093056, PROT_NONE) = 0 mmap(0x337ed89000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x189000) = 0x337ed89000 mmap(0x337ed8e000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x337ed8e000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc729dc1000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc729dc0000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc729dbf000 arch_prctl(ARCH_SET_FS, 0x7fc729dc0700) = 0 mprotect(0x337ed89000, 16384, PROT_READ) = 0 mprotect(0x337e41f000, 4096, PROT_READ) = 0 munmap(0x7fc729dc2000, 40754) = 0 brk(0) = 0x12bf000 brk(0x12e0000) = 0x12e0000 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=99158576, ...}) = 0 mmap(NULL, 99158576, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc723f2e000 close(3) = 0 stat("r", 0x7fff9ba1e460) = -1 ENOENT (No such file or directory) lstat("hosts.deny", {st_mode=S_IFREG|0644, st_size=460, ...}) = 0 linkat(AT_FDCWD, "hosts.deny", AT_FDCWD, "r", 0) = -1 ENOENT (No such file or directory) open("/usr/share/locale/locale.alias", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=2512, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc729dcb000 read(3, "# Locale name alias data base.\n#"..., 4096) = 2512 read(3, "", 4096) = 0 close(3) = 0 munmap(0x7fc729dcb000, 4096) = 0 open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=435, ...}) = 0 mmap(NULL, 435, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc729dcb000 close(3) = 0 write(2, "ln: ", 4ln: ) = 4 write(2, "creating hard link `r' => `hosts"..., 38creating hard link `r' => `hosts.deny') = 38 open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) write(2, ": No such file or directory", 27: No such file or directory) = 27 write(2, "\n", 1 ) = 1 close(0) = 0 close(1) = 0 close(2) = 0 exit_group(1) = ? [root@wingo vol0]#
Not happening in all the setups. In the setup where the bug is found, dropping the cache solves the issue. [root@wingo vol0]# ls host.conf hosts hosts.allow hosts.deny [root@wingo vol0]# rm -f * [root@wingo vol0]# cp -rf /etc/host* . [root@wingo vol0]# ls host.conf hosts hosts.allow hosts.deny [root@wingo vol0]# cd [root@wingo ~]# echo 3 > /proc/sys/vm/drop_caches [root@wingo ~]# cd - /mnt/vol0 [root@wingo vol0]# ln host.conf a [root@wingo vol0]# ls a host.conf hosts hosts.allow hosts.deny [root@wingo vol0]# ln hosts.allow b Need some more debugging. Because when gdb was attached to glusterfs and break point was set to fuse_link and hard link operation was done, the breakpoint was not hit and the application had got ENOENT (means the kernel itself had sent the error to application, instead of sending the fop to glusterfs). But when caches were dropped, break point was hit and hard link got created without any error.
The issue looks same as bug# 1024459 (Unable to create hardlinks when the source and destination are in the same directory). So marking the bug as dup of bug# 1024459 *** This bug has been marked as a duplicate of bug 1024459 ***
Giving devel ack to snapshot bugz
Marking snapshot BZs to RHS 3.0.