Hide Forgot
Description of problem: ======================= While trying to do following scenario: 1. Untar linux kernel from mount "for i in {1..10}; do gzcat < linux-2.6.5.tar.xz | tar xvf - -C o$i & done" 2. Poweroff and On of one of the server in Replicate setup (1*2) in interval of 1800 secs "for i in {1..10} ; do sleep 1800 ; virsh destroy boost ; sleep 1800 ; virsh start boost ; done &" 3. cp from the client from untar directory to another directory. "for i in {1..5}; do sleep 1800; cp -i -rf o$i b$i; sleep 1800; done &" Note: Step 2 was restarted again when it finished once. cp failed with error: cp: cannot stat `o2/linux-2.6.5/drivers/usb/misc/rio500_usb.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/usb/misc/emi62_fw_s.h': No such file or directory cp: cannot open `o2/linux-2.6.5/drivers/usb/misc/Makefile' for reading: No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/usb/misc/emi62.c': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/usb/class/cdc-acm.c': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/usb/net/kawethfw.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/usb/host/uhci-hcd.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/avm/Kconfig': No such file or directory cp: cannot open `o2/linux-2.6.5/drivers/isdn/hardware/eicon/dbgioctl.h' for reading: No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/di.c': No such file or directory cp: cannot open `o2/linux-2.6.5/drivers/isdn/hardware/eicon/maintidi.h' for reading: No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/capidtmf.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/os_4bri.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/divasproc.c': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/pc_maint.h': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hardware/eicon/dqueue.h': No such file or directory cp: cannot open `o2/linux-2.6.5/drivers/isdn/i4l/isdn_v110.h' for reading: No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/i4l/isdn_concap.c': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hisax/hscx.c': No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hisax/hisax_cfg.h': No such file or directory cp: cannot open `o2/linux-2.6.5/drivers/isdn/hisax/fsm.c' for reading: No such file or directory cp: cannot stat `o2/linux-2.6.5/drivers/isdn/hisax/isar.h': No such file or directory Client logs: ============ [2013-06-03 10:17:43.151949] W [fuse-bridge.c:725:fuse_fd_cbk] 0-glusterfs-fuse: 19071697: OPEN() /o2/linux-2.6.5/drivers/isdn/hardware/eicon/dbgioctl.h => -1 (No such file or directory) [2013-06-03 10:18:17.382915] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-0: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/hardware/eicon/maintidi.h (3c7c0d92-ae2a-41a0-b8b8-194b64bf1d4b) [2013-06-03 10:18:17.392677] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-1: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/hardware/eicon/maintidi.h (3c7c0d92-ae2a-41a0-b8b8-194b64bf1d4b) [2013-06-03 10:18:17.392703] W [fuse-bridge.c:725:fuse_fd_cbk] 0-glusterfs-fuse: 19100261: OPEN() /o2/linux-2.6.5/drivers/isdn/hardware/eicon/maintidi.h => -1 (No such file or directory) [2013-06-03 10:20:26.124005] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-0: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/i4l/isdn_v110.h (baa77c4b-f574-4a1b-8ed9-d27b081ea485) [2013-06-03 10:20:26.155556] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-1: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/i4l/isdn_v110.h (baa77c4b-f574-4a1b-8ed9-d27b081ea485) [2013-06-03 10:20:26.155645] W [fuse-bridge.c:725:fuse_fd_cbk] 0-glusterfs-fuse: 19185494: OPEN() /o2/linux-2.6.5/drivers/isdn/i4l/isdn_v110.h => -1 (No such file or directory) [2013-06-03 10:21:15.536131] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-1: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/hisax/fsm.c (0a54305a-2520-4d19-adcc-23d14e560d68) [2013-06-03 10:21:15.560464] W [client3_1-fops.c:473:client3_1_open_cbk] 0-intuit-client-0: remote operation failed: No such file or directory. Path: /o2/linux-2.6.5/drivers/isdn/hisax/fsm.c (0a54305a-2520-4d19-adcc-23d14e560d68) [2013-06-03 10:21:15.560503] W [fuse-bridge.c:725:fuse_fd_cbk] 0-glusterfs-fuse: 19222062: OPEN() /o2/linux-2.6.5/drivers/isdn/hisax/fsm.c => -1 (No such file or directory) Getting gfid for one of the file "fsm.c" [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/fsm.c getfattr: Removing leading '/' from absolute path names # file: rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/fsm.c trusted.afr.intuit-client-0=0x000000000000000000000000 trusted.afr.intuit-client-1=0x000000000000000000000000 trusted.gfid=0x9336e845f8e34564b4b5af7aae2cd079 [root@jay bin]# Comparing the gfid confirms that the open happened on "0a54305a-2520-4d19-adcc-23d14e560d68" and current gfid is "0x9336e845f8e34564b4b5af7aae2cd079" [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/misc/rio500_usb.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x67c2b23017e0415a91e9636b192a7333 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/misc/emi62_fw_s.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x414cddabf9224671a5b25902f39b138e [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/misc/Makefile | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x4edcf61456bb421c9ca8e08b754efae2 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/misc/emi62.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x12eebde8832a47aeb18300e2b7a37158 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/class/cdc-acm.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xfcde67ec7cf3404989bc21b2ceb28838 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/net/kawethfw.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x147d0f9993d54a2a8aefcb79957ed054 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/usb/host/uhci-hcd.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xbb84f4e05cf346d2ab7a1bc8a07d70a4 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/avm/Kconfig | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xdc09978da65f4fab8c60d698ee148d62 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/dbgioctl.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xf8e28bea175446cf91ca6d8d81b4f8ce [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/di.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xbffd4385f3b14861960791e40f37a525 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/maintidi.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x83807d66ba654299beb59a79e449b901 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/capidtmf.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xe0b6c73a07ca41049485b273c98b95fe [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/os_4bri.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x267aa60365fb4d8f9d5ccfbca7536721 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/divasproc.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x36f3f402f7d740cf94b03fff09f23863 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/pc_maint.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xdd6f65aa26e7477491b7dbe017c15353 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hardware/eicon/dqueue.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x58b8f21da804478d8cc1c5e6f7d0631b [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/i4l/isdn_v110.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x0cbda8006d614d348c32902ebac71356 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/i4l/isdn_concap.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xe2e1a98e4f14413e91850e03506a19d1 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/hscx.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xb2c3e934580e4db2b5932525aae83d9c [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/hisax_cfg.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x93289ab71f4541108d42408850a2d550 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/fsm.c | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0x9336e845f8e34564b4b5af7aae2cd079 [root@jay bin]# getfattr -d -m . -e hex /rhs/brick1/o2/linux-2.6.5/drivers/isdn/hisax/isar.h | grep "trusted.gfid" getfattr: Removing leading '/' from absolute path names trusted.gfid=0xe463d5a264d34e389c0b18bbf20d579f [root@jay bin]# Version-Release number of selected component (if applicable): ============================================================= [root@jay bin]# rpm -qa | grep gluster | grep 3.3.0 glusterfs-devel-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-server-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-geo-replication-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-debuginfo-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-fuse-3.3.0.10rhs-1.el6rhs.x86_64 glusterfs-rdma-3.3.0.10rhs-1.el6rhs.x86_64 [root@jay bin]# How reproducible: ================= In multiple tries
Amar, it does not seem like a bug. Let me know if you agree we shall close this as NOTABUG. Tar replaces the file when the same archive is untared multiple times. So if a file is replaced between 'lookup' fop and 'open' fop then it will fail with ENOENT. Experiment: root - /mnt/r2 10:17:03 :) ⚡ getfattr -d -m. -e hex /home/gfs/r2_?/3.3_sh getfattr: Removing leading '/' from absolute path names # file: home/gfs/r2_0/3.3_sh trusted.afr.r2-client-0=0x000000000000000000000000 trusted.afr.r2-client-1=0x000000000000000000000000 trusted.gfid=0x0f59e5e0c1c44c1db61c617eba34a924 # file: home/gfs/r2_1/3.3_sh trusted.afr.r2-client-0=0x000000000000000000000000 trusted.afr.r2-client-1=0x000000000000000000000000 trusted.gfid=0x0f59e5e0c1c44c1db61c617eba34a924 root - /mnt/r2 10:17:37 :) ⚡ ls 3.3_sh onefile.tar.gz root - /mnt/r2 10:17:39 :) ⚡ tar xvzf onefile.tar.gz 3.3_sh root - /mnt/r2 10:17:42 :) ⚡ getfattr -d -m. -e hex /home/gfs/r2_?/3.3_sh getfattr: Removing leading '/' from absolute path names # file: home/gfs/r2_0/3.3_sh trusted.afr.r2-client-0=0x000000000000000000000000 trusted.afr.r2-client-1=0x000000000000000000000000 trusted.gfid=0x899d1f7de8884a7382ff73e266bc25a8 # file: home/gfs/r2_1/3.3_sh trusted.afr.r2-client-0=0x000000000000000000000000 trusted.afr.r2-client-1=0x000000000000000000000000 trusted.gfid=0x899d1f7de8884a7382ff73e266bc25a8 root - /mnt/r2 10:17:44 :) ⚡ ls 3.3_sh onefile.tar.gz
Comment#3. Debatable on whether to call it NOTABUG or WONTFIX. Keeping it WONTFIX for now... If someone feels its better classified in NOTABUG, feel free to do so. Rahul, Please re-look at test case, and it may always result in NOTABUG.