The time stamp of the files/dirs changes when one executes a snapshot restore, resulting in a failure to read the appropriate changelogs. 'Glusterfind pre' fails with the following error: 'historical changelogs not available'
Existing glusterfind sessions fail to work after a snapshot restore
Workaround:
Gather the necessary information from existing glusterfind sessions, remove the sessions, perform a snapshot restore, and then create new glusterfind sessions.
Feel free to open this bug if the issue still persists and you require a fix. Closing this as WONTFIX as we are not working on this bug, and treating it as a 'TIMEOUT'.
Description of problem: Had a 2*3 distribute replicate volume, on a 4node cluster, and created a dataset at nfs/fuse mountpoint. Did creates/modifies/renames/deletes from root/non-root user and executed glusterfind pre and post to verify the output file. Created a snapshot 'snap1' and did a couple of more modifies/deletes and ran glusterfind pre and post. Stopped teh volume, did a snap restore to 'snap1' and restarted the volume. Ran the glusterfind pre command to verify if the changes were recorded in the output file and that is when it fails with the below error: [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt --regenerate-outfile 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: [2015-06-26 16:39:34.447049] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-06-26 16:39:34.447189] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 0] Success Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# Version-Release number of selected component (if applicable): glusterfs-3.7.1-5.el6rhs.x86_64 How reproducible: 1:1 Steps to Reproduce: 1. Have a >2 node cluster, with distribute replicate volume and glusterfind sessions created for teh same. 2. Have data set at teh mountpoint, create a snapshot, and do some changes on teh dataset. 3. Run glusterfind pre and post and verify the output file 4. Perform a snapshot restore and run glusterfind pre again 5. Create a new glusterfind session and run 'pre' Actual results: Step4 and step5 fail with errors 'changelog not available' and generate an empty output file Expected results: Existing glusterfind sessions should succeed. If we are planning to note it in the doc, at least newly created glusterfind sessions (step5) should successfully be able to log in changes to the output file. Present behaviour would make the glusterfind API to not function at all for a volume, IF a snap restore is done. There needs to be a work around to make the API work with the volume once it lands up in this situation. Pasted below are the detailed logs: ############## SERVER ################# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v list gluster_shared_storage [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v create cthree replica 3 10.70.43.191:/rhs/thinbrick1/cthree 10.70.42.202:/rhs/thinbrick1/cthree 10.70.42.30:/rhs/thinbrick1/cthree 10.70.43.191:/rhs/thinbrick2/cthree 10.70.42.202:/rhs/thinbrick2/cthree 10.70.42.30:/rhs/thinbrick2/cthree volume create: cthree: success: please start the volume to access data [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v info cthree Volume Name: cthree Type: Distributed-Replicate Volume ID: 1aea3d6b-d0fd-45cb-b9cb-da44e2c4c780 Status: Created Number of Bricks: 2 x 3 = 6 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/thinbrick1/cthree Brick2: 10.70.42.202:/rhs/thinbrick1/cthree Brick3: 10.70.42.30:/rhs/thinbrick1/cthree Brick4: 10.70.43.191:/rhs/thinbrick2/cthree Brick5: 10.70.42.202:/rhs/thinbrick2/cthree Brick6: 10.70.42.30:/rhs/thinbrick2/cthree Options Reconfigured: performance.readdir-ahead: on [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v start cthree volume start: cthree: success [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v info cthree Volume Name: cthree Type: Distributed-Replicate Volume ID: 1aea3d6b-d0fd-45cb-b9cb-da44e2c4c780 Status: Started Number of Bricks: 2 x 3 = 6 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/thinbrick1/cthree Brick2: 10.70.42.202:/rhs/thinbrick1/cthree Brick3: 10.70.42.30:/rhs/thinbrick1/cthree Brick4: 10.70.43.191:/rhs/thinbrick2/cthree Brick5: 10.70.42.202:/rhs/thinbrick2/cthree Brick6: 10.70.42.30:/rhs/thinbrick2/cthree Options Reconfigured: performance.readdir-ahead: on [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list No sessions found [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind create ct1 cthree Session ct1 created with volume cthree [root@dhcp43-191 ~]# glusterfind create ct2 cthree Session ct2 created with volume cthree [root@dhcp43-191 ~]# glusterfind create ct3 cthree Session ct3 created with volume cthree [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list SESSION VOLUME SESSION TIME --------------------------------------------------------------------------- ct3 cthree 2015-06-26 21:24:56 ct2 cthree 2015-06-26 21:24:49 ct1 cthree 2015-06-26 21:24:41 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# vi /var/log/messages messages messages-20150621 [root@dhcp43-191 ~]# vi /var/log/messages [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# vi /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt | wc -l 121 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# vi /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt MODIFY .trashcan%2Finternal_op%2F RENAME level00%2Flevel10%2Flevel20%2Flevel30%2F%2F558d769d%25%25W722Y9OXZI.tar.gz 558d769d%25%25W722Y9OXZI.tar.gz RENAME level00%2Flevel10%2Flevel20%2Flevel30%2F%2F558d769d%25%254IGURJ859U.tar.gz 558d769d%25%254IGURJ859U.tar.gz [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt Generated output file /tmp/outc2.txt [root@dhcp43-191 ~]# vi /tmp/outc2.txt [root@dhcp43-191 ~]# vi /tmp/outc2.txt [root@dhcp43-191 ~]# cat /tmp/outc2.txt | grep W722Y9OXZI.tar.gz NEW 558d769d%25%25W722Y9OXZI.tar.gz [root@dhcp43-191 ~]# cat /tmp/outc2.txt | grep 4IGURJ859U.tar.gz NEW 558d769d%25%254IGURJ859U.tar.gz [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after deleting level03 directory, entire one [root@dhcp43-191 ~]# glusterfind post ct2 cthree Session ct2 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt Generated output file /tmp/outc2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# cat /tmp/outc2.txt DELETE level03%2Flevel13%2F558d76a7%25%259YRZROSPB2.tar.gz DELETE level03%2Flevel13%2Flevel23%2F558d76a8%25%25FAGIVIATCG.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2F558d76a8%25%252M0IIHR1WB.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2F558d76a9%25%250NN9B9EGPA.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2F558d76aa%25%256IBL1NHSPF.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83%2Flevel93 DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83 DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73 DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63 DELETE level03%2F558d76a7%25%25J5311PJ8VZ.tar.gz DELETE level03%2F558d76a7%25%25W2HQNB3IPF.tar.gz DELETE level03%2Flevel13%2Flevel23%2F558d76a8%25%25NCASR13KLT.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2F558d76a9%25%25X5IWT5GOV8.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2F558d76a9%25%25UKYTNYGLWY.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83%2Flevel93%2F558d76aa%25%25DTPNENFM0Z.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83%2Flevel93%2F558d76aa%25%25NBFZ3WUOX2.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83%2F558d76aa%25%25P1ISHJX5SP.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2Flevel73%2Flevel83%2F558d76aa%25%255TJ232FJ55.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2F558d76a9%25%25MOZBY59OLU.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53%2Flevel63%2F558d76a9%25%25D3GJYQB56E.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2Flevel53 DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43 DELETE level03%2Flevel13%2Flevel23%2Flevel33 DELETE level03%2Flevel13%2Flevel23 DELETE level03%2Flevel13 DELETE level03 DELETE level03%2Flevel13%2Flevel23%2Flevel33%2Flevel43%2F558d76a9%25%25X0CI85UCR2.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2F558d76a8%25%25E62KVQBGA0.tar.gz DELETE level03%2Flevel13%2Flevel23%2Flevel33%2F558d76a8%25%259PC5TTFRQK.tar.gz DELETE level03%2Flevel13%2F558d76a8%25%25030C83CKX2.tar.gz [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after touching 4 files [root@dhcp43-191 ~]# glusterfind post ct2 cthree Session ct2 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt Generated output file /tmp/outc2.txt [root@dhcp43-191 ~]# cat /tmp/outc2.txt MODIFY level01%2Flevel11%2F558d76a0%25%25QW0ULGFRFT.tar.gz MODIFY level01%2Flevel11%2F558d76a0%25%25XMJDJSTK8Y.tar.gz MODIFY level01%2F558d76a0%25%25OWFQCW1I2G.tar.gz MODIFY level01%2F558d769f%25%25KL9SVHJA4D.tar.gz [root@dhcp43-191 ~]# # after touching 3 dirs (inside level01 ) and deleting level01 entire directory [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post ct2 cthree Session ct2 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt Generated output file /tmp/outc2.txt [root@dhcp43-191 ~]# cat /tmp/outc2.txt DELETE level01%2F558d76a0%25%25OWFQCW1I2G.tar.gz DELETE level01%2Flevel11%2F558d76a0%25%25QW0ULGFRFT.tar.gz DELETE level01%2Flevel11%2Flevel21%2F558d76a0%25%25VC5RN1SZA7.tar.gz DELETE level01%2Flevel11%2Flevel21%2F558d76a0%25%25HO5KUEMV8G.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2F558d76a1%25%25NHRU6F3MW0.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2F558d76a1%25%25M7NLK578BO.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2F558d76a1%25%25I96QX4BDRU.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2F558d76a2%25%25UPKVAIDUF0.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2F558d76a2%25%25SXPKIR83CN.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81%2F558d76a2%25%25SNXRPW1YQQ.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81%2Flevel91%2F558d76a3%25%256PSUCOIPS0.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81%2Flevel91 DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81 DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71 DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61 DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51 DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41 DELETE level01%2Flevel11%2Flevel21%2Flevel31 DELETE level01%2Flevel11%2Flevel21 DELETE level01%2Flevel11 DELETE level01 DELETE level01%2Flevel11%2F558d76a0%25%25XMJDJSTK8Y.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2F558d76a1%25%25S2Y71MY903.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2F558d76a2%25%25I3QOSABPD0.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2F558d76a2%25%25PBIKQHMM16.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81%2F558d76a3%25%25CQHMWXZJ0E.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2Flevel51%2Flevel61%2Flevel71%2Flevel81%2Flevel91%2F558d76a3%25%2597AUKYS81P.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2Flevel41%2F558d76a1%25%25Y921BM8G7Q.tar.gz DELETE level01%2Flevel11%2Flevel21%2Flevel31%2F558d76a0%25%25RRJY0ITJ3F.tar.gz DELETE level01%2F558d769f%25%25KL9SVHJA4D.tar.gz [root@dhcp43-191 ~]# cat /tmp/outc2.txt | wc -l 30 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt | wc -l 60 [root@dhcp43-191 ~]# vi /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt | grep MODIFY [root@dhcp43-191 ~]# cat /tmp/outc1.txt | grep NEW [root@dhcp43-191 ~]# cat /tmp/outc1.txt | grep RENAME [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after modifying 30 entries and deleting 9 of them [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt MODIFY level02%2F558d76a3%25%25BVAMCY62ZN.tar.gz MODIFY level02%2Flevel12%2F558d76a4%25%254MSQKRESYF.tar.gz MODIFY level02%2Flevel12%2Flevel22%2F558d76a4%25%258AP2NYK5ZK.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2F558d76a4%25%25XMKIT29PAM.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2F558d76a5%25%25QLA2J40ANC.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2F558d76a5%25%25B4HXZ0FU6Q.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2F558d76a5%25%250MC2YGZK0I.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%2550NJATX2MF.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%25SFIGGVTUNG.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2F MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2F MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2F MODIFY level02%2Flevel12%2Flevel22%2F MODIFY level02%2Flevel12%2F MODIFY level02%2F MODIFY level02%2Flevel12%2Flevel22%2F558d76a4%25%25EWGL478PL5.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2F558d76a5%25%25MLM3P4973M.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2F558d76a5%25%2569NA2DXJJV.tar.gz MODIFY level02%2Flevel12%2F558d76a4%25%25ACZELALT1M.tar.gz MODIFY level02%2F558d76a3%25%25B93S0VHWNQ.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2F558d76a6%25%25Y6SLH21JNN.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82%2F558d76a6%25%25VEDNQSCHNH.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82%2Flevel92%2F558d76a7%25%25CBVU0JSJHZ.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82%2Flevel92%2F558d76a7%25%25BG0B7CLNO0.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82%2Flevel92 DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82 DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72 DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2Flevel82%2F558d76a6%25%25KIO19WEQV0.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2Flevel72%2F558d76a6%25%252MXPXFAT4W.tar.gz [root@dhcp43-191 ~]# cat /tmp/outc1.txt | wc -l 30 [root@dhcp43-191 ~]# cat /tmp/outc1.txt | grep DELETE | wc -l 9 [root@dhcp43-191 ~]# cat /tmp/outc1.txt | grep MODIFY | wc -l 21 [root@dhcp43-191 ~]# gluster snapshot create Usage: snapshot create <snapname> <volname> [no-timestamp] [description <description>] [force] [root@dhcp43-191 ~]# gluster snapshot create snap1 cthree snapshot create: success: Snap snap1_GMT-2015.06.26-16.20.52 created successfully [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster snapshot list snap1_GMT-2015.06.26-16.20.52 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after modifying 2 files from a non-root user [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%2550NJATX2MF.tar.gz MODIFY level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%25SFIGGVTUNG.tar.gz [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after deleting the same two files [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# cat /tmp/outc1.txt DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%2550NJATX2MF.tar.gz DELETE level02%2Flevel12%2Flevel22%2Flevel32%2Flevel42%2Flevel52%2Flevel62%2F558d76a6%25%25SFIGGVTUNG.tar.gz [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v snapshot list unrecognized word: snapshot (position 1) [root@dhcp43-191 ~]# gluster snapshot list snap1_GMT-2015.06.26-16.20.52 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster snapshot activate Usage: snapshot activate <snapname> [force] [root@dhcp43-191 ~]# gluster snapshot activate snap1_GMT-2015.06.26-16.20.52 Snapshot activate: snap1_GMT-2015.06.26-16.20.52: Snap activated successfully [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster snapshot list snap1_GMT-2015.06.26-16.20.52 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster snapshot info Snapshot : snap1_GMT-2015.06.26-16.20.52 Snap UUID : cd2f5c48-dccf-48db-8112-9a5d6e229575 Created : 2015-06-26 16:20:52 Snap Volumes: Snap Volume Name : fc606a41782c415d8797bac07aa7c300 Origin Volume name : cthree Snaps taken for cthree : 1 Snaps available for cthree : 255 Status : Started [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster snapshot restore Usage: snapshot restore <snapname> [root@dhcp43-191 ~]# gluster snapshot restore snap1_GMT-2015.06.26-16.20.52 Restore operation will replace the original volume with the snapshotted volume. Do you still want to continue? (y/n) y snapshot restore: failed: Volume (cthree) has been started. Volume needs to be stopped before restoring a snapshot. Snapshot command failed [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v stop cthree Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: cthree: success [root@dhcp43-191 ~]# gluster snapshot restore snap1_GMT-2015.06.26-16.20.52 Restore operation will replace the original volume with the snapshotted volume. Do you still want to continue? (y/n) y Snapshot restore: snap1_GMT-2015.06.26-16.20.52: Snap restored successfully [root@dhcp43-191 ~]# gluster snapshot list No snapshots present [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v start cthree volume start: cthree: success [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v list gluster_shared_storage cthree [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post ct1 cthree Session ct1 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt 10.70.43.191 - pre failed: [2015-06-26 16:38:57.178423] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 0] Success 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.202 - pre failed: [2015-06-26 16:38:58.579735] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 [2015-06-26 16:38:58.579963] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2015-06-26 16:38:58.580818] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 4 [2015-06-26 16:38:58.581073] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.202 - pre failed: [2015-06-26 16:38:58.579683] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2015-06-26 16:38:58.580207] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-06-26 16:38:58.580510] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 4 [2015-06-26 16:38:58.580954] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 0] Success Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# cat /tmp/outc1.txt [root@dhcp43-191 ~]# glusterfind pre ct1 cthree /tmp/outc1.txt --regenerate-outfile 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 0] Success 10.70.42.30 - pre failed: [2015-06-26 16:39:34.447049] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-06-26 16:39:34.447189] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 0] Success Generated output file /tmp/outc1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outc1.txt -rw-r--r--. 1 root root 0 Jun 26 22:09 /tmp/outc1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# vi /var/log/glusterfs/glusterfind/ ^%#@*/ cli.log cross3s2/ ct1/ ct3/ demos/ fdjksl/ nosession/ ozone/ session/ sessn2/ sesso2/ sesso4/ sesso6/ 5543/ cross3s1/ cross3s3/ ct2/ demo/ demosess/ fds/ novolume/ sess_demo/ sessn1/ sesso1/ sesso3/ sesso5/ sess_vol1/ [root@dhcp43-191 ~]# vi /var/log/glusterfs/glusterfind/ct1/cthree/c changelog.6054968fe2031536d3d2ff230fb52e0d6adb8ba9.log changelog.d64a565ffec1c323662a9992145421a5818434dd.log changelog.log changelog.8058a1a97f3de22d46b6e2dee0cc8d42cd00bd2b.log changelog.dea6e936fed0a5a7f76415c0cbeae0e2fee35f2d.log cli.log [root@dhcp43-191 ~]# vi /var/log/glusterfs/glusterfind/ct1/cthree/changelog.log [root@dhcp43-191 ~]# vi /var/log/glusterfs/etc-glusterfs-glusterd.vol.log [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# rpm -qa | grep gluster glusterfs-3.7.1-5.el6rhs.x86_64 glusterfs-cli-3.7.1-5.el6rhs.x86_64 glusterfs-libs-3.7.1-5.el6rhs.x86_64 glusterfs-client-xlators-3.7.1-5.el6rhs.x86_64 glusterfs-api-3.7.1-5.el6rhs.x86_64 glusterfs-server-3.7.1-5.el6rhs.x86_64 glusterfs-fuse-3.7.1-5.el6rhs.x86_64 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list SESSION VOLUME SESSION TIME --------------------------------------------------------------------------- ct3 cthree 2015-06-26 21:24:56 ct2 cthree 2015-06-26 21:43:57 ct1 cthree 2015-06-26 22:05:54 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt Post command is not run after last pre, use --regenerate-outfile [root@dhcp43-191 ~]# glusterfind post ct2 cthree Session ct2 with volume cthree updated [root@dhcp43-191 ~]# glusterfind pre ct2 cthree /tmp/outc2.txt 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:52:42.518791] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: [2015-06-26 16:52:43.140745] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: [2015-06-26 16:52:43.142100] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 2] No such file or directory Generated output file /tmp/outc2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outc2.txt -rw-r--r--. 1 root root 0 Jun 26 22:22 /tmp/outc2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct3 cthree /tmp/outc3.txt 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:52:59.517165] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:52:59.513643] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: [2015-06-26 16:53:00.164337] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 2] No such file or directory Generated output file /tmp/outc3.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list SESSION VOLUME SESSION TIME --------------------------------------------------------------------------- ct3 cthree 2015-06-26 21:24:56 ct2 cthree 2015-06-26 21:45:42 ct1 cthree 2015-06-26 22:05:54 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind create ct4 cthree Session ct4 created with volume cthree [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list SESSION VOLUME SESSION TIME --------------------------------------------------------------------------- ct3 cthree 2015-06-26 21:24:56 ct2 cthree 2015-06-26 21:45:42 ct1 cthree 2015-06-26 22:05:54 ct4 cthree 2015-06-26 22:23:30 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre ct4 cthree /tmp/outc4.txt 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:53:32.147307] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:53:32.158716] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 2] No such file or directory Generated output file /tmp/outc4.txt [root@dhcp43-191 ~]# glusterfind pre ct4 cthree /tmp/outc4.txt --regenerate-outfile 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick4/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.43.191 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick1/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick2/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.202 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick5/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick3/cthree Historical Changelogs not available: [Errno 2] No such file or directory 10.70.42.30 - pre failed: [2015-06-26 16:53:48.389700] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2015-06-26 16:53:48.389680] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-06-26 16:53:48.392236] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 [2015-06-26 16:53:48.393020] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 4 /var/run/gluster/snaps/fc606a41782c415d8797bac07aa7c300/brick6/cthree Historical Changelogs not available: [Errno 2] No such file or directory Generated output file /tmp/outc4.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ############# CLIENT ################# ###### NFS ######## [root@dhcp43-59 ~]# [root@dhcp43-59 ~]# mkdir /mnt/cthree [root@dhcp43-59 ~]# mount -t nfs 10.70.42.202:/cthree /mnt/cthree/ [root@dhcp43-59 ~]# cd /mnt/cthree/ [root@dhcp43-59 cthree]# ls -a . .. .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level01 level02 level03 .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# rm -rf level03 rm: fts_read failed: Stale file handle [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level01 level02 level03 .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# df -k . Filesystem 1K-blocks Used Available Use% Mounted on 10.70.42.202:/cthree 16755712 73728 16681984 1% /mnt/cthree [root@dhcp43-59 cthree]# rm -rf level03/ [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level01 level02 .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# find level01 | wc -l 30 [root@dhcp43-59 cthree]# touch level01/level11/level21/level31/ [root@dhcp43-59 cthree]# touch level01/level11/level21/level31/level41/ [root@dhcp43-59 cthree]# touch level01/level11/level21/level31/level41/level51/ [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# rm -rf level01 rm: fts_read failed: Stale file handle [root@dhcp43-59 cthree]# rm -rf level01/ [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-59 cthree]# find * | wc -l 49 [root@dhcp43-59 cthree]# # after doing a snapshot restore [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-59 cthree]# find * | wc -l 51 [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# [root@dhcp43-59 cthree]# ########## FUSE ############ [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# mkdir /mnt/cthree [root@dhcp43-71 ~]# mount -t glusterfs 10.70.43.191:/cthree /mnt/cthree [root@dhcp43-71 ~]# ls -a . All_NetBackup_7.6.1_PDFs.zip .bash_history .bash_profile .cshrc install.log .lesshst .ssh test.txt .. anaconda-ks.cfg .bash_logout .bashrc geo-automation install.log.syslog .pki .tcshrc .viminfo [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# cd /mnt/cthree [root@dhcp43-71 cthree]# ls -a . .. .trashcan [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# crefi usage: crefi [-h] [-V] [-n FILES] [--size SIZE] [--random] [--max MAX] [--min MIN] [--single] [--multi] [-b BRDTH] [-d DEPTH] [-l FLEN] [-t {text,sparse,binary,tar}] [-I INTER] [--fop {create,rename,chmod,chown,chgrp,symlink,hardlink,truncate,setxattr}] [-R] [-T THREADS] mntpnt crefi: error: too few arguments [root@dhcp43-71 cthree]# crefi --multi -b 4 -d 10 -n 2 -t tar usage: crefi [-h] [-V] [-n FILES] [--size SIZE] [--random] [--max MAX] [--min MIN] [--single] [--multi] [-b BRDTH] [-d DEPTH] [-l FLEN] [-t {text,sparse,binary,tar}] [-I INTER] [--fop {create,rename,chmod,chown,chgrp,symlink,hardlink,truncate,setxattr}] [-R] [-T THREADS] mntpnt crefi: error: too few arguments [root@dhcp43-71 cthree]# crefi --multi -b 4 -d 10 -n 2 -t tar . 1 2015-06-26 21:28:19,892 - thread-0: Entering the directory level00 2015-06-26 21:28:20,318 - thread-0: Entering the directory level10 2015-06-26 21:28:20,720 - thread-0: Entering the directory level20 2015-06-26 21:28:21,112 - thread-0: Entering the directory level30 2015-06-26 21:28:21,515 - thread-0: Entering the directory level40 2015-06-26 21:28:21,909 - thread-0: Entering the directory level50 2015-06-26 21:28:22,323 - thread-0: Entering the directory level60 2015-06-26 21:28:22,723 - thread-0: Entering the directory level70 2015-06-26 21:28:23,093 - thread-0: Entering the directory level80 2015-06-26 21:28:23,481 - thread-0: Entering the directory level90 2015-06-26 21:28:23,839 - thread-0: Entering the directory level01 2015-06-26 21:28:24,195 - thread-0: Entering the directory level11 2015-06-26 21:28:24,574 - thread-0: Entering the directory level21 2015-06-26 21:28:24,955 - thread-0: Entering the directory level31 2015-06-26 21:28:25,352 - thread-0: Entering the directory level41 2015-06-26 21:28:25,737 - thread-0: Entering the directory level51 2015-06-26 21:28:26,092 - thread-0: Entering the directory level61 2015-06-26 21:28:26,524 - thread-0: Entering the directory level71 2015-06-26 21:28:26,950 - thread-0: Entering the directory level81 2015-06-26 21:28:27,373 - thread-0: Entering the directory level91 2015-06-26 21:28:27,764 - thread-0: Entering the directory level02 2015-06-26 21:28:28,150 - thread-0: Entering the directory level12 2015-06-26 21:28:28,541 - thread-0: Entering the directory level22 2015-06-26 21:28:28,955 - thread-0: Entering the directory level32 2015-06-26 21:28:29,313 - thread-0: Entering the directory level42 2015-06-26 21:28:29,654 - thread-0: Entering the directory level52 2015-06-26 21:28:30,017 - thread-0: Entering the directory level62 2015-06-26 21:28:30,420 - thread-0: Entering the directory level72 2015-06-26 21:28:30,827 - thread-0: Entering the directory level82 2015-06-26 21:28:31,185 - thread-0: Entering the directory level92 2015-06-26 21:28:31,534 - thread-0: Entering the directory level03 2015-06-26 21:28:31,857 - thread-0: Entering the directory level13 2015-06-26 21:28:32,198 - thread-0: Entering the directory level23 2015-06-26 21:28:32,528 - thread-0: Entering the directory level33 2015-06-26 21:28:32,929 - thread-0: Entering the directory level43 2015-06-26 21:28:33,286 - thread-0: Entering the directory level53 2015-06-26 21:28:33,602 - thread-0: Entering the directory level63 2015-06-26 21:28:33,946 - thread-0: Entering the directory level73 2015-06-26 21:28:34,294 - thread-0: Entering the directory level83 2015-06-26 21:28:34,646 - thread-0: Entering the directory level93 2015-06-26 21:28:34,971 - thread-0: finished creating files with throughput ---- 2.054 MBps [root@dhcp43-71 cthree]# ls -a . .. level00 level01 level02 level03 .trashcan [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# find | wc -l 123 [root@dhcp43-71 cthree]# ls level00/ 558d769b%%YSL26V60P2.tar.gz 558d769c%%JX26ZOCNJ4.tar.gz level10/ [root@dhcp43-71 cthree]# ls level00/level10/level20/level30/ 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level40/ [root@dhcp43-71 cthree]# mv level00/level10/level20/level30/*.gz . [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level01 level02 .trashcan [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# touch level01/ 558d769f%%KL9SVHJA4D.tar.gz 558d76a0%%OWFQCW1I2G.tar.gz level11/ [root@dhcp43-71 cthree]# touch level01/level11/*.gz [root@dhcp43-71 cthree]# touch level01/*.gz [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# chown -R u1 level02/ [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# chmod -R 755 level02/ [root@dhcp43-71 cthree]# find level02 | wc -l 30 [root@dhcp43-71 cthree]# rm -rf level02/level12/level22/level32/level42/level52/level62/level72/ [root@dhcp43-71 cthree]# find level02 | wc -l 21 [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# # taken a snapshot snap1 [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# ls -a . .. 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 .trashcan [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# su u1 [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ chmod 777 level02/level12/level22/level32/level42/level52/level62/558d76a6%% 558d76a6%%50NJATX2MF.tar.gz 558d76a6%%SFIGGVTUNG.tar.gz [u1@dhcp43-71 cthree]$ chmod 777 level02/level12/level22/level32/level42/level52/level62/558d76a6%% chmod: cannot access `level02/level12/level22/level32/level42/level52/level62/558d76a6%%': No such file or directory [u1@dhcp43-71 cthree]$ chmod 777 level02/level12/level22/level32/level42/level52/level62/* [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ rm rf *.gz rm: cannot remove `rf': No such file or directory rm: remove write-protected regular file `558d769d%%4IGURJ859U.tar.gz'? y^C [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ [u1@dhcp43-71 cthree]$ rm -rf *.gz rm: cannot remove `558d769d%%4IGURJ859U.tar.gz': Permission denied rm: cannot remove `558d769d%%W722Y9OXZI.tar.gz': Permission denied [u1@dhcp43-71 cthree]$ exit [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# ls 558d769d%%4IGURJ859U.tar.gz 558d769d%%W722Y9OXZI.tar.gz level00 level02 [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# rm -rf level02/level12/level22/level32/level42/level52/level62/558d76a6%% 558d76a6%%50NJATX2MF.tar.gz 558d76a6%%SFIGGVTUNG.tar.gz [root@dhcp43-71 cthree]# rm -rf level02/level12/level22/level32/level42/level52/level62/558d76a6%%* [root@dhcp43-71 cthree]# [root@dhcp43-71 cthree]# Additional info: