Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: In my 4 node cluster setup, had a 4*2 dist-rep volume 'ozone'. Got node3 down (which had 2 data bricks belonging to ozone), and created a directory tree structure and ran glusterfind pre on session 'sesso1'. That displayed the error 'no route to host', and gave an output file which was empty. Executed the same command on another session 'sesso2' and that failed with 'no route to host' error. BUT, that populated the output file correctly, as expected. When a node is down, ideally 1. It should not have impacted the output of glusterfind. 2. Error messages should not have been seen on the console stating 'pre failed.. connection timed out.. no route to host'. 3. Whatever behaviour is expected, should be the same across all glusterfind sessions (of the same volume) Version-Release number of selected component (if applicable): glusterfs-3.7.1-4.el6rhs.x86_64 How reproducible: 1:1 Steps that I followed: 1. Have a 4 node cluster, with node1, node2, node3 and node4. Create a 4*2 distribute replicate volume 'ozone', with node1 and node3 having the data bricks and node2 and node4 having the replica bricks. 2. Create glusterfind session 'sesso1' and 'sesso2' for 'ozone', and mount it using fuse/nfs 3. Create a data set at the mountpoint and run glusterfind pre and post a couple of times and verify the output file, regularly at 'sesso1' and intermittently at 'sesso2' 4. Halt node3 5. Create a directory tree structure at the mountpoint (>100 entries) and run glusterfind pre and post on 'sesso1' 6. Do a couple of changes and again run glusterfind pre and post on 'sesso1' 7. Run glusterfind pre and post on 'sesso2' Actual results: Step5 gives the below output, with an *empty* output file. [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outo1.txt -rw-r--r--. 1 root root 0 Jun 19 20:28 /tmp/outo1.txt [root@dhcp43-191 ~]# Step6 gives the same output as mentioned above Step7 gives the below output, but *does* populate the output file as expected [root@dhcp43-191 ~]# glusterfind pre sesso2 ozone /tmp/outo2.txt 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outo2.txt -rw-r--r--. 1 root root 10233 Jun 19 20:28 /tmp/outo2.txt [root@dhcp43-191 ~]# vi /tmp/outo2.txt [root@dhcp43-191 ~]# Expected results: Step5 and step6 should NOT have displayed errors on the console. Also, the output file should have got correctly populated. Step7 should not have displayed the errors on the console. Additional info: [root@dhcp42-202 ~]# [root@dhcp42-202 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Connected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) Hostname: 10.70.43.191 Uuid: eff916e2-3d5f-43ae-abab-05bbcbdf473c State: Peer in Cluster (Connected) [root@dhcp42-202 ~]# rpm -qa | grep glusterfs glusterfs-libs-3.7.1-4.el6rhs.x86_64 glusterfs-api-3.7.1-4.el6rhs.x86_64 glusterfs-client-xlators-3.7.1-4.el6rhs.x86_64 glusterfs-fuse-3.7.1-4.el6rhs.x86_64 glusterfs-cli-3.7.1-4.el6rhs.x86_64 glusterfs-3.7.1-4.el6rhs.x86_64 glusterfs-server-3.7.1-4.el6rhs.x86_64 [root@dhcp42-202 ~]#
Pasting everything that I did, since morning: [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v info Volume Name: gluster_shared_storage Type: Replicate Volume ID: ced3ec30-654b-4bf5-956b-9e99bc51d445 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/brick1/gss Brick2: 10.70.42.202:/rhs/brick1/gss Brick3: 10.70.42.30:/rhs/brick1/gss Options Reconfigured: performance.readdir-ahead: on Volume Name: ozone Type: Distributed-Replicate Volume ID: 9ef1ace8-505d-4d97-aa23-4296aa685f76 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/thinbrick1/ozone Brick2: 10.70.42.202:/rhs/thinbrick1/ozone Brick3: 10.70.43.191:/rhs/thinbrick2/ozone Brick4: 10.70.42.202:/rhs/thinbrick2/ozone Brick5: 10.70.42.30:/rhs/thinbrick1/ozone Brick6: 10.70.42.147:/rhs/thinbrick1/ozone Brick7: 10.70.42.30:/rhs/thinbrick2/ozone Brick8: 10.70.42.147:/rhs/thinbrick2/ozone Options Reconfigured: changelog.capture-del-path: on changelog.changelog: on storage.build-pgfid: on performance.readdir-ahead: on [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind list SESSION VOLUME SESSION TIME --------------------------------------------------------------------------- sesso3 ozone 2015-06-18 16:27:30 sesso1 ozone 2015-06-18 21:05:19 sesso2 ozone 2015-06-18 20:00:38 sesso4 ozone 2015-06-18 16:27:38 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind post sesso2 ozone Session sesso2 with volume ozone updated [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# cat /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# after doing chown as well as chgrp to level01 -bash: after: command not found [root@dhcp43-191 ~]# glusterfind post sesso2 ozone Pre script is not run [root@dhcp43-191 ~]# glusterfind post sesso2 ozone Pre script is not run [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Post command is not run after last pre, use --regenerate-outfile [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt MODIFY level01%2F MODIFY level20%2F [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# # after touching a symbolic link file [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt MODIFY level20%2F [root@dhcp43-191 ~]# # after renaming a symbolic link file [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt RENAME level20_sln level20_sln2 [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW level20%2F%2Flevel01 [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW a [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW a NEW a_symlink [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW a NEW a_symlink [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW b NEW a_symlink_to_b [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt DELETE a_symlink_to_b [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt DELETE a_symlink DELETE a_symlink_to_b [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt DELETE a_symlink DELETE a_symlink_to_b [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after changing recursively the group ownership of level31 [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt MODIFY level01%2Flevel11%2Flevel21%2Flevel31%2F5582c256%25%25MPIBDVQJQ4 MODIFY level01%2Flevel11%2Flevel21%2Flevel31%2F5582c255%25%25WRBYSGT7PZ MODIFY level01%2Flevel11%2Flevel21%2Flevel31%2F MODIFY level01%2Flevel11%2Flevel21%2Flevel31%2F5582c256%25%251QNX332SKS MODIFY level01%2Flevel11%2Flevel21%2Flevel31%2F5582c255%25%25OQ65TSMRUD [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after creating a hardlink file [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt NEW level01%2Flevel11%2F%2Fhardlink_file [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after rename and simulateneous movement of hardlink file [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt 10.70.42.30 - pre failed: ^[[A^[[AGenerated output file /tmp/outo1.txt [root@dhcp43-191 ~]# glusterfind post sesso1 ozone --regenerate-outfile usage: glusterfind [-h] {pre,create,list,post,delete} ... glusterfind: error: unrecognized arguments: --regenerate-outfile [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt RENAME level01%2Flevel11%2F%2Fhardlink_file level01%2F%2Fhardlink [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# cd [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Connected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ########## changed the glusterfs build from 3.7.1-3 to 3.7.1-4 ########## [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# service glusterd stop [root@dhcp43-191 ~]# [ OK ] [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# pkill glusterfs [root@dhcp43-191 ~]# pkill glusterfsd [root@dhcp43-191 ~]# pkill gsyncd.py [root@dhcp43-191 ~]# yum remove gluster Loaded plugins: product-id, security, subscription-manager Setting up Remove Process No Match for argument: gluster epel/metalink | 4.1 kB 00:00 rhel-6-server-rpms | 3.7 kB 00:00 rhel-6-server-rpms/primary_db | 32 MB 03:52 rhel-scalefs-for-rhel-6-server-rpms | 4.6 kB 00:00 rhs-3-for-rhel-6-server-rpms | 3.1 kB 00:00 rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ | 2.9 kB 00:00 rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_/primary_db | 13 kB 00:00 No Packages marked for removal [root@dhcp43-191 ~]# yum remove glusterfs Loaded plugins: product-id, security, subscription-manager Setting up Remove Process Resolving Dependencies --> Running transaction check ---> Package glusterfs.x86_64 0:3.7.1-3.el6rhs will be erased --> Processing Dependency: glusterfs = 3.7.1-3.el6rhs for package: glusterfs-api-3.7.1-3.el6rhs.x86_64 --> Processing Dependency: glusterfs = 3.7.1-3.el6rhs for package: glusterfs-server-3.7.1-3.el6rhs.x86_64 --> Processing Dependency: glusterfs = 3.7.1-3.el6rhs for package: glusterfs-fuse-3.7.1-3.el6rhs.x86_64 --> Running transaction check ---> Package glusterfs-api.x86_64 0:3.7.1-3.el6rhs will be erased ---> Package glusterfs-fuse.x86_64 0:3.7.1-3.el6rhs will be erased ---> Package glusterfs-server.x86_64 0:3.7.1-3.el6rhs will be erased --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================== Removing: glusterfs x86_64 3.7.1-3.el6rhs @rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 1.5 M Removing for dependencies: glusterfs-api x86_64 3.7.1-3.el6rhs @rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 132 k glusterfs-fuse x86_64 3.7.1-3.el6rhs @rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 304 k glusterfs-server x86_64 3.7.1-3.el6rhs @rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 4.0 M Transaction Summary ============================================================================================================================================================================================== Remove 4 Package(s) Installed size: 5.9 M Is this ok [y/N]: y Downloading Packages: Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Erasing : glusterfs-server-3.7.1-3.el6rhs.x86_64 1/4 Erasing : glusterfs-fuse-3.7.1-3.el6rhs.x86_64 2/4 Erasing : glusterfs-api-3.7.1-3.el6rhs.x86_64 3/4 Erasing : glusterfs-3.7.1-3.el6rhs.x86_64 4/4 rhel-6-server-rpms/productid | 1.7 kB 00:00 Verifying : glusterfs-fuse-3.7.1-3.el6rhs.x86_64 1/4 Verifying : glusterfs-api-3.7.1-3.el6rhs.x86_64 2/4 Verifying : glusterfs-3.7.1-3.el6rhs.x86_64 3/4 Verifying : glusterfs-server-3.7.1-3.el6rhs.x86_64 4/4 Removed: glusterfs.x86_64 0:3.7.1-3.el6rhs Dependency Removed: glusterfs-api.x86_64 0:3.7.1-3.el6rhs glusterfs-fuse.x86_64 0:3.7.1-3.el6rhs glusterfs-server.x86_64 0:3.7.1-3.el6rhs Complete! [root@dhcp43-191 ~]# yum install glusterfs-server --nogpgcheck Loaded plugins: product-id, security, subscription-manager Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package glusterfs-server.x86_64 0:3.7.1-4.el6rhs will be installed --> Processing Dependency: glusterfs-libs = 3.7.1-4.el6rhs for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: glusterfs-fuse = 3.7.1-4.el6rhs for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: glusterfs-client-xlators = 3.7.1-4.el6rhs for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: glusterfs-cli = 3.7.1-4.el6rhs for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: glusterfs = 3.7.1-4.el6rhs for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_PRIVATE_3.7.0)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_PRIVATE_3.4.0)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_3.7.0)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_3.6.0)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_3.5.1)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_3.4.2)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0(GFAPI_3.4.0)(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Processing Dependency: libgfapi.so.0()(64bit) for package: glusterfs-server-3.7.1-4.el6rhs.x86_64 --> Running transaction check ---> Package glusterfs.x86_64 0:3.7.1-4.el6rhs will be installed ---> Package glusterfs-api.x86_64 0:3.7.1-4.el6rhs will be installed ---> Package glusterfs-cli.x86_64 0:3.7.1-3.el6rhs will be updated ---> Package glusterfs-cli.x86_64 0:3.7.1-4.el6rhs will be an update ---> Package glusterfs-client-xlators.x86_64 0:3.7.1-3.el6rhs will be updated ---> Package glusterfs-client-xlators.x86_64 0:3.7.1-4.el6rhs will be an update ---> Package glusterfs-fuse.x86_64 0:3.7.1-4.el6rhs will be installed ---> Package glusterfs-libs.x86_64 0:3.7.1-3.el6rhs will be updated ---> Package glusterfs-libs.x86_64 0:3.7.1-4.el6rhs will be an update --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================== Installing: glusterfs-server x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 1.2 M Installing for dependencies: glusterfs x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 386 k glusterfs-api x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 51 k glusterfs-fuse x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 100 k Updating for dependencies: glusterfs-cli x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 146 k glusterfs-client-xlators x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 898 k glusterfs-libs x86_64 3.7.1-4.el6rhs rhsqe-repo.lab.eng.blr.redhat.com_rhgs3.1-server-latest-el6_ 297 k Transaction Summary ============================================================================================================================================================================================== Install 4 Package(s) Upgrade 3 Package(s) Total download size: 3.0 M Is this ok [y/N]: y Downloading Packages: (1/7): glusterfs-3.7.1-4.el6rhs.x86_64.rpm | 386 kB 00:00 (2/7): glusterfs-api-3.7.1-4.el6rhs.x86_64.rpm | 51 kB 00:00 (3/7): glusterfs-cli-3.7.1-4.el6rhs.x86_64.rpm | 146 kB 00:00 (4/7): glusterfs-client-xlators-3.7.1-4.el6rhs.x86_64.rpm | 898 kB 00:00 (5/7): glusterfs-fuse-3.7.1-4.el6rhs.x86_64.rpm | 100 kB 00:00 (6/7): glusterfs-libs-3.7.1-4.el6rhs.x86_64.rpm | 297 kB 00:00 (7/7): glusterfs-server-3.7.1-4.el6rhs.x86_64.rpm | 1.2 MB 00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 25 MB/s | 3.0 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : glusterfs-libs-3.7.1-4.el6rhs.x86_64 1/10 Updating : glusterfs-client-xlators-3.7.1-4.el6rhs.x86_64 2/10 Installing : glusterfs-3.7.1-4.el6rhs.x86_64 3/10 Installing : glusterfs-fuse-3.7.1-4.el6rhs.x86_64 4/10 Installing : glusterfs-api-3.7.1-4.el6rhs.x86_64 5/10 Updating : glusterfs-cli-3.7.1-4.el6rhs.x86_64 6/10 Installing : glusterfs-server-3.7.1-4.el6rhs.x86_64 7/10 warning: /var/lib/glusterd/vols/ozone/ozone-rebalance.vol saved as /var/lib/glusterd/vols/ozone/ozone-rebalance.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.147.rhs-thinbrick2-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.147.rhs-thinbrick2-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/trusted-ozone.tcp-fuse.vol saved as /var/lib/glusterd/vols/ozone/trusted-ozone.tcp-fuse.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.43.191.rhs-thinbrick2-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.43.191.rhs-thinbrick2-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.147.rhs-thinbrick1-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.147.rhs-thinbrick1-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.30.rhs-thinbrick2-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.30.rhs-thinbrick2-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.tcp-fuse.vol saved as /var/lib/glusterd/vols/ozone/ozone.tcp-fuse.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.202.rhs-thinbrick1-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.202.rhs-thinbrick1-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.43.191.rhs-thinbrick1-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.43.191.rhs-thinbrick1-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.30.rhs-thinbrick1-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.30.rhs-thinbrick1-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/ozone/ozone.10.70.42.202.rhs-thinbrick2-ozone.vol saved as /var/lib/glusterd/vols/ozone/ozone.10.70.42.202.rhs-thinbrick2-ozone.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.42.30.rhs-brick1-gss.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.42.30.rhs-brick1-gss.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.43.191.rhs-brick1-gss.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.43.191.rhs-brick1-gss.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage-rebalance.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage-rebalance.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.42.202.rhs-brick1-gss.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.10.70.42.202.rhs-brick1-gss.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/trusted-gluster_shared_storage.tcp-fuse.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/trusted-gluster_shared_storage.tcp-fuse.vol.rpmsave warning: /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.tcp-fuse.vol saved as /var/lib/glusterd/vols/gluster_shared_storage/gluster_shared_storage.tcp-fuse.vol.rpmsave Cleanup : glusterfs-cli-3.7.1-3.el6rhs.x86_64 8/10 Cleanup : glusterfs-client-xlators-3.7.1-3.el6rhs.x86_64 9/10 Cleanup : glusterfs-libs-3.7.1-3.el6rhs.x86_64 10/10 Verifying : glusterfs-fuse-3.7.1-4.el6rhs.x86_64 1/10 Verifying : glusterfs-libs-3.7.1-4.el6rhs.x86_64 2/10 Verifying : glusterfs-api-3.7.1-4.el6rhs.x86_64 3/10 Verifying : glusterfs-server-3.7.1-4.el6rhs.x86_64 4/10 Verifying : glusterfs-client-xlators-3.7.1-4.el6rhs.x86_64 5/10 Verifying : glusterfs-cli-3.7.1-4.el6rhs.x86_64 6/10 Verifying : glusterfs-3.7.1-4.el6rhs.x86_64 7/10 Verifying : glusterfs-client-xlators-3.7.1-3.el6rhs.x86_64 8/10 Verifying : glusterfs-libs-3.7.1-3.el6rhs.x86_64 9/10 Verifying : glusterfs-cli-3.7.1-3.el6rhs.x86_64 10/10 Installed: glusterfs-server.x86_64 0:3.7.1-4.el6rhs Dependency Installed: glusterfs.x86_64 0:3.7.1-4.el6rhs glusterfs-api.x86_64 0:3.7.1-4.el6rhs glusterfs-fuse.x86_64 0:3.7.1-4.el6rhs Dependency Updated: glusterfs-cli.x86_64 0:3.7.1-4.el6rhs glusterfs-client-xlators.x86_64 0:3.7.1-4.el6rhs glusterfs-libs.x86_64 0:3.7.1-4.el6rhs Complete! [root@dhcp43-191 ~]# ########### updation complete ############### [root@dhcp43-191 ~]# rpm -qa | grep glusterfs glusterfs-libs-3.7.1-4.el6rhs.x86_64 glusterfs-api-3.7.1-4.el6rhs.x86_64 glusterfs-client-xlators-3.7.1-4.el6rhs.x86_64 glusterfs-fuse-3.7.1-4.el6rhs.x86_64 glusterfs-cli-3.7.1-4.el6rhs.x86_64 glusterfs-3.7.1-4.el6rhs.x86_64 glusterfs-server-3.7.1-4.el6rhs.x86_64 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# service glusterd start Starting glusterd: [ OK ] [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Connected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster v info Volume Name: gluster_shared_storage Type: Replicate Volume ID: ced3ec30-654b-4bf5-956b-9e99bc51d445 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/brick1/gss Brick2: 10.70.42.202:/rhs/brick1/gss Brick3: 10.70.42.30:/rhs/brick1/gss Options Reconfigured: performance.readdir-ahead: on Volume Name: ozone Type: Distributed-Replicate Volume ID: 9ef1ace8-505d-4d97-aa23-4296aa685f76 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/thinbrick1/ozone Brick2: 10.70.42.202:/rhs/thinbrick1/ozone Brick3: 10.70.43.191:/rhs/thinbrick2/ozone Brick4: 10.70.42.202:/rhs/thinbrick2/ozone Brick5: 10.70.42.30:/rhs/thinbrick1/ozone Brick6: 10.70.42.147:/rhs/thinbrick1/ozone Brick7: 10.70.42.30:/rhs/thinbrick2/ozone Brick8: 10.70.42.147:/rhs/thinbrick2/ozone Options Reconfigured: performance.readdir-ahead: on storage.build-pgfid: on changelog.changelog: on changelog.capture-del-path: on [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Connected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post sesso1 ozone Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Disconnected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# ###### got node 3 down ################ [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Disconnected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: Connection timed out 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: Connection timed out 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# vi /tmp/outo1.txt [root@dhcp43-191 ~]# glusterfind post sesso1 ozone 10.70.42.30 - post failed: ssh: connect to host 10.70.42.30 port 22: Connection timed out 10.70.42.30 - post failed: ssh: connect to host 10.70.42.30 port 22: Connection timed out Session sesso1 with volume ozone updated [root@dhcp43-191 ~]# date Fri Jun 19 20:19:32 IST 2015 [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# vi /tmp/outo1.txt [root@dhcp43-191 ~]# date Fri Jun 19 20:26:35 IST 2015 [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt Post command is not run after last pre, use --regenerate-outfile [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# vi /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt [root@dhcp43-191 ~]# date Fri Jun 19 20:27:38 IST 2015 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# date Fri Jun 19 20:28:10 IST 2015 [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outo1.txt -rw-r--r--. 1 root root 0 Jun 19 20:28 /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre sesso2 ozone /tmp/outo2.txt 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host Generated output file /tmp/outo2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outo2.txt -rw-r--r--. 1 root root 10233 Jun 19 20:28 /tmp/outo2.txt [root@dhcp43-191 ~]# vi /tmp/outo2.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind post sesso2 ozone 10.70.42.30 - post failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - post failed: ssh: connect to host 10.70.42.30 port 22: No route to host Session sesso2 with volume ozone updated [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - pre failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: No route to host 10.70.42.30 - cleanup failed: ssh: connect to host 10.70.42.30 port 22: Connection timed out Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# ls -l /tmp/outo1.txt -rw-r--r--. 1 root root 0 Jun 19 20:34 /tmp/outo1.txt [root@dhcp43-191 ~]#
Client logs: root@dhcp43-71 ~]# [root@dhcp43-71 ~]# mount | grep ozone 10.70.42.202:/ozone on /mnt/ozone type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) [root@dhcp43-71 ~]# cd /mnt/ozone [root@dhcp43-71 ozone]# ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 etc level01 level20 level20_sln .trashcan V6MO_newhdln [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# tail /etc/passwd ntp:x:38:38::/etc/ntp:/sbin/nologin saslauth:x:499:76:Saslauthd user:/var/empty/saslauth:/sbin/nologin postfix:x:89:89::/var/spool/postfix:/sbin/nologin sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin oprofile:x:16:16:Special user account to be used by OProfile:/home/oprofile:/sbin/nologin tcpdump:x:72:72::/:/sbin/nologin usr1:x:500:501::/home/usr1:/bin/bash u1:x:501:500::/home/u1:/bin/bash u2:x:502:500::/home/u2:/bin/bash u3:x:503:0::/home/u3:/bin/bash [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -ld level* drwxr-xr-x. 3 root root 134 Jun 18 18:36 level01 drwxrwxrwx. 3 root g1 4203 Jun 18 21:07 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# chgrp g1 level01 [root@dhcp43-71 ozone]# chown u1 level01 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -ld level* drwxr-xr-x. 3 u1 g1 134 Jun 18 18:36 level01 drwxrwxrwx. 3 root g1 4203 Jun 18 21:07 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# chown u1 level20_sln [root@dhcp43-71 ozone]# ls -ld level* drwxr-xr-x. 3 u1 g1 134 Jun 18 18:36 level01 drwxrwxrwx. 3 u1 g1 4203 Jun 18 21:07 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [root@dhcp43-71 ozone]# ls -ld level* drwxr-xr-x. 3 u1 g1 134 Jun 18 18:36 level01 drwxrwxrwx. 3 u1 g1 4203 Jun 18 21:07 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [root@dhcp43-71 ozone]# chown u2 level01 [root@dhcp43-71 ozone]# ls -ld level* drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 drwxrwxrwx. 3 u1 g1 4203 Jun 18 21:07 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [root@dhcp43-71 ozone]# su u1 [u1@dhcp43-71 ozone]$ cd level01 [u1@dhcp43-71 level01]$ ls -lrt total 204 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%4IDQ2AUHU8 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%BKCRD1RKEW drwxr-xr-x. 3 root root 4156 Jun 18 18:36 level11 [u1@dhcp43-71 level01]$ chown u1 level11 chown: changing ownership of `level11': Operation not permitted [u1@dhcp43-71 level01]$ chmod 777 level111 chmod: cannot access `level111': No such file or directory [u1@dhcp43-71 level01]$ chmod 777 level11 chmod: changing permissions of `level11': Operation not permitted [u1@dhcp43-71 level01]$ [u1@dhcp43-71 level01]$ [u1@dhcp43-71 level01]$ [u1@dhcp43-71 level01]$ [u1@dhcp43-71 level01]$ date Fri Jun 19 15:41:07 IST 2015 [u1@dhcp43-71 level01]$ pwd /mnt/ozone/level01 [u1@dhcp43-71 level01]$ cd .. [u1@dhcp43-71 ozone]$ ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 etc level01 level20 level20_sln .trashcan V6MO_newhdln [u1@dhcp43-71 ozone]$ [u1@dhcp43-71 ozone]$ [u1@dhcp43-71 ozone]$ touch level20_sln/ [u1@dhcp43-71 ozone]$ ls -ld level* drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 drwxrwxrwx. 3 u1 g1 4203 Jun 19 15:41 level20 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 [u1@dhcp43-71 ozone]$ [u1@dhcp43-71 ozone]$ [u1@dhcp43-71 ozone]$ mv level20_sln/ level20_sln2 mv: cannot move `level20_sln/' to `level20_sln2': Not a directory [u1@dhcp43-71 ozone]$ mv level20_sln level20_sln2 mv: cannot move `level20_sln' to `level20_sln2': Permission denied [u1@dhcp43-71 ozone]$ ls -lrt total 321 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln -> level20 drwxrwxrwx. 3 u1 g1 4203 Jun 19 15:41 level20 [u1@dhcp43-71 ozone]$ exit [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# mv level20_sln level20_sln2 [root@dhcp43-71 ozone]# ls -lrt total 321 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 4203 Jun 19 15:41 level20 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ln -s level01 level20_sln2 [root@dhcp43-71 ozone]# ls -lrt total 325 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 8252 Jun 19 15:44 level20 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -lrt total 325 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 8252 Jun 19 15:44 level20 [root@dhcp43-71 ozone]# ln -s level01 level20_sln2 ln: creating symbolic link `level20_sln2/level01': File exists [root@dhcp43-71 ozone]# cd level20_sln2/ 5582c254%%HVP139N6JI 5582c254%%UF1W41V6M0 level01 level30/ V6MO_hdln_sln [root@dhcp43-71 ozone]# cd level20_sln2/^C [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# touch a [root@dhcp43-71 ozone]# ln -s a a_symlink [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -lrt total 326 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 8252 Jun 19 15:44 level20 -rw-r--r--. 1 root root 0 Jun 19 15:46 a lrwxrwxrwx. 1 root root 1 Jun 19 15:46 a_symlink -> a [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# touch b [root@dhcp43-71 ozone]# ln -s b a_symlink_to_b [root@dhcp43-71 ozone]# ls -lrt total 326 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 8252 Jun 19 15:44 level20 -rw-r--r--. 1 root root 0 Jun 19 15:46 a lrwxrwxrwx. 1 root root 1 Jun 19 15:46 a_symlink -> a -rw-r--r--. 1 root root 0 Jun 19 15:47 b lrwxrwxrwx. 1 root root 1 Jun 19 15:47 a_symlink_to_b -> b [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ln -s b a_symlink ln: creating symbolic link `a_symlink': File exists [root@dhcp43-71 ozone]# rm -f a_symlink* [root@dhcp43-71 ozone]# ls -lrt total 325 drwxr-xr-x. 100 root root 16384 Jun 18 17:51 etc -rw-r--r--. 1 root root 102728 Jun 18 18:07 5582ba18%%FTZ39AW4RV.tar.gz drwxr-xr-x. 3 u2 g1 134 Jun 18 18:36 level01 -rwxr-xr-x. 2 u1 g1 102400 Jun 18 18:36 V6MO_newhdln -rw-r--r--. 1 root root 102400 Jun 18 20:47 5582baeb%%SNRP8ENTE9 lrwxrwxrwx. 1 root root 7 Jun 18 21:00 level20_sln2 -> level20 drwxrwxrwx. 3 u1 g1 8252 Jun 19 15:44 level20 -rw-r--r--. 1 root root 0 Jun 19 15:46 a -rw-r--r--. 1 root root 0 Jun 19 15:47 b [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# cd level01 [root@dhcp43-71 level01]# ls -lrt total 204 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%4IDQ2AUHU8 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%BKCRD1RKEW drwxr-xr-x. 3 root root 4156 Jun 18 18:36 level11 [root@dhcp43-71 level01]# cd level11/ [root@dhcp43-71 level11]# ls -lrt total 200 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%MSNPHQ27AD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%FLGHY1NZ1A drwxr-xr-x. 3 root root 134 Jun 18 18:36 level21 [root@dhcp43-71 level11]# cd level21/ 5582c255%%DMGID3IO9G 5582c256%%F8GH8RXVKJ level31/ [root@dhcp43-71 level11]# cd level21/ [root@dhcp43-71 level21]# ls -lrt total 208 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%DMGID3IO9G -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c256%%F8GH8RXVKJ drwxr-xr-x. 2 root root 8204 Jun 18 18:51 level31 [root@dhcp43-71 level21]# man chgrp [root@dhcp43-71 level21]# ls -lrt level31/ total 400 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%OQ65TSMRUD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%WRBYSGT7PZ -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c256%%1QNX332SKS -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c256%%MPIBDVQJQ4 [root@dhcp43-71 level21]# chgrp -R g1 level31 [root@dhcp43-71 level21]# ls -lrt level31/ total 400 -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c255%%OQ65TSMRUD -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c255%%WRBYSGT7PZ -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c256%%1QNX332SKS -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c256%%MPIBDVQJQ4 [root@dhcp43-71 level21]# ls -ld level31/ drwxr-xr-x. 2 root g1 8204 Jun 18 18:51 level31/ [root@dhcp43-71 level21]# cd ../../ [root@dhcp43-71 level01]# ls -lrt total 204 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%4IDQ2AUHU8 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%BKCRD1RKEW drwxr-xr-x. 3 root root 4156 Jun 18 18:36 level11 [root@dhcp43-71 level01]# pwd /mnt/ozone/level01 [root@dhcp43-71 level01]# cd .. [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# cd level11 -bash: cd: level11: No such file or directory [root@dhcp43-71 ozone]# cd level level01/ level20/ level20_sln2/ [root@dhcp43-71 ozone]# cd level01/level11/ [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# ls -lrt total 200 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%MSNPHQ27AD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%FLGHY1NZ1A drwxr-xr-x. 3 root root 134 Jun 18 18:36 level21 [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# ln hardlink_file 5582c255%%MSNPHQ27AD ln: accessing `hardlink_file': No such file or directory [root@dhcp43-71 level11]# ln 5582c255%%MSNPHQ27AD hardlink_file [root@dhcp43-71 level11]# ls -lrt total 300 -rw-r--r--. 2 root root 102400 Jun 18 18:36 hardlink_file -rw-r--r--. 2 root root 102400 Jun 18 18:36 5582c255%%MSNPHQ27AD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%FLGHY1NZ1A drwxr-xr-x. 3 root root 134 Jun 18 18:36 level21 [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# [root@dhcp43-71 level11]# mv hardlink_file ../hardlink [root@dhcp43-71 level11]# ls -lrt total 200 -rw-r--r--. 2 root root 102400 Jun 18 18:36 5582c255%%MSNPHQ27AD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%FLGHY1NZ1A drwxr-xr-x. 3 root root 134 Jun 18 18:36 level21 [root@dhcp43-71 level11]# ls -lrt ../ total 304 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%4IDQ2AUHU8 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%BKCRD1RKEW -rw-r--r--. 2 root root 102400 Jun 18 18:36 hardlink drwxr-xr-x. 3 root root 4156 Jun 19 15:54 level11 [root@dhcp43-71 level11]# cd [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# mount | grep ozone 10.70.42.202:/ozone on /mnt/ozone type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# [root@dhcp43-71 ~]# cd /mnt/ozone [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# df -k . Filesystem 1K-blocks Used Available Use% Mounted on 10.70.42.202:/ozone 33511424 172672 33338752 1% /mnt/ozone [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 a b etc level01 level20 level20_sln2 .trashcan V6MO_newhdln [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# crefi usage: crefi [-h] [-V] [-n FILES] [--size SIZE] [--random] [--max MAX] [--min MIN] [--single] [--multi] [-b BRDTH] [-d DEPTH] [-l FLEN] [-t {text,sparse,binary,tar}] [-I INTER] [--fop {create,rename,chmod,chown,chgrp,symlink,hardlink,truncate,setxattr}] [-R] [-T THREADS] mntpnt crefi: error: too few arguments [root@dhcp43-71 ozone]# rm -rf level20 [root@dhcp43-71 ozone]# crefi --multi -n 2 -b 4 -d 10 -t tar . 1 2015-06-19 18:04:41,246 - thread-0: Entering the directory level00 2015-06-19 18:04:41,560 - thread-0: Entering the directory level10 2015-06-19 18:04:41,879 - thread-0: Entering the directory level20 2015-06-19 18:04:42,185 - thread-0: Entering the directory level30 2015-06-19 18:04:42,500 - thread-0: Entering the directory level40 2015-06-19 18:04:42,847 - thread-0: Entering the directory level50 2015-06-19 18:04:43,151 - thread-0: Entering the directory level60 2015-06-19 18:04:43,479 - thread-0: Entering the directory level70 2015-06-19 18:04:43,837 - thread-0: Entering the directory level80 2015-06-19 18:04:44,168 - thread-0: Entering the directory level90 2015-06-19 18:04:44,463 - thread-0: Entering the directory level01 2015-06-19 18:04:44,714 - thread-0: Entering the directory level11 2015-06-19 18:04:44,967 - thread-0: Entering the directory level21 2015-06-19 18:04:45,212 - thread-0: Entering the directory level31 2015-06-19 18:04:45,520 - thread-0: Entering the directory level41 2015-06-19 18:04:45,863 - thread-0: Entering the directory level51 2015-06-19 18:04:46,186 - thread-0: Entering the directory level61 2015-06-19 18:04:46,551 - thread-0: Entering the directory level71 2015-06-19 18:04:46,909 - thread-0: Entering the directory level81 2015-06-19 18:04:47,223 - thread-0: Entering the directory level91 2015-06-19 18:04:47,549 - thread-0: Entering the directory level02 2015-06-19 18:04:47,832 - thread-0: Entering the directory level12 2015-06-19 18:04:48,134 - thread-0: Entering the directory level22 2015-06-19 18:04:48,437 - thread-0: Entering the directory level32 2015-06-19 18:04:48,751 - thread-0: Entering the directory level42 2015-06-19 18:04:49,064 - thread-0: Entering the directory level52 2015-06-19 18:04:49,385 - thread-0: Entering the directory level62 2015-06-19 18:04:49,695 - thread-0: Entering the directory level72 2015-06-19 18:04:49,999 - thread-0: Entering the directory level82 2015-06-19 18:04:50,306 - thread-0: Entering the directory level92 2015-06-19 18:04:50,664 - thread-0: Entering the directory level03 2015-06-19 18:04:50,955 - thread-0: Entering the directory level13 2015-06-19 18:04:51,311 - thread-0: Entering the directory level23 2015-06-19 18:04:51,628 - thread-0: Entering the directory level33 2015-06-19 18:04:51,950 - thread-0: Entering the directory level43 2015-06-19 18:04:52,262 - thread-0: Entering the directory level53 2015-06-19 18:04:52,597 - thread-0: Entering the directory level63 2015-06-19 18:04:52,913 - thread-0: Entering the directory level73 2015-06-19 18:04:53,227 - thread-0: Entering the directory level83 2015-06-19 18:04:53,542 - thread-0: Entering the directory level93 2015-06-19 18:04:53,872 - thread-0: finished creating files with throughput ---- 2.598 MBps [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 a b etc level00 level01 level02 level03 level20_sln2 .trashcan V6MO_newhdln [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# cd level01/ [root@dhcp43-71 level01]# ls 5582c255%%4IDQ2AUHU8 5582c255%%BKCRD1RKEW 55840c64%%KDYOF9Y4XD.tar.gz 55840c64%%MPVF41SC4I.tar.gz hardlink level11 [root@dhcp43-71 level01]# ls -lrt level11/ total 406 -rw-r--r--. 2 root root 102400 Jun 18 18:36 5582c255%%MSNPHQ27AD -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%FLGHY1NZ1A -rw-r--r--. 1 root root 102748 Jun 19 18:04 55840c64%%H4NYV4N8FY.tar.gz -rw-r--r--. 1 root root 102750 Jun 19 18:04 55840c64%%EWWTBU4WD1.tar.gz drwxr-xr-x. 3 root root 4217 Jun 19 18:04 level21 [root@dhcp43-71 level01]# ls -lrt level11/level21/ total 406 -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c256%%F8GH8RXVKJ -rw-r--r--. 1 root root 102400 Jun 18 18:36 5582c255%%DMGID3IO9G -rw-r--r--. 1 root root 102762 Jun 19 18:04 55840c64%%JAS3FVJCJF.tar.gz drwxr-xr-x. 3 root g1 4237 Jun 19 18:04 level31 -rw-r--r--. 1 root root 102770 Jun 19 18:04 55840c65%%PZML6HA4Y2.tar.gz [root@dhcp43-71 level01]# ls -lrt level11/level21/level31/ total 603 -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c255%%OQ65TSMRUD -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c255%%WRBYSGT7PZ -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c256%%MPIBDVQJQ4 -rw-r--r--. 1 root g1 102400 Jun 18 18:36 5582c256%%1QNX332SKS -rw-r--r--. 1 root root 102773 Jun 19 18:04 55840c65%%H68PIMOPLI.tar.gz -rw-r--r--. 1 root root 102775 Jun 19 18:04 55840c65%%B7UQFKODEC.tar.gz drwxr-xr-x. 3 root root 148 Jun 19 18:04 level41 [root@dhcp43-71 level01]# ls -lrt level11/level21/level31/level41/ total 203 -rw-r--r--. 1 root root 102785 Jun 19 18:04 55840c65%%B54R31P2H6.tar.gz drwxr-xr-x. 3 root root 148 Jun 19 18:04 level51 -rw-r--r--. 1 root root 102779 Jun 19 18:04 55840c65%%KY7890UJVK.tar.gz [root@dhcp43-71 level01]# ls -lrt level11/level21/level31/level41/level51/ total 203 -rw-r--r--. 1 root root 102799 Jun 19 18:04 55840c65%%5PR73SKQMG.tar.gz -rw-r--r--. 1 root root 102794 Jun 19 18:04 55840c66%%V67OV1DJZY.tar.gz drwxr-xr-x. 3 root root 148 Jun 19 18:04 level61 [root@dhcp43-71 level01]# ^C [root@dhcp43-71 level01]# [root@dhcp43-71 level01]# [root@dhcp43-71 level01]# mv level11/level21/level31/level41/level51/level61/level71/level81/level91/55840c67%% 55840c67%%SDS87LVZNA.tar.gz 55840c67%%ZBNTKUGCSO.tar.gz [root@dhcp43-71 level01]# mv level11/level21/level31/level41/level51/level61/level71/level81/level91/55840c67%% 55840c67%%SDS87LVZNA.tar.gz 55840c67%%ZBNTKUGCSO.tar.gz [root@dhcp43-71 level01]# mv level11/level21/level31/level41/level51/level61/level71/level81/level91/55840c67%%ZBNTKUGCSO.tar.gz 55840c67%%ZBNTKUGCSZZZZZZZ.tar.gz [root@dhcp43-71 level01]# ls 5582c255%%4IDQ2AUHU8 5582c255%%BKCRD1RKEW 55840c64%%KDYOF9Y4XD.tar.gz 55840c64%%MPVF41SC4I.tar.gz 55840c67%%ZBNTKUGCSZZZZZZZ.tar.gz hardlink level11 [root@dhcp43-71 level01]# [root@dhcp43-71 level01]# [root@dhcp43-71 level01]# [root@dhcp43-71 level01]# cd .. [root@dhcp43-71 ozone]# ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 a b etc level00 level01 level02 level03 level20_sln2 .trashcan V6MO_newhdln [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# mv level00/level10/ . [root@dhcp43-71 ozone]# ls -a . .. 5582ba18%%FTZ39AW4RV.tar.gz 5582baeb%%SNRP8ENTE9 a b etc level00 level01 level02 level03 level10 level20_sln2 .trashcan V6MO_newhdln [root@dhcp43-71 ozone]# ls -lrt level10 total 203 -rw-r--r--. 1 root root 102747 Jun 19 18:04 55840c61%%1FBXTH35MS.tar.gz -rw-r--r--. 1 root root 102748 Jun 19 18:04 55840c61%%SZRHLM6VL6.tar.gz drwxr-xr-x. 3 root root 148 Jun 19 18:04 level20 [root@dhcp43-71 ozone]# find level10/ | wc -l 27 [root@dhcp43-71 ozone]# find level03 | wc -l 30 [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]# [root@dhcp43-71 ozone]#
The only difference that I see between 'sesso1' and 'sesso2' is that I ran a 'glusterfind post' for 'sesso1' after updating the build, whereas it was already done for 'sesso2' before itself. Not really sure how that would affect.
Got the node 'node3' back up and waiting for the cluster to get back to healthy state. Ran 'glusterfind pre' command again on 'sesso1' and it does succeed without any error (as expected- since all the nodes are up). BUT, the output file that it generates is incorrect (displaying just 4 entries, whereas >100 entries are expected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# # after getting the node back up [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Disconnected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# gluster peer status Number of Peers: 3 Hostname: 10.70.42.202 Uuid: 2abb8dbb-48db-4c26-97bb-1ae1e11964f1 State: Peer in Cluster (Connected) Hostname: 10.70.42.30 Uuid: 38189182-2e48-4f9d-a342-f595389bf1bd State: Peer in Cluster (Connected) Hostname: 10.70.42.147 Uuid: dc3c70a2-aa43-4bf1-ba4d-1272616b69c1 State: Peer in Cluster (Connected) [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# ls -l /tmp/outo1.txt -rw-r--r--. 1 root root 68 Jun 19 21:18 /tmp/outo1.txt [root@dhcp43-191 ~]# vi /tmp/outo1.txt [root@dhcp43-191 ~]# ^C [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# cat /tmp/outo1.txt DELETE level20 NEW level00 NEW level02 NEW level03 NEW level10 [root@dhcp43-191 ~]# [root@dhcp43-191 ~]# glusterfind pre sesso1 ozone /tmp/outo1.txt --regenerate-outfile Generated output file /tmp/outo1.txt [root@dhcp43-191 ~]# ls -l /tmp/outo1.txt -rw-r--r--. 1 root root 68 Jun 19 21:19 /tmp/outo1.txt [root@dhcp43-191 ~]# cat /tmp/outo1.txt DELETE level20 NEW level00 NEW level02 NEW level03 NEW level10 [root@dhcp43-191 ~]# gluster v status ozone Status of volume: ozone Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.43.191:/rhs/thinbrick1/ozone 49153 0 Y 6807 Brick 10.70.42.202:/rhs/thinbrick1/ozone 49153 0 Y 31482 Brick 10.70.43.191:/rhs/thinbrick2/ozone 49154 0 Y 6815 Brick 10.70.42.202:/rhs/thinbrick2/ozone 49154 0 Y 31489 Brick 10.70.42.30:/rhs/thinbrick1/ozone 49153 0 Y 1999 Brick 10.70.42.147:/rhs/thinbrick1/ozone 49152 0 Y 31451 Brick 10.70.42.30:/rhs/thinbrick2/ozone 49154 0 Y 1998 Brick 10.70.42.147:/rhs/thinbrick2/ozone 49153 0 Y 31456 NFS Server on localhost 2049 0 Y 6783 Self-heal Daemon on localhost N/A N/A Y 6792 NFS Server on 10.70.42.202 2049 0 Y 31460 Self-heal Daemon on 10.70.42.202 N/A N/A Y 31467 NFS Server on 10.70.42.147 2049 0 Y 31437 Self-heal Daemon on 10.70.42.147 N/A N/A Y 31444 NFS Server on 10.70.42.30 2049 0 Y 2572 Self-heal Daemon on 10.70.42.30 N/A N/A Y 2580 Task Status of Volume ozone ------------------------------------------------------------------------------ There are no active volume tasks [root@dhcp43-191 ~]# gluster v info ozone Volume Name: ozone Type: Distributed-Replicate Volume ID: 9ef1ace8-505d-4d97-aa23-4296aa685f76 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: 10.70.43.191:/rhs/thinbrick1/ozone Brick2: 10.70.42.202:/rhs/thinbrick1/ozone Brick3: 10.70.43.191:/rhs/thinbrick2/ozone Brick4: 10.70.42.202:/rhs/thinbrick2/ozone Brick5: 10.70.42.30:/rhs/thinbrick1/ozone Brick6: 10.70.42.147:/rhs/thinbrick1/ozone Brick7: 10.70.42.30:/rhs/thinbrick2/ozone Brick8: 10.70.42.147:/rhs/thinbrick2/ozone Options Reconfigured: performance.readdir-ahead: on storage.build-pgfid: on changelog.changelog: on changelog.capture-del-path: on [root@dhcp43-191 ~]#
Sosreports updated at: http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1233622/
Feel free to open this bug if the issue still persists and you require a fix. Closing this as WONTFIX as we are not working on this bug, and treating it as a 'TIMEOUT'.