Description of problem: #gluster volume info Volume Name: repl Type: Distributed-Replicate Volume ID: c5ebb095-8fa1-48d3-bab0-4a906891bde8 Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: deepthought:/data/3.5b1 Brick2: deepthought:/data/3.5b2 Brick3: deepthought:/data/3.5b3 Brick4: deepthought:/data/3.5b4 Options Reconfigured: network.compression.mode: server network.compression: on features.encryption: on performance.quick-read: off performance.write-behind: off performance.open-behind: off encryption.master-key: /var/lib/aes.key /mnt/glusterfs is a native client mount. [root@deepthought mnt]# echo "hello,world" > /mnt/glusterfs/2 [root@deepthought mnt]# cat /mnt/glusterfs/2 1�E�{�v��g Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
AFR does not propagate xdata. This could be the reason for this bug as the volume type is Distributed-Replicate.
Prashanth, do you know if this is still the case with current releases?
Haven't tested yet. Will test and update the BZ.
Ppai , It seems to be pretty old bug , can you please provide an update? If the issue is not present in the latest code, please close the bug
# gluster volume info test Volume Name: test Type: Distributed-Replicate Volume ID: 2a31cc3b-0a39-4aeb-a041-265330a8a14e Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: saio:/export/brick1/b Brick2: saio:/export/brick2/b Brick3: saio:/export/brick3/b Brick4: saio:/export/brick4/b Options Reconfigured: performance.io-cache: off performance.read-ahead: off features.encryption: on encryption.master-key: /root/keys/master.key performance.open-behind: off performance.write-behind: off performance.quick-read: off network.compression: on performance.readdir-ahead: on auto-delete: disable Tried this out and at first it did seem to work fine as shown below. # echo "hello world" > /mnt/gluster-object/test/o1 # cat /mnt/gluster-object/test/o1 hello world But NOT quite for bigger files: root# ls -l ./rabbitmq-server-3.3.4-1.noarch.rpm -rw-r--r--. 1 root root 3943201 Jun 24 2014 ./rabbitmq-server-3.3.4-1.noarch.rpm root# cp ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/ cp: overwrite ‘/mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm’? y root# md5sum ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm da246de9bd2288b090c699e272b19ece ./rabbitmq-server-3.3.4-1.noarch.rpm 1f19aa26bb5cace41d022483fd6df4d5 /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm root# echo 3 > /proc/sys/vm/drop_caches root# md5sum ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm da246de9bd2288b090c699e272b19ece ./rabbitmq-server-3.3.4-1.noarch.rpm 211482e270fa7c21ff66a0ac7eba6f9e /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm root# echo 3 > /proc/sys/vm/drop_caches root# md5sum ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm da246de9bd2288b090c699e272b19ece ./rabbitmq-server-3.3.4-1.noarch.rpm 379d6b5dde099579ef6a2fc52baae1c3 /mnt/gluster-object/test/rabbitmq-server-3.3.4-1.noarch.rpm As seen above, the file content is different EVERY time it is read! I also see the mount logs flooded with this: [2016-01-19 13:05:03.962772] E [mem-pool.c:554:mem_put] (-->/usr/local/lib/glusterfs/3.8dev/xlator/mount/fuse.so(+0x8a4b) [0x7f259df4fa4b] -->/usr/local/lib/glusterfs/3.8dev/xlator/mount/fuse.so(+0x899c) [0x7f259df4f99c] -->/usr/local/lib/libglusterfs.so.0(mem_put+0x162) [0x7f25a6b24961] ) 0-mem-pool: mem-pool ptr is NULL So this issue exists in latest master code too.
However, compression and encryption xlator work fine when each are exclusively turned on (with other being off): root# gluster volume set test encryption off volume set: success root# umount /mnt/gluster-object/test root# mount -t glusterfs `hostname`:test /mnt/gluster-object/test root# cp ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/dir/ cp: overwrite ‘/mnt/gluster-object/test/dir/rabbitmq-server-3.3.4-1.noarch.rpm’? y root# md5sum ./rabbitmq-server-3.3.4-1.noarch.rpm /mnt/gluster-object/test/dir/rabbitmq-server-3.3.4-1.noarch.rpm da246de9bd2288b090c699e272b19ece ./rabbitmq-server-3.3.4-1.noarch.rpm da246de9bd2288b090c699e272b19ece /mnt/gluster-object/test/dir/rabbitmq-server-3.3.4-1.noarch.rpm
The feature is now deprecated from glusterfs-6.0 release. We will not be taking any work on this area for now. If this is a critical feature for you, please feel free to raise the issue in github or mailing list.