Bug 1851424

Summary: Bitrot - File is not signed after doing bitrot disable/enable
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Arthy Loganathan <aloganat>
Component: bitrotAssignee: Raghavendra Bhat <rabhat>
Status: CLOSED ERRATA QA Contact: Arthy Loganathan <aloganat>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.5CC: dwalveka, pprakash, puebele, rabhat, rhs-bugs, rkothiya, sheggodu, storage-qa-internal, sunkumar
Target Milestone: ---Keywords: Automation, AutomationBlocker, Regression, ZStream
Target Release: RHGS 3.5.z Batch Update 3   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-6.0-41 Doc Type: Bug Fix
Doc Text:
Previously, bit-rot feature had an issue with disabling and reenabling it on the same volume. If the bit-rot feature was disabled and enabled on the same volume it would mark some files as corrupted. With this update, the files are signed unconditionally when the bit-rot daemon comes up instead of skipping the files with bit-rot extended attributes.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2020-12-17 04:51:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Arthy Loganathan 2020-06-26 13:39:10 UTC 
Description of problem:
Files are not signed after doing disable and enable of bitrot.

Version-Release number of selected component (if applicable):
6.0-39.el8rhgs.x86_64
6.0-37.el7rhgs.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create 2*3 volume.
2. Enable bitrot.
3. Write a file, and ensure that file is signed.
4. Disable and enable bitrot.
5. Write a file.
6. Wait for 120s, to get file signed. 

Actual results:
File is not getting signed.

Expected results:
File should get signed.

Additional info:

[root@dhcp42-237 ~]# gluster volume bitrot testvol disable
volume bitrot: success bitrot disabled for volume testvol
[root@dhcp42-237 ~]# 
[root@dhcp42-237 ~]# gluster volume get testvol features.bitrot
Option                                  Value                                   
------                                  -----                                   
features.bitrot                         off                                     
[root@dhcp42-237 ~]# gluster volume bitrot testvol enable
volume bitrot: success bitrot enabled for volume testvol
[root@dhcp42-237 ~]# gluster volume get testvol features.bitrot
Option                                  Value                                   
------                                  -----                                   
features.bitrot                         on                                      
[root@dhcp42-237 ~]# 

[root@dhcp42-237 ~]# getfattr -d -m . -e hex /bricks/brick0/testvol_brick0/f1.txt 
getfattr: Removing leading '/' from absolute path names
# file: bricks/brick0/testvol_brick0/f1.txt
security.selinux=0x73797374656d5f753a6f626a6563745f723a676c7573746572645f627269636b5f743a733000
trusted.afr.dirty=0x000000000000000000000000
trusted.bit-rot.version=0x02000000000000005ef5d6bf0000f288
trusted.gfid=0xec6c484bf0074170b732c14755c00cab
trusted.gfid2path.75d497075af12ac9=0x30303030303030302d303030302d303030302d303030302d3030303030303030303030312f66312e747874

[root@dhcp42-237 ~]# sleep 120
[root@dhcp42-237 ~]# getfattr -d -m . -e hex /bricks/brick0/testvol_brick0/f1.txt 
getfattr: Removing leading '/' from absolute path names
# file: bricks/brick0/testvol_brick0/f1.txt
security.selinux=0x73797374656d5f753a6f626a6563745f723a676c7573746572645f627269636b5f743a733000
trusted.afr.dirty=0x000000000000000000000000
trusted.bit-rot.version=0x02000000000000005ef5d6bf0000f288
trusted.gfid=0xec6c484bf0074170b732c14755c00cab
trusted.gfid2path.75d497075af12ac9=0x30303030303030302d303030302d303030302d303030302d3030303030303030303030312f66312e747874



Bitd log snippet:
==================

[2020-06-26 11:48:39.141099] E [rpc-clnt.c:183:call_bail] 0-testvol-client-0: bailing out frame type(GlusterFS 4.x v1), op(FSETXATTR(36)), xid = 0x1a, unique = 27, sent = 2020-06-26 11:18:38.828546, timeout = 1800 for 10.70.42.237:49152
[2020-06-26 11:48:39.141159] W [MSGID: 114031] [client-rpc-fops_v2.c:1682:client4_0_fsetxattr_cbk] 0-testvol-client-0: remote operation failed [Transport endpoint is not connected]
[2020-06-26 11:48:39.141226] E [MSGID: 118007] [bit-rot.c:436:br_object_read_sign] 0-testvol-bit-rot-0: fsetxattr of signature to the object ec6c484b-f007-4170-b732-c14755c00cab failed
[2020-06-26 11:48:39.141248] E [MSGID: 118009] [bit-rot.c:624:br_sign_object] 0-testvol-bit-rot-0: reading and signing of the object ec6c484b-f007-4170-b732-c14755c00cab failed
[2020-06-26 11:48:39.141368] E [MSGID: 118010] [bit-rot.c:679:br_process_object] 0-testvol-bit-rot-0: SIGNING FAILURE [ec6c484b-f007-4170-b732-c14755c00cab]
[2020-06-26 12:07:39.299058] E [rpc-clnt.c:183:call_bail] 0-testvol-client-0: bailing out frame type(GlusterFS 4.x v1), op(LOOKUP(27)), xid = 0x35, unique = 54, sent = 2020-06-26 11:37:32.157904, timeout = 1800 for 10.70.42.237:49152
[2020-06-26 12:07:39.299133] W [MSGID: 114031] [client-rpc-fops_v2.c:2634:client4_0_lookup_cbk] 0-testvol-client-0: remote operation failed. Path: <gfid:a0ad302e-bd8f-4dd9-aa3c-01dff299adc9> (a0ad302e-bd8f-4dd9-aa3c-01dff299adc9) [Transport endpoint is not connected]
[2020-06-26 12:07:39.299375] E [MSGID: 118008] [bit-rot.c:470:br_log_object] 0-testvol-bit-rot-0: lookup() failed on object a0ad302e-bd8f-4dd9-aa3c-01dff299adc9 [Transport endpoint is not connected]
[2020-06-26 12:07:39.299406] E [MSGID: 118010] [bit-rot.c:679:br_process_object] 0-testvol-bit-rot-0: SIGNING FAILURE [a0ad302e-bd8f-4dd9-aa3c-01dff299adc9]
Comment 15 Arthy Loganathan 2020-09-15 05:41:50 UTC 
Files are getting signed after doing disable and enable of bitrot and no errors are seen in bitd.log

Verified the fix in,
glusterfs-server-6.0-45.el7rhgs.x86_64
glusterfs-server-6.0-45.el8rhgs.x86_64
Comment 22 errata-xmlrpc 2020-12-17 04:51:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (glusterfs bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5603