Bug 1801557 - Glusterfind pre command fails
Summary: Glusterfind pre command fails
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterfind
Version: 6
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On: 1789439
Blocks: 1790423
TreeView+ depends on / blocked
 
Reported: 2020-02-11 08:32 UTC by Sunny Kumar
Modified: 2020-03-12 12:56 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1789439
Environment:
Last Closed: 2020-03-12 12:56:58 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Sunny Kumar 2020-02-11 08:32:55 UTC
+++ This bug was initially created as a clone of Bug #1789439 +++

Description of problem:
Glusterfind pre command fails 

Version-Release number of selected component (if applicable):
Distro : RHEL7.7 
Kernel version : 3.10.0-1061.el7.x86_64
Python 3.6.8
Gluster version : upstream at commit 8d55951 as shown below

    # git branch -v
    * master 8d55951 leases: re-alignment of structures

    # gluster --version | head -1 
    glusterfs 8dev


How reproducible:
Easily 

Steps to Reproduce:

1. Create and mount a volume replicated volume. 
 
2. create a session as follows :

        # glusterfind create s1 repvol

3. Update some files on the gluster client 

4. fire the glusterfind pre command :

       # glusterfind pre s1 repvol output.txt

===========================
Exact steps that I did : 
===========================
1. Create a replicated volume.

2. Mount the volume on the client as follows : 

    # mount -t glusterfs 192.168.1.18:/repvol /mnt/t1


3. Create a few files on the mounted volume : 

    # cd /mnt/t1
    # echo "hi" > file1
    # echo "hi" > file2    

4. Create a session on the gluster server :

    # glusterfind create s1 repvol
    192.168.1.22 - pre failed; stdout (including remote stderr):
    b'192.168.1.22: /testxfs/brick-a1/brick Historical Changelogs not available: [Errno 2] gf_history_changelog: No such file or directory\n'
    stderr:
    b''
    192.168.1.18 - pre failed; stdout (including remote stderr):
    b'Connection to 192.168.1.18 closed.\r\n'
    stderr:
    b'192.168.1.18: /testxfs/brick-a1/brick Historical Changelogs not available: [Errno 2] gf_history_changelog: No such file or directory\r\n'
    Generated output file /testxfs/output.txt



5. Create one more file on the client : 

    # echo "hi" > file3 

6. Fire glusterfind pre command :

    # glusterfind pre s1 repvol output.txt
    Post command is not run after last pre, use --regenerate-outfile

7. Fire glusterfind post command :  

    # glusterfind post s1 repvol 
    Session s1 with volume repvol updated

8. Fire the glusterfind pre command : 

    # glusterfind pre s1 repvol output.txt
    Generated output file /testxfs/output.txt

9. You can see the newly created file :  

    # cat /testxfs/output.txt
    NEW file3

10. Now append the same file on the client : 

    # echo "hi" >> file3 

11. On the server fire the glusterfind post command :  

    # glusterfind post s1 repvol 
    Session s1 with volume repvol updated

12. Now when pre command is fired we get the below error : 

    # glusterfind pre s1 repvol output.txt
    192.168.1.22 - pre failed; stdout (including remote stderr):
    b'Traceback (most recent call last):\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 458, in <module>\n    actual_end = changelog_crawl(args.brick, start, end, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 392, in changelog_crawl\n    return get_changes(brick, working_dir, log_file, start, end, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 355, in get_changes\n    gfid_to_all_paths_using_gfid2path(brick, changelog_data, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 147, in gfid_to_all_paths_using_gfid2path\n    hardlinks = enum_hard_links_using_gfid2path(brick, gfid, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 123, in enum_hard_links_using_gfid2path\n    file_xattrs = xattr.list(p)\nAttributeError: module \'xattr\' has no attribute \'list\'\n'
    stderr:
    b''
    192.168.1.18 - pre failed; stdout (including remote stderr):
    b'Connection to 192.168.1.18 closed.\r\n'
    stderr:
    b'Traceback (most recent call last):\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 458, in <module>\r\n    actual_end = changelog_crawl(args.brick, start, end, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 392, in changelog_crawl\r\n    return get_changes(brick, working_dir, log_file, start, end, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 355, in get_changes\r\n    gfid_to_all_paths_using_gfid2path(brick, changelog_data, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 147, in gfid_to_all_paths_using_gfid2path\r\n    hardlinks = enum_hard_links_using_gfid2path(brick, gfid, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 123, in enum_hard_links_using_gfid2path\r\n    file_xattrs = xattr.list(p)\r\nAttributeError: module \'xattr\' has no attribute \'list\'\r\n'
    Generated output file /testxfs/output.txt



Actual results:
glusterfind pre command gives error 

Expected results:
glusterfind pre command should not give error 

Additional info:

--- Additional comment from Worker Ant on 2020-01-10 11:28:00 UTC ---

REVIEW: https://review.gluster.org/23993 (glusterfind: Fix py2/py3 issues) posted (#1) for review on master by Kotresh HR

--- Additional comment from Kotresh HR on 2020-01-10 11:32:18 UTC ---

Two things here:

1. glusterfind uses pyxattr and not xattr module. I think, it's taken care in rpm install but on source install, it's taken fore python2. So install as below if not installed.
         #pip3 install pyxattr (for python3)
         #pip install pyxattr (for python2)
   
   And remove xattr module if installed. I think both share the same namespace and might conflict.
         #pip3 uninstall xattr  (for python3)
         #pip uninstall xattr (for python2)

2. There were issues w.r.t python2/python2 compatibility related dictionary and bytes/string handling. The patch https://review.gluster.org/23993 fixes the same.

--- Additional comment from Worker Ant on 2020-01-10 17:11:30 UTC ---

REVIEW: https://review.gluster.org/23993 (glusterfind: Fix py2/py3 issues) merged (#1) on master by Kotresh HR

Comment 1 Ravishankar N 2020-02-18 06:40:33 UTC
Assigning to Sunny since he has cloned the BZ from master, possibly for backporting the fix(es). Leaving a need-info for confirmation.

Comment 2 Sunny Kumar 2020-02-18 11:57:15 UTC
Blocked on https://review.gluster.org/#/c/glusterfs/+/24107/. Once it gets merged will backport it.

Comment 3 Worker Ant 2020-03-12 12:56:58 UTC
This bug is moved to https://github.com/gluster/glusterfs/issues/974, and will be tracked there from now on. Visit GitHub issues URL for further details


Note You need to log in before you can comment on or make changes to this bug.