Bug 1790423 - Glusterfind pre command fails
Summary: Glusterfind pre command fails
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterfind
Version: 7
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Sunny Kumar
QA Contact: bugs@gluster.org
URL:
Whiteboard:
Depends On: 1789439 1801557
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-13 10:56 UTC by Sunny Kumar
Modified: 2020-02-11 08:32 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1789439
Environment:
Last Closed: 2020-01-14 19:44:58 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gluster.org Gerrit 24007 0 None Merged glusterfind: Fix py2/py3 issues 2020-01-14 19:44:57 UTC

Description Sunny Kumar 2020-01-13 10:56:02 UTC
+++ This bug was initially created as a clone of Bug #1789439 +++

Description of problem:
Glusterfind pre command fails 

Version-Release number of selected component (if applicable):
Distro : RHEL7.7 
Kernel version : 3.10.0-1061.el7.x86_64
Python 3.6.8
Gluster version : upstream at commit 8d55951 as shown below

    # git branch -v
    * master 8d55951 leases: re-alignment of structures

    # gluster --version | head -1 
    glusterfs 8dev


How reproducible:
Easily 

Steps to Reproduce:

1. Create and mount a volume replicated volume. 
 
2. create a session as follows :

        # glusterfind create s1 repvol

3. Update some files on the gluster client 

4. fire the glusterfind pre command :

       # glusterfind pre s1 repvol output.txt

===========================
Exact steps that I did : 
===========================
1. Create a replicated volume.

2. Mount the volume on the client as follows : 

    # mount -t glusterfs 192.168.1.18:/repvol /mnt/t1


3. Create a few files on the mounted volume : 

    # cd /mnt/t1
    # echo "hi" > file1
    # echo "hi" > file2    

4. Create a session on the gluster server :

    # glusterfind create s1 repvol
    192.168.1.22 - pre failed; stdout (including remote stderr):
    b'192.168.1.22: /testxfs/brick-a1/brick Historical Changelogs not available: [Errno 2] gf_history_changelog: No such file or directory\n'
    stderr:
    b''
    192.168.1.18 - pre failed; stdout (including remote stderr):
    b'Connection to 192.168.1.18 closed.\r\n'
    stderr:
    b'192.168.1.18: /testxfs/brick-a1/brick Historical Changelogs not available: [Errno 2] gf_history_changelog: No such file or directory\r\n'
    Generated output file /testxfs/output.txt



5. Create one more file on the client : 

    # echo "hi" > file3 

6. Fire glusterfind pre command :

    # glusterfind pre s1 repvol output.txt
    Post command is not run after last pre, use --regenerate-outfile

7. Fire glusterfind post command :  

    # glusterfind post s1 repvol 
    Session s1 with volume repvol updated

8. Fire the glusterfind pre command : 

    # glusterfind pre s1 repvol output.txt
    Generated output file /testxfs/output.txt

9. You can see the newly created file :  

    # cat /testxfs/output.txt
    NEW file3

10. Now append the same file on the client : 

    # echo "hi" >> file3 

11. On the server fire the glusterfind post command :  

    # glusterfind post s1 repvol 
    Session s1 with volume repvol updated

12. Now when pre command is fired we get the below error : 

    # glusterfind pre s1 repvol output.txt
    192.168.1.22 - pre failed; stdout (including remote stderr):
    b'Traceback (most recent call last):\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 458, in <module>\n    actual_end = changelog_crawl(args.brick, start, end, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 392, in changelog_crawl\n    return get_changes(brick, working_dir, log_file, start, end, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 355, in get_changes\n    gfid_to_all_paths_using_gfid2path(brick, changelog_data, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 147, in gfid_to_all_paths_using_gfid2path\n    hardlinks = enum_hard_links_using_gfid2path(brick, gfid, args)\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 123, in enum_hard_links_using_gfid2path\n    file_xattrs = xattr.list(p)\nAttributeError: module \'xattr\' has no attribute \'list\'\n'
    stderr:
    b''
    192.168.1.18 - pre failed; stdout (including remote stderr):
    b'Connection to 192.168.1.18 closed.\r\n'
    stderr:
    b'Traceback (most recent call last):\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 458, in <module>\r\n    actual_end = changelog_crawl(args.brick, start, end, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 392, in changelog_crawl\r\n    return get_changes(brick, working_dir, log_file, start, end, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 355, in get_changes\r\n    gfid_to_all_paths_using_gfid2path(brick, changelog_data, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 147, in gfid_to_all_paths_using_gfid2path\r\n    hardlinks = enum_hard_links_using_gfid2path(brick, gfid, args)\r\n  File "/usr/local/libexec/glusterfs/glusterfind/changelog.py", line 123, in enum_hard_links_using_gfid2path\r\n    file_xattrs = xattr.list(p)\r\nAttributeError: module \'xattr\' has no attribute \'list\'\r\n'
    Generated output file /testxfs/output.txt



Actual results:
glusterfind pre command gives error 

Expected results:
glusterfind pre command should not give error 

Additional info:

--- Additional comment from Worker Ant on 2020-01-10 11:28:00 UTC ---

REVIEW: https://review.gluster.org/23993 (glusterfind: Fix py2/py3 issues) posted (#1) for review on master by Kotresh HR

--- Additional comment from Kotresh HR on 2020-01-10 11:32:18 UTC ---

Two things here:

1. glusterfind uses pyxattr and not xattr module. I think, it's taken care in rpm install but on source install, it's taken fore python2. So install as below if not installed.
         #pip3 install pyxattr (for python3)
         #pip install pyxattr (for python2)
   
   And remove xattr module if installed. I think both share the same namespace and might conflict.
         #pip3 uninstall xattr  (for python3)
         #pip uninstall xattr (for python2)

2. There were issues w.r.t python2/python2 compatibility related dictionary and bytes/string handling. The patch https://review.gluster.org/23993 fixes the same.

--- Additional comment from Worker Ant on 2020-01-10 17:11:30 UTC ---

REVIEW: https://review.gluster.org/23993 (glusterfind: Fix py2/py3 issues) merged (#1) on master by Kotresh HR

Comment 1 Worker Ant 2020-01-13 15:22:50 UTC
REVIEW: https://review.gluster.org/24007 (glusterfind: Fix py2/py3 issues) posted (#1) for review on release-7 by Sunny Kumar

Comment 2 Worker Ant 2020-01-14 19:44:58 UTC
REVIEW: https://review.gluster.org/24007 (glusterfind: Fix py2/py3 issues) merged (#2) on release-7 by Rinku Kothiya


Note You need to log in before you can comment on or make changes to this bug.