Bug 1023382 - Dist-geo-rep : geo-rep went to faulty with traceback OSError: [Errno 22] Invalid argument: '.gfid/00000000-0000-0000-0000-000000000000'
Summary: Dist-geo-rep : geo-rep went to faulty with traceback OSError: [Errno 22] Inva...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: geo-replication
Version: 2.1
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Amar Tumballi
QA Contact: Vijaykumar Koppad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-25 10:16 UTC by Vijaykumar Koppad
Modified: 2014-08-25 00:50 UTC (History)
7 users (show)

Fixed In Version: glusterfs-3.4.0.42rhs-1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-11-27 15:44:15 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1769 0 normal SHIPPED_LIVE Red Hat Storage 2.1 enhancement and bug fix update #1 2013-11-27 20:17:39 UTC

Description Vijaykumar Koppad 2013-10-25 10:16:23 UTC
Description of problem: While doing renames, geo-rep went to faulty with the traceback OSError: [Errno 22] Invalid argument: '.gfid/00000000-0000-0000-0000-000000000000', and got stuck there. 

Traceback
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[2013-10-25 15:31:35.497792] I [master(/bricks/brick1):358:crawlwrap] _GMaster: 20 crawls, 0 turns
[2013-10-25 15:32:35.567895] I [master(/bricks/brick1):358:crawlwrap] _GMaster: 19 crawls, 2 turns
[2013-10-25 15:33:10.913541] E [syncdutils(/bricks/brick1):207:log_raise_exception] <top>: FAIL: 
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 150, in main
    main_i()
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 530, in main_i
    local.service_loop(*[r for r in [remote] if r])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1074, in service_loop
    g2.crawlwrap()
  File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 369, in crawlwrap
    self.crawl()
  File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 799, in crawl
    self.process(changes)
  File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 760, in process
    if self.process_change(change, done, retry):
  File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 735, in process_change
    st = lstat(go)
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 399, in lstat
    return os.lstat(e)
OSError: [Errno 22] Invalid argument: '.gfid/00000000-0000-0000-0000-000000000000'
[2013-10-25 15:33:10.919718] I [syncdutils(/bricks/brick1):159:finalize] <top>: exiting.
[2013-10-25 15:33:10.933976] I [monitor(monitor):81:set_state] Monitor: new state: faulty

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Version-Release number of selected component (if applicable): glusterfs-3.4.0.36rhs-1.el6rhs.x86_64


How reproducible: Didn't try to reproduce.


Steps to Reproduce: 
1.Create and start a geo-rep relationship between master and slave.
2. Enable quota on master 
3.create data, symlink and hardlinks let them sync 
4.And start renames of the files created 
5.Check the status

Actual results: the status went to faulty and stuck there . 


Expected results: It shouldn't go to faulty. 


Additional info:

Comment 2 Vijaykumar Koppad 2013-10-25 11:08:59 UTC
I was able to reproduce them even while creating hardlinks. 

The particular changelogs have NULL GFID, like

E 69be80b8-6037-44fd-9ba1-0523e896b7f6 LINK 11794ef1-7632-42ba-ac28-42ab9406042f%2F526a4de8%25%257WUVHEDYJH
M 69be80b8-6037-44fd-9ba1-0523e896b7f6
M 00000000-0000-0000-0000-000000000000
M 61bd5f6c-9d62-4262-98d7-723a4647f8bb
M 00000000-0000-0000-0000-000000000000
M 94df0a4a-17eb-4d37-ac58-dbefb8ec9b05

Comment 3 Vijaykumar Koppad 2013-10-25 13:15:50 UTC
Earlier, the quota was enabled. Even with quota disables it is reproducible.

Comment 4 Amar Tumballi 2013-11-01 10:10:10 UTC
can you confirm with .38rhs? because two fixes which went into solve some setxattr() inode being NULL here.. Also looks like bug 1025476 is related.

Comment 5 Vijaykumar Koppad 2013-11-13 06:48:30 UTC
Tried couple of times on the build glusterfs-3.4.0.43rhs-1. Seems to work fine.

Comment 6 errata-xmlrpc 2013-11-27 15:44:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1769.html


Note You need to log in before you can comment on or make changes to this bug.