Bug 1538366

Summary: [GSS] Git clone --bare --mirror of git bundle fails when cloning on gluster storage
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Oonkwee Lim <olim>
Component: write-behindAssignee: Csaba Henk <csaba>
Status: CLOSED ERRATA QA Contact: Vivek Das <vdas>
Severity: low Docs Contact:
Priority: high    
Version: rhgs-3.2CC: amukherj, atumball, msaini, rgowdapp, rhinduja, rhs-bugs, storage-qa-internal, vdas
Target Milestone: ---   
Target Release: RHGS 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.12.2-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-04 06:42:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1503138    
Attachments:
Description Flags
the ansible bundle none

Description Oonkwee Lim 2018-01-24 22:59:47 UTC
Created attachment 1385817 [details]
the ansible bundle

Issue : Git clone --bare --mirror of git bundle fails when cloning on gluster storage

Issue description ------

The customer is encountering the following issue:

$ git clone --bare --mirror ../ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects

After a few retries it eventually succeeds:

$ git clone --bare --mirror ../ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.


Is reproducible ?  Yes, below are my steps:

Create a replica 3 volume.

Copy the ansible.bundle to /tmp.

Run a while loop:

[root@v3317a mnt]# mount | grep glusterfs
v3317a:/rep3-vol on /mnt type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
[root@v3317a mnt]# while :; do rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp; sleep 1; done
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
^C
=-=-=-=-=
Customer's Environment details :
GlusterFS version (from all nodes)
Kernel version used (if required)

I was able to reproduced  this on my 3.3.1 async.
I could not reproduce this on the local FS.

I have attached the ansible bundle file that is causing grief.

Comment 2 Amar Tumballi 2018-01-30 12:21:40 UTC
Looks like the issue is similar to the other DB consistency issues we saw. 

Can it be tried with disable open-behind translator (and if that doesn't work, try with disabling other perf translators.

Also, if its reproducible, can we add this as a test case in QE testing?

Comment 3 Oonkwee Lim 2018-01-31 03:10:51 UTC
Hello Amar,

Reproducer steps:

1) create and start a replica 3 volume
2) copy ansible.bundle to /tmp
3) mount the volume onto /mnt
4) run while loop
# while :; do rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp; sleep 1; done

I am trying out your suggestions.

Comment 4 Oonkwee Lim 2018-01-31 03:19:29 UTC
Turn off open-behind did not help:

[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle tempCloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects
[root@v3317a mnt]# gluster volume set tprep3-vol open-behind off
volume set: success
[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects

Comment 5 Oonkwee Lim 2018-01-31 03:27:05 UTC
Works with write-behind:

[root@v3317a mnt]# gluster v info tprep3-vol
 
Volume Name: tprep3-vol
Type: Replicate
Volume ID: d70f3972-e48e-4fb9-b8fa-39a64190bd0c
Status: Started
Snapshot Count: 2
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: v3317a:/rhs/brick2/threp3brick
Brick2: v3317b:/rhs/brick2/threp3brick
Brick3: v3317c:/rhs/brick2/threp3brick
Options Reconfigured:
nfs.disable: on

[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects

[root@v3317a mnt]# gluster volume set tprep3-vol performance.write-behind on
volume set: success

[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
fatal: premature end of pack file, 28 bytes missing
error: index-pack died
fatal: bad object bf4086161ade363a3a851805690f5babb24e9914
fatal: remote did not send all necessary objects

[root@v3317a mnt]# gluster volume set tprep3-vol performance.write-behind off
volume set: success

[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.

[root@v3317a mnt]#  git clone --bare --mirror /tmp/ansible.bundle temp
fatal: destination path 'temp' already exists and is not an empty directory.

[root@v3317a mnt]# rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.

[root@v3317a mnt]# rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.

=-=-=-=-= With the while loop =-=-=-=-=
[root@v3317a mnt]# while :; do rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp; sleep 1; done
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
^C

Comment 13 Vivek Das 2018-05-06 08:41:52 UTC
while :; do rm -rf temp; git clone --bare --mirror /tmp/ansible.bundle temp; sleep 1; done
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.
Resolving deltas: 100% (3/3), done.
Cloning into bare repository 'temp'...
Receiving objects: 100% (17/17), done.



Validated this bug as per the reproducer steps mentioned above. The issue is resolved. Marking this verified against the below version.

Version:
---------------
glusterfs-server-3.12.2-8.el7rhgs.x86_64

Comment 15 errata-xmlrpc 2018-09-04 06:42:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607