Bug 1002419 - Unable to remove directories after upgrade from U5 to 2.1
Unable to remove directories after upgrade from U5 to 2.1
Status: CLOSED EOL
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterfs (Show other bugs)
2.1
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Bug Updates Notification Mailing List
storage-qa-internal@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-29 03:29 EDT by Sachidananda Urs
Modified: 2015-11-27 05:50 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-11-27 05:50:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
sosreports (1.58 MB, application/x-bzip)
2013-08-29 03:29 EDT, Sachidananda Urs
no flags Details

  None (edit)
Description Sachidananda Urs 2013-08-29 03:29:30 EDT
Created attachment 791627 [details]
sosreports

Description of problem:
Not able to delete directories after upgrade. This happens intermittently, had not observed in previous upgrade exercises.

rm -rf exits with error message:
rm: cannot remove `sync-2.1/2/linux-3.10.9/drivers/atm': Directory not empty

And rsync throws errors like:

rsync: mkstemp "/mnt/replicate/sync-2.1/2/linux-3.10.9/drivers/auxdisplay/.Kconfig.hjyl3R" failed: No such file 
or directory (2)
rsync: mkstemp "/mnt/replicate/sync-2.1/2/linux-3.10.9/drivers/auxdisplay/.Makefile.0jeGqB" failed: No such file
 or directory (2)
rsync: mkstemp "/mnt/replicate/sync-2.1/2/linux-3.10.9/drivers/auxdisplay/.cfag12864b.c.W20pOk" failed: No such 
file or directory (2)

NOTE: There were not disconnects or server downs. And strangely no `E' or `W' level logs in servers or clients.

Version-Release number of selected component (if applicable):
glusterfs 3.4.0.24rhs built on Aug 27 2013 08:08:42

How reproducible:
Intermittent 

Steps to Reproduce:
1. Install U5 create lots of files and directories
2. Upgrade from U5 to Big Bend release 3.4.0.24 or latest
3. Try to remove the files and directories, create new files using rsync.

Actual results:
rm exits with failure
rsync intermittently fails

Expected results:
There should not be any failures.

Additional info:

[root@Anshi-1 ~]# gluster vol info
 
Volume Name: upgrade
Type: Replicate
Volume ID: a4a20b8c-e6fa-4b2a-b69c-106aebc5db2f
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.70.37.75:/rhs/brick1/rep-0
Brick2: 10.70.37.116:/rhs/brick1/rep-0
Options Reconfigured:
performance.read-ahead: on
performance.write-behind: on
performance.flush-behind: on


Attaching sosreports and client logs. Log snippet from client:

[2013-08-29 08:39:14.489896] I [dht-layout.c:633:dht_layout_normalize] 0-upgrade-dht: found anomalies in /2/linux-3.10.9/arch/powerpc/configs/44x. holes=1 overlaps=0 missing=0 down=0 misc=0
[2013-08-29 08:39:14.504621] I [dht-layout.c:633:dht_layout_normalize] 0-upgrade-dht: found anomalies in /2/linux-3.10.9/arch/powerpc/configs/52xx. holes=1 overlaps=0 missing=0 down=0 misc=0
[2013-08-29 08:39:14.516102] I [dht-layout.c:633:dht_layout_normalize] 0-upgrade-dht: found anomalies in /2/linux-3.10.9/arch/powerpc/configs/83xx. holes=1 overlaps=0 missing=0 down=0 misc=0
[2013-08-29 08:39:14.529906] I [dht-layout.c:633:dht_layout_normalize] 0-upgrade-dht: found anomalies in /2/linux-3.10.9/arch/powerpc/configs/85xx. holes=1 overlaps=0 missing=0 down=0 misc=0
[2013-08-29 08:39:14.544157] I [dht-layout.c:633:dht_layout_normalize] 0-upgrade-dht: found anomalies in /2/linux-3.10.9/arch/powerpc/configs/86xx. holes=1 overlaps=0 missing=0 down=0 misc=0

Note You need to log in before you can comment on or make changes to this bug.