Bug 1282319 - DHT : for many operation directory/file path is '(null)' in brick log
DHT : for many operation directory/file path is '(null)' in brick log
Status: ASSIGNED
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute (Show other bugs)
3.1
x86_64 Linux
low Severity medium
: ---
: ---
Assigned To: Nithya Balachandran
Prasad Desala
: ZStream
: 1031016 (view as bug list)
Depends On: 1031016
Blocks: 1456361
  Show dependency treegraph
 
Reported: 2015-11-16 00:39 EST by Sakshi
Modified: 2017-11-05 13:03 EST (History)
9 users (show)

See Also:
Fixed In Version: 3.7.9-10
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1031016
: 1456361 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sakshi 2015-11-16 00:39:01 EST
+++ This bug was initially created as a clone of Bug #1031016 +++

Description of problem:
In case of DHT volume, brick logs has '(null)' as a path for operations like OPENDIR, MKDIR, MKNOD, SETXATTR

e.g
log snippet from brick

[2013-11-14 01:52:01.592994] I [server-rpc-fops.c:705:server_opendir_cbk] 0-dht-server: 61510: OPENDIR (null) (16bb3787-361d-4977-8c66-
a62b8d2f70a3) ==> (No such file or directory)
[2013-11-14 03:14:56.011608] I [server-rpc-fops.c:528:server_mkdir_cbk] 0-dht-server: 32254: MKDIR (null) (00000000-0000-0000-0000-0000
00000001/{1.1000000}) ==> (File exists)
[2013-11-14 03:18:18.304290] I [server-rpc-fops.c:438:server_access_cbk] 0-dht-server: 47194: ACCESS (null) (2d0887f8-358d-4758-949f-db
bd06d67265) ==> (No such file or directory)
[2013-11-14 03:18:18.317609] I [server-rpc-fops.c:705:server_opendir_cbk] 0-dht-server: 47197: OPENDIR (null) (2d0887f8-358d-4758-949f-
dbbd06d67265) ==> (No such file or directory)
...
[2013-11-14 06:32:14.172144] I [server-rpc-fops.c:898:_gf_server_log_setxattr_failure] 0-dht-server: 164153: SETXATTR (null) (85258d5e-
fde6-4452-a4cf-71ef0568129a) ==> trusted.glusterfs.dht
[2013-11-14 06:32:14.174551] I [server-rpc-fops.c:528:server_mkdir_cbk] 0-dht-server: 285540: MKDIR (null) (00000000-0000-0000-0000-000
000000001/118) ==> (File exists)
[2013-11-14 06:32:14.202874] I [server-rpc-fops.c:575:server_mknod_cbk] 0-dht-server: 285551: MKNOD (null) (bf070eaf-2822-40b7-ab44-24d
91926db45/f7) ==> (File exists)

In most of the cases reason  is '(File exists)' or ' (No such file or directory)'


Version-Release number of selected component (if applicable):
============================================================= 
3.4.0.44rhs-1.el6rhs.x86_64



How reproducible:
always


Steps to Reproduce:
1.create DHT volume and mout it using FUSE and NFS
2. create, delete files/Directories from both mount point simultaneously.
3. Keep checking logs.

Actual results:
for many operation directory/file path is  '(null)' in brick  log 

Expected results:
It should show path of that Dir/file

Additional info:

--- Additional comment from Rachana Patel on 2013-11-15 08:34:31 EST ---

sosreport @ http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1030309

volume info:-
[root@7-VM3 glusterfs]# gluster v info dht
 
Volume Name: dht
Type: Distribute
Volume ID: 29b35eed-9b1a-443b-914a-e77d9452eb47
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 10.70.36.132:/rhs/brick4/d1
Brick2: 10.70.36.132:/rhs/brick4/d2
Brick3: 10.70.36.132:/rhs/brick4/d3
Brick4: 10.70.36.132:/rhs/brick4/d4



mount info:-

[root@rhs-client22 dht-nfs]# mount | grep dht
10.70.36.132:/dht on /mnt/dht type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
10.70.36.132:/dht on /mnt/dht-nfs type nfs (rw,addr=10.70.36.132)


--- Additional comment from Amar Tumballi on 2013-12-02 05:09:24 EST ---

(null) in here is the 'path' of the file. Yes it could have not been printed at all as a proper fix.

Above there are two type of operations:

inode ops (Access, opendir etc), where there is a log of 'gfid' which can help people to debug. (Also, these calls may be also path less, gfid only fops)..

in entry ops: (mkdir and mknod), the gfid of parent, and basename is logged to help debugging.
Comment 2 Susant Kumar Palai 2015-11-27 07:21:58 EST
*** Bug 1031016 has been marked as a duplicate of this bug. ***
Comment 6 Nithya Balachandran 2016-09-21 00:52:17 EDT
Most of these messages are now logged only in case of errors.
Comment 7 Prasad Desala 2016-09-21 02:17:55 EDT
Tested this BZ with glusterfs version 3.7.9-12.el7rhgs.x86_64. 

Brick logs has '(null)' as path for ACCESS, STAT, LOOKUP.

Below are brick logs output snippets:
=====================================
[2016-09-21 05:05:41.320200] I [MSGID: 115055] [server-rpc-fops.c:423:server_access_cbk] 0-distrep-server: 1421998: ACCESS (null) (2bb72069-3217-4c38-b660-abc90b40f118) ==> (Stale file handle) [Stale file handle]
rpc-fops.c:1748:server_stat_cbk] 0-distrep-server: 1405862: STAT (null) (5c449912-1199-48c1-8fbc-996ec6a2962a) ==> (Stale file handle) [Stale file handle]
[2016-09-21 05:03:34.570429] D [MSGID: 115050] [server-rpc-fops.c:179:server_lookup_cbk] 0-distrep-server: 1131709: LOOKUP /linux-4.6.4/Documentation/RCU/rculist_nulls.txt (fa65c27b-d7db-41ab-8459-6b7c827dd78e/rculist_nulls.txt) ==> (No such file or directory) [No such file or directory]

Moving this bug to ASSIGNED.

Note You need to log in before you can comment on or make changes to this bug.