Bug 1282319 - DHT : for many operation directory/file path is '(null)' in brick log
Summary: DHT : for many operation directory/file path is '(null)' in brick log
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: core
Version: rhgs-3.1
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ---
: ---
Assignee: Vijay Bellur
QA Contact: Rahul Hinduja
URL:
Whiteboard:
: 1031016 (view as bug list)
Depends On: 1031016
Blocks: 1456361
TreeView+ depends on / blocked
 
Reported: 2015-11-16 05:39 UTC by Sakshi
Modified: 2018-04-16 03:05 UTC (History)
10 users (show)

Fixed In Version: 3.7.9-10
Doc Type: Bug Fix
Doc Text:
Clone Of: 1031016
: 1456361 (view as bug list)
Environment:
Last Closed: 2018-04-16 03:05:59 UTC
Embargoed:


Attachments (Terms of Use)

Description Sakshi 2015-11-16 05:39:01 UTC
+++ This bug was initially created as a clone of Bug #1031016 +++

Description of problem:
In case of DHT volume, brick logs has '(null)' as a path for operations like OPENDIR, MKDIR, MKNOD, SETXATTR

e.g
log snippet from brick

[2013-11-14 01:52:01.592994] I [server-rpc-fops.c:705:server_opendir_cbk] 0-dht-server: 61510: OPENDIR (null) (16bb3787-361d-4977-8c66-
a62b8d2f70a3) ==> (No such file or directory)
[2013-11-14 03:14:56.011608] I [server-rpc-fops.c:528:server_mkdir_cbk] 0-dht-server: 32254: MKDIR (null) (00000000-0000-0000-0000-0000
00000001/{1.1000000}) ==> (File exists)
[2013-11-14 03:18:18.304290] I [server-rpc-fops.c:438:server_access_cbk] 0-dht-server: 47194: ACCESS (null) (2d0887f8-358d-4758-949f-db
bd06d67265) ==> (No such file or directory)
[2013-11-14 03:18:18.317609] I [server-rpc-fops.c:705:server_opendir_cbk] 0-dht-server: 47197: OPENDIR (null) (2d0887f8-358d-4758-949f-
dbbd06d67265) ==> (No such file or directory)
...
[2013-11-14 06:32:14.172144] I [server-rpc-fops.c:898:_gf_server_log_setxattr_failure] 0-dht-server: 164153: SETXATTR (null) (85258d5e-
fde6-4452-a4cf-71ef0568129a) ==> trusted.glusterfs.dht
[2013-11-14 06:32:14.174551] I [server-rpc-fops.c:528:server_mkdir_cbk] 0-dht-server: 285540: MKDIR (null) (00000000-0000-0000-0000-000
000000001/118) ==> (File exists)
[2013-11-14 06:32:14.202874] I [server-rpc-fops.c:575:server_mknod_cbk] 0-dht-server: 285551: MKNOD (null) (bf070eaf-2822-40b7-ab44-24d
91926db45/f7) ==> (File exists)

In most of the cases reason  is '(File exists)' or ' (No such file or directory)'


Version-Release number of selected component (if applicable):
============================================================= 
3.4.0.44rhs-1.el6rhs.x86_64



How reproducible:
always


Steps to Reproduce:
1.create DHT volume and mout it using FUSE and NFS
2. create, delete files/Directories from both mount point simultaneously.
3. Keep checking logs.

Actual results:
for many operation directory/file path is  '(null)' in brick  log 

Expected results:
It should show path of that Dir/file

Additional info:

--- Additional comment from Rachana Patel on 2013-11-15 08:34:31 EST ---

sosreport @ http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1030309

volume info:-
[root@7-VM3 glusterfs]# gluster v info dht
 
Volume Name: dht
Type: Distribute
Volume ID: 29b35eed-9b1a-443b-914a-e77d9452eb47
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 10.70.36.132:/rhs/brick4/d1
Brick2: 10.70.36.132:/rhs/brick4/d2
Brick3: 10.70.36.132:/rhs/brick4/d3
Brick4: 10.70.36.132:/rhs/brick4/d4



mount info:-

[root@rhs-client22 dht-nfs]# mount | grep dht
10.70.36.132:/dht on /mnt/dht type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
10.70.36.132:/dht on /mnt/dht-nfs type nfs (rw,addr=10.70.36.132)


--- Additional comment from Amar Tumballi on 2013-12-02 05:09:24 EST ---

(null) in here is the 'path' of the file. Yes it could have not been printed at all as a proper fix.

Above there are two type of operations:

inode ops (Access, opendir etc), where there is a log of 'gfid' which can help people to debug. (Also, these calls may be also path less, gfid only fops)..

in entry ops: (mkdir and mknod), the gfid of parent, and basename is logged to help debugging.

Comment 2 Susant Kumar Palai 2015-11-27 12:21:58 UTC
*** Bug 1031016 has been marked as a duplicate of this bug. ***

Comment 6 Nithya Balachandran 2016-09-21 04:52:17 UTC
Most of these messages are now logged only in case of errors.

Comment 7 Prasad Desala 2016-09-21 06:17:55 UTC
Tested this BZ with glusterfs version 3.7.9-12.el7rhgs.x86_64. 

Brick logs has '(null)' as path for ACCESS, STAT, LOOKUP.

Below are brick logs output snippets:
=====================================
[2016-09-21 05:05:41.320200] I [MSGID: 115055] [server-rpc-fops.c:423:server_access_cbk] 0-distrep-server: 1421998: ACCESS (null) (2bb72069-3217-4c38-b660-abc90b40f118) ==> (Stale file handle) [Stale file handle]
rpc-fops.c:1748:server_stat_cbk] 0-distrep-server: 1405862: STAT (null) (5c449912-1199-48c1-8fbc-996ec6a2962a) ==> (Stale file handle) [Stale file handle]
[2016-09-21 05:03:34.570429] D [MSGID: 115050] [server-rpc-fops.c:179:server_lookup_cbk] 0-distrep-server: 1131709: LOOKUP /linux-4.6.4/Documentation/RCU/rculist_nulls.txt (fa65c27b-d7db-41ab-8459-6b7c827dd78e/rculist_nulls.txt) ==> (No such file or directory) [No such file or directory]

Moving this bug to ASSIGNED.

Comment 11 Amar Tumballi 2018-04-16 03:05:59 UTC
This is expected to be null in some calls as there can be anon-fd (ie, open on gfid) type of operation. Closing as WONTFIX, feel free to open this if found required.


Note You need to log in before you can comment on or make changes to this bug.