Description of problem: a stripe volume mounted and I/O happening, client log shows "dict is NULL" messages. Version-Release number of selected component (if applicable): Mainline How reproducible: Steps to Reproduce: 1. create a stripe volume. 2. mount the volume and run the sanity tests 3. check the client log Actual results: I/O is happening but log file is full of "dict id NULL" messages. Expected results: Additional info: ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.121316] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.122612] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.123945] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.125234] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.126613] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.127926] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.129232] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.130550] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.139098] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.140452] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.141835] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.154886] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.156412] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.157866] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.159306] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.160760] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.161706] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-1: remote operation failed: No such file or directory [2012-03-27 02:46:54.161737] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-2: remote operation failed: No such file or directory [2012-03-27 02:46:54.162275] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.163211] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-1: remote operation failed: No such file or directory [2012-03-27 02:46:54.163241] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-2: remote operation failed: No such file or directory [2012-03-27 02:46:54.163749] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.165162] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3gift/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.166135] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-1: remote operation failed: No such file or directory [2012-03-27 02:46:54.166169] W [client3_1-fops.c:593:client3_1_unlink_cbk] 0-stripe-nfs-client-2: remote operation failed: No such file or directory [2012-03-27 02:46:54.167956] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.169242] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL [2012-03-27 02:46:54.170605] W [dict.c:458:dict_ref] (-->/usr/local/lib/glusterfs/3git/xlator/nfs/server.so(nfs_fop_unlink+0x459) [0x7f0b7f413f6d] (-->/usr/local/lib/glusterfs/3git/xlator/d ebug/io-stats.so(io_stats_unlink+0x296) [0x7f0b7f669caa] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/stripe.so(stripe_unlink+0x381) [0x7f0b7f881c88]))) 0-dict: dict is NULL
This happens both on fuse and nfs mounts
CHANGE: http://review.gluster.com/3024 (cluster/stripe- do not ref if dict is null) merged in master by Vijay Bellur (vijay)
verified on 3.3.0qa33