Hide Forgot
This bug is an enhancement bug. Tasks involved are 1. Cover whole code path and add the logs in all the missing places 2. Change the log-level of logs to appropriate levels 3. Utilize 'gf_log_callingfn()' (to get the backtrace) wherever necessary. 4. Make log messages more informative (to both users and developers) 5. By default all the critical logs should goto 'syslog'
PATCH: http://patches.gluster.com/patch/6272 in master (dht: log filename on failed reavalidate)
PATCH: http://patches.gluster.com/patch/6275 in master (libglusterfs/xlator.c: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6276 in master (cluster/afr: Whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6277 in master (cluster/afr: Re-factoring code for alignment)
PATCH: http://patches.gluster.com/patch/6278 in master (libglusterfs/dict.c: Whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6279 in master (performance/io-cache: Whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6350 in release-3.1 (libglusterfs: add 'graph_id' in logging)
PATCH: http://patches.gluster.com/patch/6458 in master (libglusterfs: logging/mem-pool section white-space cleanup)
PATCH: http://patches.gluster.com/patch/6459 in master (libglusterfs: gf_log_nomem() and other minor updates)
PATCH: http://patches.gluster.com/patch/6460 in master (libglusterfs: all header files updated)
PATCH: http://patches.gluster.com/patch/6462 in master (storage/posix: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6463 in master (libglusterfs/src/*.c: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6464 in master (storage/posix: log enhancements)
PATCH: http://patches.gluster.com/patch/6465 in master (glusterfsd: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6466 in master (glusterfsd: log enhancement)
PATCH: http://patches.gluster.com/patch/6467 in master (rpc/rpc-lib : log enhancements)
PATCH: http://patches.gluster.com/patch/6468 in master (rpc-transport: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6469 in master (rpc-transport: log enhancement)
PATCH: http://patches.gluster.com/patch/6470 in master (performance/io-cache: whitespace related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6471 in master (performance/io-cache: logging enhancements)
PATCH: http://patches.gluster.com/patch/6472 in master (performance/quick-read: whitespace related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6473 in master (performance/quick-read: propagate errors during open back to actual fops.)
PATCH: http://patches.gluster.com/patch/6474 in master (performance/quick-read: logging enhancements)
PATCH: http://patches.gluster.com/patch/6475 in master (performance/read-ahead: white space related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6476 in master (performance/read-ahead: logging enhancements.)
PATCH: http://patches.gluster.com/patch/6477 in master (performance/write-behind: whitespace cleanup.)
PATCH: http://patches.gluster.com/patch/6478 in master (performance/write-behind: logging enhancements)
PATCH: http://patches.gluster.com/patch/6479 in master (performance/stat-prefetch: whitespace related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6480 in master (performance/stat-prefetch: logging enhancements)
PATCH: http://patches.gluster.com/patch/6481 in master (features/access-control: White space related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6482 in master (features/access-control: logging enhancement)
PATCH: http://patches.gluster.com/patch/6483 in master (features/locks: White space related changes to adhere to coding guidelines.)
PATCH: http://patches.gluster.com/patch/6484 in master (features/locks: logging enhancement)
PATCH: http://patches.gluster.com/patch/6485 in master (cluster/dht: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6486 in master (cluster/dht: log enhancements)
PATCH: http://patches.gluster.com/patch/6487 in master (protocol/auth directory updated with whitespace cleanup and proper logs)
PATCH: http://patches.gluster.com/patch/6488 in master (protocol/server: White space cleanup and NULL check validations.)
PATCH: http://patches.gluster.com/patch/6489 in master (protocol/server: log enhancements)
PATCH: http://patches.gluster.com/patch/6490 in master (protocol/client: log enhancement)
PATCH: http://patches.gluster.com/patch/6461 in master (libglusterfs/src/*.c: log enhancement)
PATCH: http://patches.gluster.com/patch/6508 in master (libglusterfs: added graph id in logging)
PATCH: http://patches.gluster.com/patch/6509 in master (debug/io-stats: don't log the dangling volume message)
PATCH: http://patches.gluster.com/patch/6553 in master (free priv structure if something fails in init for locks)
Round-1 of commits went in. There are 3 major modules pending (cluster/afr, nfs, glusterd/cli), of which I will try to finish afr before the 3.2.0 code-freeze. Moving this out of the 3.2.0 target.
PATCH: http://patches.gluster.com/patch/6558 in master (do not check loc and fd since they can be passed as NULL only)
PATCH: http://patches.gluster.com/patch/6575 in master (cluster/afr: whitespace cleanup)
PATCH: http://patches.gluster.com/patch/6627 in master (cluster/afr: white-space cleanup - part 2)
PATCH: http://patches.gluster.com/patch/6626 in master (cluster/afr: log enhancement - part 2)
PATCH: http://patches.gluster.com/patch/6628 in master (cluster/afr: log enhancement - part 3)
011-04-09 08:48:31.547443] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.547507] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.829272] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.848020] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.897977] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.898221] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.910918] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.976308] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.976426] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.988662] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.1679] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.8139] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.15971] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.48038] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.59444] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.74549] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.79570] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.92815] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.106067] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.106172] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.199387] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.204672] W [iobuf.c:669:iobref_size] 0-: iobref not found These logs have no meaning and we are seeing them in client log more than 10 entries per second. Need to make it 'gf_log_callingfn()'. http://patches.gluster.com/patch/6680/ does the same. Need to get it in.
> [2011-04-09 08:48:32.199387] W [iobuf.c:669:iobref_size] 0-: iobref not found > [2011-04-09 08:48:32.204672] W [iobuf.c:669:iobref_size] 0-: iobref not found > > These logs have no meaning and we are seeing them in client log more than 10 > entries per second. Need to make it 'gf_log_callingfn()'. > > http://patches.gluster.com/patch/6680/ does the same. Need to get it in. [2011-04-09 09:39:23.217592] E [iobuf.c:718:iobref_size] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_fault_cbk+0x6d9) [0x7ff51bc44fa9] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(__ioc_inode_flush+0x2b) [0x7ff51bc3f99b] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_page_destroy+0x2b) [0x7ff51bc4348b]))) 0-iobuf: invalid argument: iobref [2011-04-09 09:39:23.237073] E [iobuf.c:718:iobref_size] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_fault_cbk+0x6d9) [0x7ff51bc44fa9] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(__ioc_inode_flush+0x2b) [0x7ff51bc3f99b] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_page_destroy+0x2b) [0x7ff51bc4348b]))) 0-iobuf: invalid argument: iobref With the patch, it seems to be called from ioc_page_destroy().
PATCH: http://patches.gluster.com/patch/6766 in master (protocol/server: prevent unnessary logs in INFO level)
PATCH: http://patches.gluster.com/patch/6782 in master (libglusterfs/src: bring in VALIDATE_OR_GOTO to args)
PATCH: http://patches.gluster.com/patch/6812 in master (protocol/client: convert to errno before checking for EAGAIN)
PATCH: http://patches.gluster.com/patch/6834 in master (remove excessive logs due to log enhancement)
PATCH: http://patches.gluster.com/patch/6801 in master (mgmt/glusterd: Remove excessive INFO logs)
PATCH: http://patches.gluster.com/patch/6992 in master (glusterd: log the error immedietly after the syscall failure)
PATCH: http://patches.gluster.com/patch/7025 in master (prevent few excessive logs)
PATCH: http://patches.gluster.com/patch/7108 in master (loc_t: add 'gfid' and 'pargfid' fields)
PATCH: http://patches.gluster.com/patch/7142 in master (cluster/afr: set loc gfids for fresh lookup)
PATCH: http://patches.gluster.com/patch/7453 in release-3.1 (loc_t: add 'gfid' and 'pargfid' fields)
PATCH: http://patches.gluster.com/patch/7452 in release-3.2 (loc_t: add 'gfid' and 'pargfid' fields)
PATCH: http://patches.gluster.com/patch/7435 in master (rdma: fix the excessive logs)
PATCH: http://patches.gluster.com/patch/7522 in master (logging: fill proper domain names at places where it is missing)
With 3.2.x releases, Phase-1 of log message improvement is complete. I will be opening another bug for Phase-2, which will include more enhancements. Will be targeting that for 3.3.0 release.
CHANGE: http://review.gluster.com/49 (- If open fails and there are any other fops waiting on that open, those fops) merged in release-3.1 by Anand Avati (avati)
*** Bug 3379 has been marked as a duplicate of this bug. ***
CHANGE: http://review.gluster.com/372 (most of this resulted in taking a 'ref' on NULL iobuf.) merged in release-3.2 by Anand Avati (avati)
CHANGE: http://review.gluster.com/417 (helps getting output of 'glusterfs --version' from the users while) merged in release-3.1 by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/415 (helps getting output of 'glusterfs --version' from the users while) merged in master by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/416 (helps getting output of 'glusterfs --version' from the users while) merged in release-3.2 by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/562 (Change-Id: I5c2b170977f21c9a83d36f60d9a145c6cc221358) merged in master by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/567 (Change-Id: I95271def60a1977777f0b5576ae5fbbb59352778) merged in release-3.2 by Vijay Bellur (vijay)
CHANGE: http://review.gluster.com/621 (Change-Id: Ic4cd16618925656a12049f28f9b30bcac2242d4b) merged in release-3.2 by Vijay Bellur (vijay)