| Summary: | Log message enhancements in GlusterFS - phase 1 | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Amar Tumballi <amarts> |
| Component: | logging | Assignee: | Amar Tumballi <amarts> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | low | Docs Contact: | |
| Priority: | urgent | ||
| Version: | mainline | CC: | gluster-bugs, vikas, vraman |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | RTNR | Mount Type: | All |
| Documentation: | DP | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Bug Depends On: | 763994, 764064 | ||
| Bug Blocks: | |||
| Deadline: | 2011-03-01 | ||
|
Description
Amar Tumballi
2011-02-01 07:21:53 UTC
PATCH: http://patches.gluster.com/patch/6272 in master (dht: log filename on failed reavalidate) PATCH: http://patches.gluster.com/patch/6275 in master (libglusterfs/xlator.c: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6276 in master (cluster/afr: Whitespace cleanup) PATCH: http://patches.gluster.com/patch/6277 in master (cluster/afr: Re-factoring code for alignment) PATCH: http://patches.gluster.com/patch/6278 in master (libglusterfs/dict.c: Whitespace cleanup) PATCH: http://patches.gluster.com/patch/6279 in master (performance/io-cache: Whitespace cleanup) PATCH: http://patches.gluster.com/patch/6350 in release-3.1 (libglusterfs: add 'graph_id' in logging) PATCH: http://patches.gluster.com/patch/6458 in master (libglusterfs: logging/mem-pool section white-space cleanup) PATCH: http://patches.gluster.com/patch/6459 in master (libglusterfs: gf_log_nomem() and other minor updates) PATCH: http://patches.gluster.com/patch/6460 in master (libglusterfs: all header files updated) PATCH: http://patches.gluster.com/patch/6462 in master (storage/posix: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6463 in master (libglusterfs/src/*.c: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6464 in master (storage/posix: log enhancements) PATCH: http://patches.gluster.com/patch/6465 in master (glusterfsd: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6466 in master (glusterfsd: log enhancement) PATCH: http://patches.gluster.com/patch/6467 in master (rpc/rpc-lib : log enhancements) PATCH: http://patches.gluster.com/patch/6468 in master (rpc-transport: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6469 in master (rpc-transport: log enhancement) PATCH: http://patches.gluster.com/patch/6470 in master (performance/io-cache: whitespace related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6471 in master (performance/io-cache: logging enhancements) PATCH: http://patches.gluster.com/patch/6472 in master (performance/quick-read: whitespace related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6473 in master (performance/quick-read: propagate errors during open back to actual fops.) PATCH: http://patches.gluster.com/patch/6474 in master (performance/quick-read: logging enhancements) PATCH: http://patches.gluster.com/patch/6475 in master (performance/read-ahead: white space related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6476 in master (performance/read-ahead: logging enhancements.) PATCH: http://patches.gluster.com/patch/6477 in master (performance/write-behind: whitespace cleanup.) PATCH: http://patches.gluster.com/patch/6478 in master (performance/write-behind: logging enhancements) PATCH: http://patches.gluster.com/patch/6479 in master (performance/stat-prefetch: whitespace related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6480 in master (performance/stat-prefetch: logging enhancements) PATCH: http://patches.gluster.com/patch/6481 in master (features/access-control: White space related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6482 in master (features/access-control: logging enhancement) PATCH: http://patches.gluster.com/patch/6483 in master (features/locks: White space related changes to adhere to coding guidelines.) PATCH: http://patches.gluster.com/patch/6484 in master (features/locks: logging enhancement) PATCH: http://patches.gluster.com/patch/6485 in master (cluster/dht: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6486 in master (cluster/dht: log enhancements) PATCH: http://patches.gluster.com/patch/6487 in master (protocol/auth directory updated with whitespace cleanup and proper logs) PATCH: http://patches.gluster.com/patch/6488 in master (protocol/server: White space cleanup and NULL check validations.) PATCH: http://patches.gluster.com/patch/6489 in master (protocol/server: log enhancements) PATCH: http://patches.gluster.com/patch/6490 in master (protocol/client: log enhancement) PATCH: http://patches.gluster.com/patch/6461 in master (libglusterfs/src/*.c: log enhancement) PATCH: http://patches.gluster.com/patch/6508 in master (libglusterfs: added graph id in logging) PATCH: http://patches.gluster.com/patch/6509 in master (debug/io-stats: don't log the dangling volume message) PATCH: http://patches.gluster.com/patch/6553 in master (free priv structure if something fails in init for locks) Round-1 of commits went in. There are 3 major modules pending (cluster/afr, nfs, glusterd/cli), of which I will try to finish afr before the 3.2.0 code-freeze. Moving this out of the 3.2.0 target. PATCH: http://patches.gluster.com/patch/6558 in master (do not check loc and fd since they can be passed as NULL only) PATCH: http://patches.gluster.com/patch/6575 in master (cluster/afr: whitespace cleanup) PATCH: http://patches.gluster.com/patch/6627 in master (cluster/afr: white-space cleanup - part 2) PATCH: http://patches.gluster.com/patch/6626 in master (cluster/afr: log enhancement - part 2) PATCH: http://patches.gluster.com/patch/6628 in master (cluster/afr: log enhancement - part 3) 011-04-09 08:48:31.547443] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.547507] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.829272] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.848020] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.897977] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.898221] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.910918] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.976308] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.976426] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:31.988662] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.1679] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.8139] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.15971] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.48038] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.59444] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.74549] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.79570] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.92815] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.106067] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.106172] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.199387] W [iobuf.c:669:iobref_size] 0-: iobref not found [2011-04-09 08:48:32.204672] W [iobuf.c:669:iobref_size] 0-: iobref not found These logs have no meaning and we are seeing them in client log more than 10 entries per second. Need to make it 'gf_log_callingfn()'. http://patches.gluster.com/patch/6680/ does the same. Need to get it in. > [2011-04-09 08:48:32.199387] W [iobuf.c:669:iobref_size] 0-: iobref not found
> [2011-04-09 08:48:32.204672] W [iobuf.c:669:iobref_size] 0-: iobref not found
>
> These logs have no meaning and we are seeing them in client log more than 10
> entries per second. Need to make it 'gf_log_callingfn()'.
>
> http://patches.gluster.com/patch/6680/ does the same. Need to get it in.
[2011-04-09 09:39:23.217592] E [iobuf.c:718:iobref_size] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_fault_cbk+0x6d9) [0x7ff51bc44fa9] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(__ioc_inode_flush+0x2b) [0x7ff51bc3f99b] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_page_destroy+0x2b) [0x7ff51bc4348b]))) 0-iobuf: invalid argument: iobref
[2011-04-09 09:39:23.237073] E [iobuf.c:718:iobref_size] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_fault_cbk+0x6d9) [0x7ff51bc44fa9] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(__ioc_inode_flush+0x2b) [0x7ff51bc3f99b] (-->/usr/local/lib/glusterfs/3.1.0git/xlator/performance/io-cache.so(ioc_page_destroy+0x2b) [0x7ff51bc4348b]))) 0-iobuf: invalid argument: iobref
With the patch, it seems to be called from ioc_page_destroy().
PATCH: http://patches.gluster.com/patch/6766 in master (protocol/server: prevent unnessary logs in INFO level) PATCH: http://patches.gluster.com/patch/6782 in master (libglusterfs/src: bring in VALIDATE_OR_GOTO to args) PATCH: http://patches.gluster.com/patch/6812 in master (protocol/client: convert to errno before checking for EAGAIN) PATCH: http://patches.gluster.com/patch/6834 in master (remove excessive logs due to log enhancement) PATCH: http://patches.gluster.com/patch/6801 in master (mgmt/glusterd: Remove excessive INFO logs) PATCH: http://patches.gluster.com/patch/6992 in master (glusterd: log the error immedietly after the syscall failure) PATCH: http://patches.gluster.com/patch/7025 in master (prevent few excessive logs) PATCH: http://patches.gluster.com/patch/7108 in master (loc_t: add 'gfid' and 'pargfid' fields) PATCH: http://patches.gluster.com/patch/7142 in master (cluster/afr: set loc gfids for fresh lookup) PATCH: http://patches.gluster.com/patch/7453 in release-3.1 (loc_t: add 'gfid' and 'pargfid' fields) PATCH: http://patches.gluster.com/patch/7452 in release-3.2 (loc_t: add 'gfid' and 'pargfid' fields) PATCH: http://patches.gluster.com/patch/7435 in master (rdma: fix the excessive logs) PATCH: http://patches.gluster.com/patch/7522 in master (logging: fill proper domain names at places where it is missing) With 3.2.x releases, Phase-1 of log message improvement is complete. I will be opening another bug for Phase-2, which will include more enhancements. Will be targeting that for 3.3.0 release. CHANGE: http://review.gluster.com/49 (- If open fails and there are any other fops waiting on that open, those fops) merged in release-3.1 by Anand Avati (avati) *** Bug 3379 has been marked as a duplicate of this bug. *** CHANGE: http://review.gluster.com/372 (most of this resulted in taking a 'ref' on NULL iobuf.) merged in release-3.2 by Anand Avati (avati) CHANGE: http://review.gluster.com/417 (helps getting output of 'glusterfs --version' from the users while) merged in release-3.1 by Vijay Bellur (vijay) CHANGE: http://review.gluster.com/415 (helps getting output of 'glusterfs --version' from the users while) merged in master by Vijay Bellur (vijay) CHANGE: http://review.gluster.com/416 (helps getting output of 'glusterfs --version' from the users while) merged in release-3.2 by Vijay Bellur (vijay) CHANGE: http://review.gluster.com/562 (Change-Id: I5c2b170977f21c9a83d36f60d9a145c6cc221358) merged in master by Vijay Bellur (vijay) CHANGE: http://review.gluster.com/567 (Change-Id: I95271def60a1977777f0b5576ae5fbbb59352778) merged in release-3.2 by Vijay Bellur (vijay) CHANGE: http://review.gluster.com/621 (Change-Id: Ic4cd16618925656a12049f28f9b30bcac2242d4b) merged in release-3.2 by Vijay Bellur (vijay) |