| Summary: | Writing "pure path resolution ..." log message when creating linkfile influence on performance | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | hz02ruc |
| Component: | distribute | Assignee: | shishir gowda <sgowda> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.1.3 | CC: | amarts, gluster-bugs, lakshmipathi, nsathyan |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | RTNR | Mount Type: | fuse |
| Documentation: | --- | CRM: | |
| Verified Versions: | 3.2.3 | Category: | --- |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Amar Tumballi
2011-05-24 05:04:21 UTC
BACKGROUND:
We are using GlusterFS 3.1.3 as our storage system. Recently
A brick was removed from our system which stores about 50TB data,
then the IO performance of the storage nodes declined very much.
I noticed that the log files of storage nodes grows fast, i have to
truncate them until they reached above 1GB. The log files are full of
warning messages "[server-resolve.c:server_resolve]pure path resolution for <path>".
It seems that only linkfiles can cause this log message, and i did a
test to reproduce the problem.
Reproduce:
4 server process, each export a directory
server : ~/share
server2: ~/share2
server3: ~/share3
server4: ~/share4
client: configured with dht
mountpoint: ~/mnt
1. create 4 files.
huz@furutuki:~/dht$ touch ~/mnt/tags-1 ~/mnt/tags-2 ~/mnt/tags-6 ~/mnt/tags-11
huz@furutuki:~/dht$ ll ~/share
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1
huz@furutuki:~/dht$ ll ~/share2
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2
huz@furutuki:~/dht$ ll ~/share3
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6
huz@furutuki:~/dht$ ll ~/share4
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-11
huz@furutuki:~/dht$ ll ~/mnt
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-11
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6
2. remove brick4 and reconfigure
huz@furutuki:~/dht$ ps -ef | grep glusterfs
root 13762 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server.vol -l /home/huz/dht/server.log
root 13767 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server2.vol -l /home/huz/dht/server2.log
root 13772 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server3.vol -l /home/huz/dht/server3.log
root 13777 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server4.vol -l /home/huz/dht/server4.log
root 13787 1 0 14:36 ? 00:00:00 glusterfs -f /home/huz/dht/client.vol -l /home/huz/dht/client.log/home/huz/mnt
huz@furutuki:~/dht$ vi client.vol
huz@furutuki:~/dht$ sudo kill -s TERM 13777
huz@furutuki:~/dht$ sudo kill -s HUP 13787
3. ls ~/mnt (lookup) and check that linkfiles are created(brick2, brick3)
huz@furutuki:~/dht$ ll ~/mnt
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6
huz@furutuki:~/dht$ ll ~/share
total 0
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1
huz@furutuki:~/dht$ ll ~/share2
total 0
---------T 1 huz huz 0 2011-05-24 14:38 tags-1
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2
huz@furutuki:~/dht$ ll ~/share3
total 0
---------T 1 huz huz 0 2011-05-24 14:38 tags-2
-rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6
4. "pure path resolution for <path> (SETXATTR)" are wrote in the log file: server2.log server3.log
-------------------------------------
It seems in function server_resolve(), gfid & paragfid == null can lead such a message.
http://patches.gluster.com/patch/4504/ set gfid in *fuse layer* for new entries, but
in my test, after removing brick 4 and reconfigure, MKNOD will be called in *dht layer*
in dht_linkfile_create() to create linkfile and call SETXATTR in its callback function,
but no gfid is set before, so in server path resolution function, the log message outputs.
By the way,I searched the web, this log message was added in 3.1.X and the loglevel
was changed to "INFO" in GlusterFS 3.2.0. I really want to know why the dev team maded
these changes.
In order to avoid the problem, i need to change the loglevel to "TRACE", "DEBUG" or
delete this log message for the present.
it's appreciated if you can give me any advise and tell me whether my understanding
is correct or not.
Thanks in advance.
tested with 3.2.3. tried to create few files with 'T' - log file (with default log level)doesn't show message like pure path resoultion. |