Hide Forgot
patch http://patches.gluster.com/patch/7108/ fixes this. will be available in 3.2.1 release (its already committed to git master branch).
BACKGROUND: We are using GlusterFS 3.1.3 as our storage system. Recently A brick was removed from our system which stores about 50TB data, then the IO performance of the storage nodes declined very much. I noticed that the log files of storage nodes grows fast, i have to truncate them until they reached above 1GB. The log files are full of warning messages "[server-resolve.c:server_resolve]pure path resolution for <path>". It seems that only linkfiles can cause this log message, and i did a test to reproduce the problem. Reproduce: 4 server process, each export a directory server : ~/share server2: ~/share2 server3: ~/share3 server4: ~/share4 client: configured with dht mountpoint: ~/mnt 1. create 4 files. huz@furutuki:~/dht$ touch ~/mnt/tags-1 ~/mnt/tags-2 ~/mnt/tags-6 ~/mnt/tags-11 huz@furutuki:~/dht$ ll ~/share total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1 huz@furutuki:~/dht$ ll ~/share2 total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2 huz@furutuki:~/dht$ ll ~/share3 total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6 huz@furutuki:~/dht$ ll ~/share4 total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-11 huz@furutuki:~/dht$ ll ~/mnt total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-11 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6 2. remove brick4 and reconfigure huz@furutuki:~/dht$ ps -ef | grep glusterfs root 13762 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server.vol -l /home/huz/dht/server.log root 13767 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server2.vol -l /home/huz/dht/server2.log root 13772 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server3.vol -l /home/huz/dht/server3.log root 13777 1 0 14:36 ? 00:00:00 glusterfsd -f /home/huz/dht/server4.vol -l /home/huz/dht/server4.log root 13787 1 0 14:36 ? 00:00:00 glusterfs -f /home/huz/dht/client.vol -l /home/huz/dht/client.log/home/huz/mnt huz@furutuki:~/dht$ vi client.vol huz@furutuki:~/dht$ sudo kill -s TERM 13777 huz@furutuki:~/dht$ sudo kill -s HUP 13787 3. ls ~/mnt (lookup) and check that linkfiles are created(brick2, brick3) huz@furutuki:~/dht$ ll ~/mnt total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6 huz@furutuki:~/dht$ ll ~/share total 0 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-1 huz@furutuki:~/dht$ ll ~/share2 total 0 ---------T 1 huz huz 0 2011-05-24 14:38 tags-1 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-2 huz@furutuki:~/dht$ ll ~/share3 total 0 ---------T 1 huz huz 0 2011-05-24 14:38 tags-2 -rw-r--r-- 1 huz huz 0 2011-05-24 14:37 tags-6 4. "pure path resolution for <path> (SETXATTR)" are wrote in the log file: server2.log server3.log ------------------------------------- It seems in function server_resolve(), gfid & paragfid == null can lead such a message. http://patches.gluster.com/patch/4504/ set gfid in *fuse layer* for new entries, but in my test, after removing brick 4 and reconfigure, MKNOD will be called in *dht layer* in dht_linkfile_create() to create linkfile and call SETXATTR in its callback function, but no gfid is set before, so in server path resolution function, the log message outputs. By the way,I searched the web, this log message was added in 3.1.X and the loglevel was changed to "INFO" in GlusterFS 3.2.0. I really want to know why the dev team maded these changes. In order to avoid the problem, i need to change the loglevel to "TRACE", "DEBUG" or delete this log message for the present. it's appreciated if you can give me any advise and tell me whether my understanding is correct or not. Thanks in advance.
tested with 3.2.3. tried to create few files with 'T' - log file (with default log level)doesn't show message like pure path resoultion.