Description of problem:
Users can configure the no. of worker poller threads for a volume via the following volume options.

client.event-threads - for GlusterFS native mount, gluster-NFS server and other glusterfs daemons that load the protocol/client translator.

server.event-threads - for brick processes

Sub-problem 1:
--------------
The default for both of the above options is 2. In this release, we can observe that only one poller worker thread is waiting for network events by default.

Using the following bash function, we can verify the above claim. It doubles up as a means to verify the fix.

/* Returns a line containing "SyS_epoll_wait+0xb5/0xe0"
   for every epoll thread running in a process.
   Usage: nepoll <pid>
*/
function nepoll ()
{
    local pid=$1;
    for i in $(ls /proc/$pid/task);
    do
        cat /proc/$pid/task/$i/stack | grep epoll_wait;
    done
}


With this helper you can count the no. of epoll threads in a process
at any given time, by doing "# nepoll <pid> | wc -l".

Sample output:
[root@trantor codebase]# nepoll 3685
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0

Sub-problem 2:
--------------
After setting the client.event-threads to 5 (for e.g), volume-reset doesn't bring down the count back to 2 (viz. the default) and instead remains at 5.

Sample output:
[root@trantor codebase]# nepoll 3685
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0
[<ffffffff81208c25>] SyS_epoll_wait+0xb5/0xe0

Version-Release number of selected component (if applicable):
glusterfs-3.6.0.44-1

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: