Bug 782286

Summary: [glusterfs-3.2.6qa1]: possible deadlock in io-cache reconfigure
Product: [Community] GlusterFS Reporter: Raghavendra Bhat <rabhat>
Component: io-cacheAssignee: Raghavendra Bhat <rabhat>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: urgent    
Version: pre-releaseCC: gluster-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 815040 (view as bug list) Environment:
Last Closed: 2013-07-24 17:44:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 811632, 815040, 817967    

Description Raghavendra Bhat 2012-01-17 04:50:02 UTC
Description of problem:

There is a possibility of deadlock in reconfigure of io-cache xlator.

In reconfigure function we do this.

int
reconfigure (xlator_t *this, dict_t *options)
{
	ioc_table_t *table             = NULL;
	int32_t      cache_timeout     = 0;
	int64_t      min_file_size     = 0;
	int64_t      max_file_size     = 0;
	char        *tmp               = NULL;
	uint64_t     cache_size        = 0;
        char        *cache_size_string = NULL;
	int          ret               = 0;

        if (!this || !this->private)
                goto out;

        table = this->private;

	ioc_table_lock (table);
        {
                if (dict_get (options, "cache-timeout")) {
			cache_timeout =
		                data_to_uint32 (dict_get (options,
                	                                  "cache-timeout"));
		        if (cache_timeout < 0){

i.e. we take the lock in ioc_table_t, then we go on looking for any io-cache options i the dict. Suppose some options are given wrong or dict_get from the dictionary fails, then we do this (i.e. goto out)

if (cache_timeout < 0){
                                gf_log (this->name, GF_LOG_WARNING,
                                        "cache-timeout %d seconds invalid,"
                                        " has to be  >=0", cache_timeout);
                                goto out;
                        }


But in the "out" we directly return instead of unlocking the table, thus leading to the deadlock and application hang.

	ioc_table_unlock (table);
out:
        return ret;

}

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Anand Avati 2012-01-20 06:30:00 UTC
CHANGE: http://review.gluster.com/2649 (performance/io-cache: if the reconfigure option given is wrong, then unlock and return) merged in release-3.2 by Vijay Bellur (vijay)

Comment 2 Raghavendra Bhat 2012-02-20 05:56:42 UTC
Tested with glusterfs-3.2.6qa3. Now there is no deadlock in io-cache reconfigure since for every case we are unlocking before returning.