Bug 762253 (GLUSTER-521)

Summary: SPECFS validation fails over distribute + replicate
Product: [Community] GlusterFS Reporter: Harshavardhana <fharshav>
Component: replicateAssignee: Vikas Gorur <vikas>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: mainlineCC: amarts, avati, cww, gluster-bugs, pavan, rabhat, shehjart
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
SPECFS 2008 - Run Results over Replicate share
none
SPECFS 2008 - Run Results over Distribute share none

Description Harshavardhana 2010-01-05 21:37:41 UTC
Created attachment 130 [details]
Modified wrapper for testing only

Comment 1 Harshavardhana 2010-01-06 00:37:08 UTC
Setup is CIFS reexport from Gluster Storage Platform 

fully loaded configuration using "distribute + replicate" 

Samba server version 3.4.2 

GlusterFS version 3.0.0

cifs module from client under use with kernel version 2.6.18-164.6.1 

Please find the attachment with the error during runs. 

No Logs found on backend glusterfs server/client and no logs reported 
in samba logs too. 

SPECFS command used for validation is  

"cd /opt/benchmarks/SFS2008/spec-sfs2008/manager"
"java SfsManager -r sfs_config_cifs_replicate -s avail_media_test -v 2" 

sfs_config_cifs_replicate contains configuration for replicate validation run. 

NOTE: Validation test passes for pure distribute without replicate fully loaded
configuration. Working distribute share use the below command. 

"cd /opt/benchmarks/SFS2008/spec-sfs2008/manager"
"java SfsManager -r sfs_config_cifs_distribute -s avail_media_test -v 2"

Comment 2 Shehjar Tikoo 2010-01-06 00:50:41 UTC
What is the CIFS server here? Samba over booster or samba over FUSE?

We've never tested the CIFS part so first you should run SFS with just storage/posix as the volume.

Comment 3 Harshavardhana 2010-01-06 00:59:04 UTC
CIFS over booster never go off the initial testing so its not working at all. CIFS
is native glusterfs reexported. 

Apart from that we have a customer requirement/deployment at stake so we have to 
do it with fully loaded configuration and its part of the acceptance test. So if 
this is never tested i am going to tick off this as not working for now.

NOTE: We have CIFS reexport running at Partners since approx 2yrs. so i don't 
think this is something new for us.

Comment 4 Shehjar Tikoo 2010-01-06 03:58:19 UTC
(In reply to comment #3)
> CIFS over booster never go off the initial testing so its not working at all.
> CIFS
> is native glusterfs reexported. 
> 

OK.

> Apart from that we have a customer requirement/deployment at stake so we have
> to 
> do it with fully loaded configuration and its part of the acceptance test. So
> if 
> this is never tested i am going to tick off this as not working for now.
> 

What the deployment needs is different from how we need to test this. We need to test by starting small. I can tell by experience of compliance testing on NFS that if certain things fail with posix-only, they'll certainly fail with other configurations. Plus, if the tests fail in posix, you know where to look while in a fully loaded config it'll be hard to tell where to find the problem.

Say the test succeeds on posix and then you start adding protocol/client and protocol/server, and the test fails at this point, you'll know these two are to blame and you can diagnose accordingly. This is again from experience of posix compliance testing on NFS xlator where even if posix passes the tests, it does not guarantee that it'll pass on replicate and distribute.

> NOTE: We have CIFS reexport running at Partners since approx 2yrs. so i don't 
> think this is something new for us.
Thats fine too. Even if the server is working it does not mean it is working exactly in compliance. Most of the time users wont notice or care about compliance since their tools may not be sensitive to compliance related problems. This is not so for SFS-like tools which test specifically for compliance.

Comment 5 Amar Tumballi 2010-01-06 04:18:03 UTC
(In reply to comment #4)
> 
> > NOTE: We have CIFS reexport running at Partners since approx 2yrs. so i don't 
> > think this is something new for us.
> Thats fine too. Even if the server is working it does not mean it is working
> exactly in compliance. Most of the time users wont notice or care about
> compliance since their tools may not be sensitive to compliance related
> problems. This is not so for SFS-like tools which test specifically for
> compliance.

Shehjar, below quoted were the paragraph from bug report.

| NOTE: Validation test passes for pure distribute without replicate fully 
| loaded
| configuration. Working distribute share use the below command. 
| 
| "cd /opt/benchmarks/SFS2008/spec-sfs2008/manager"
| "java SfsManager -r sfs_config_cifs_distribute -s avail_media_test -v 2"

Harsha did start small, and in his tests, he achieved successful tests upto distribute only tests. Hence the bug report on replicate + distribute scenario.

-Amar

Comment 6 Shehjar Tikoo 2010-01-06 04:23:04 UTC
My big bad! bad! bad!

Comment 7 Harshavardhana 2010-01-06 04:24:36 UTC
I don't agree. Test runs are running fine with "Distribute" which was stated in this bug already (even with fully loaded config). So issues with the other translators are ruled out already. Its just the replicate+distribute which is making the test runs fail. 

Idea of Nit picking each translators is perfect with coding perspective and a developer point of view which i am not really concerned at. Requirement is just a working setup with testcases completed successfully.

Comment 8 Anand Avati 2010-01-21 11:41:02 UTC
PATCH: http://patches.gluster.com/patch/2670 in master (cluster/afr: Send the struct flock returned by the server to the user.)

Comment 9 Anand Avati 2010-01-24 15:59:18 UTC
PATCH: http://patches.gluster.com/patch/2689 in release-2.0 (cluster/afr: Send flock returned by locks xlator to user.)

Comment 10 Anand Avati 2010-04-27 10:32:06 UTC
PATCH: http://patches.gluster.com/patch/3171 in release-2.0 (cluster/afr: Send the struct flock returned by the server to the user only in the UNWIND path.)

Comment 11 Amar Tumballi 2010-05-04 08:00:25 UTC
*** Bug 863 has been marked as a duplicate of this bug. ***