Bug 1017305

Summary: packstack: no timeout configured for nfsmount.conf
Product: Red Hat OpenStack Reporter: Dafna Ron <dron>
Component: openstack-packstackAssignee: Ivan Chavero <ichavero>
Status: CLOSED NOTABUG QA Contact: Nir Magnezi <nmagnezi>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0CC: aortega, derekh, ewarszaw, hateya, slong, yeylon
Target Milestone: rcKeywords: Reopened, Triaged
Target Release: 4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: storage
Fixed In Version: Doc Type: Known Issue
Doc Text:
Network-based file systems are susceptible to timeout errors, usually related to the particular setups in which they are deployed (for example, network topology or switches). If a network file system is used as a backend for Block Storage (cinder), it must be properly tuned to its specific purpose according best practices recommendations.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-10 09:53:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
log none

Description Dafna Ron 2013-10-09 15:29:05 UTC
Created attachment 810000 [details]
log

Description of problem:

I installed gluster for cinder using packstack. 
my gluster servers went down and we have stale connection but no timeout (so they are not disconnecting) 

Version-Release number of selected component (if applicable):

openstack-packstack-2013.2.1-0.6.dev763.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. install openstack with packstack with cinder configured to work with gluster
2. run an instance from a volume 
3. shut down the gluster server

Actual results:

nfsmount has no time out

Expected results:

nfsmount should have time out configured by packstack 

Additional info:

Stderr: "mkdir: cannot create directory `/var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364': File exists\n"
2013-10-09 09:44:24.445 7297 WARNING cinder.volume.drivers.glusterfs [-] Exception during mounting Unexpected error while running command.
Command: mkdir -p /var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364
Exit code: 1
Stdout: ''

10.35.2.44:/Dafna_glance on /var/lib/glance/images type fuse.glusterfs (rw,default_permissions,default_permissions,allow_other,max_read=131072,allow_other,max_read=131072)
10.35.2.44:/Dafna_rhos on /var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
10.35.2.44:/Dafna_rhos on /var/lib/nova/mnt/4a31bc6e5fb9244971075aa23d364364 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)

ls: cannot access /var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364: Transport endpoint is not connected
total 0
d????????? ? ? ? ?            ? 4a31bc6e5fb9244971075aa23d364364

stale connection can cause issues in the future even if the storage is restored so we need to have a timeout.

Comment 1 Alvaro Lopez Ortega 2013-11-15 12:28:58 UTC
This will require some investigation.
@dron, do you have any pointer on how to solve this?

Comment 2 Dafna Ron 2013-11-15 12:47:38 UTC
I think we can use fork but I am adding Eduardo since I think he can offer some good advice on how to solve this issue.

Comment 3 Ivan Chavero 2013-12-04 08:08:50 UTC
currently packstack does not configure gluster, it just installs it as a dependency of glusterfs-fuse wich is installed by cinder if it is configured as a backend.

Comment 4 Dafna Ron 2013-12-09 13:05:15 UTC
This does not mean that packstack should not configured gluster or any other storage that it installed from the answer file.
If you think there is no capacity to do that please add future flag but if a user has to configure the system by themselves they don't need to use packstack. 
I'm re-opening this.

Comment 5 Alvaro Lopez Ortega 2013-12-10 09:53:28 UTC
We have agreed on adding a doc string and close this bug.