Bug 1077406

Summary: Striped volume does not work with VMware esxi v4.1, 5.1 or 5.5
Product: [Community] GlusterFS Reporter: capriotti.carlos
Component: nfsAssignee: bugs <bugs>
Status: CLOSED INSUFFICIENT_DATA QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs, capriotti.carlos, gluster-bugs, ndevos, nlevinki, nsathyan, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-12-15 08:18:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
VMWare documentation making reference to the symptom described. none

Description capriotti.carlos 2014-03-17 22:36:29 UTC 
Description of problem:

Striped volume cannot be used (mounted as NFS) on esxi. Trying to browse the volume from esxi's file browser never returns any file. File/folder creating returns an error message.

Other volume types work well. Other NFS-based storage solutions work on the same configuration.


Version-Release number of selected component (if applicable):

Glusterfs-server verions 3.4.2.1 and 3.5.0beta4 working on CentOS 6.5, latest updates applied.


How reproducible:

Can be reproduced easily. 100% of the times. 

1.run ESXi server
2.create and start striped volume over two or more bricks, replicated or not
3.create storage instance on esxi, using the volume
4a) browse the volume from esxi and observe it stall on the host monitor (file browser)

OR

4b) Attempt creating a virtual machine on volume and receive error message referring to files not being accessible. Likewise, creating files/folders on volume returns an error.

5) observe over tcpdump a tight loop: "readdir" from the host followed by "ack" from the server.

Actual results:

Creating a VM on the volume will actually create its containing folder, and one or two files that are part of the VM, but esxi will return an access error referring to one of the files that were created and can be seen on the volume.


Expected results:

Regular creation of VMs, files, folders, and normal file browsing/management.

Additional info:

Test environment used: two Dell PE2950 servers for Gluster and one R710 for esxi.

ALSO used 3 VMs for esxi (one for each version tested), and two VMs for CentOS/Gluster.

If necessary I can make the esxi and/or CentOS VM images available for download.

Note: this is not related to bug 852582
Comment 2 capriotti.carlos 2014-04-19 04:39:12 UTC 
After further investigation, the following VMWare document

http://www.vmware.com/files/pdf/techpaper/VMware-NFS-BestPractices-WP-EN.pdf

under the Appendix 1, describes a symptom like this, being due to non-availability the "no_root_squash" option for the NFS share.
Comment 3 capriotti.carlos 2014-04-19 04:41:47 UTC 
Created attachment 887687 [details]
VMWare documentation making reference to the symptom described.

VMWare documentation making reference to the symptom described.
Comment 4 Niels de Vos 2014-08-28 17:36:31 UTC 
> 5) observe over tcpdump a tight loop: "readdir" from the host followed by
> "ack" from the server.

Bug 1130969 was used to resolve a readdir issue related to stripe volumes. Could you try if the most recent nightly build from 3.4 or 3.5 fixes it for you?

- http://download.gluster.org/pub/gluster/glusterfs/nightly/
Comment 5 capriotti.carlos 2014-09-17 18:12:26 UTC 
@Niels, hello.

I no longer had my test environment configured, so resuming tests required creating everything again.

Installing CentOS 6.5 from scratch, along with Gluster 3.5 (latest release), showed the same result on ESXI 5.1. ESXI 5.5 returns an error upon (attempt of) VM creation, immediately.

The important question here, before moving on to additional troubleshooting:

Were the fixes from the nightly release from 28/Aug been committed to the latest (two days ago) release of Gluster 3.5 ? If not, I'll have to install "nightly" instead, and restart testing.
Comment 6 Niels de Vos 2014-09-17 20:00:48 UTC 
Thanks for the efforts on testing, that is much appreciated!

(In reply to capriotti.carlos from comment #5)
> Were the fixes from the nightly release from 28/Aug been committed to the
> latest (two days ago) release of Gluster 3.5 ? If not, I'll have to install
> "nightly" instead, and restart testing.

The most important change has been committed to the source repository, but there has not been a minor 3.5 release that contains it. It would be great if you can test this nightly build:
- http://download.gluster.org/pub/gluster/glusterfs/nightly/glusterfs-3.5/epel-6-x86_64/glusterfs-3.5.20140909.6ad6661-1.autobuild/
Comment 7 Niels de Vos 2014-12-15 08:18:25 UTC 
No answer on comment #6 since about 2 months. Closing this one now, please re-open if the issue is not fixed with a recent 3.5 version.
Comment 8 Niels de Vos 2015-05-26 12:33:50 UTC 
This bug has been CLOSED, and there has not been a response to the requested NEEDINFO in more than 4 weeks. The NEEDINFO flag is now getting cleared so that our Bugzilla household is getting more in order.