Bug 963176 - G4S: there is no automounting for gluster volumes for UFO thus all REST request fails,PUT with 404 ,GET/HEAD with 503
Summary: G4S: there is no automounting for gluster volumes for UFO thus all REST reque...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: GlusterFS
Classification: Community
Component: object-storage
Version: 3.3.1
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Thiago da Silva
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-05-15 10:33 UTC by pushpesh sharma
Modified: 2014-12-14 19:40 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-12-14 19:40:31 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)
dmesgs (549.17 KB, text/plain)
2013-05-15 10:39 UTC, pushpesh sharma
no flags Details

Description pushpesh sharma 2013-05-15 10:33:59 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.followed to create setup 
http://www.gluster.org/2012/09/howto-using-ufo-swift-a-quick-and-dirty-setup-guide/
 
and created gluster setup as follows on fresh F18:- 

#!/bin/bash

pvcreate /dev/vdb
vgcreate vg /dev/vdb
for ((i=1;i<10;i++)) ; do lvcreate --name lv$i --size 10G vg ;done
for ((i=1;i<10;i++)) ; do mkdir -p /mnt/lv$i ;done
yum install xfsprogs curl -y
for ((i=1;i<10;i++)) ; do mkfs.xfs -i size=1024 /dev/vg/lv$i;done
for ((i=1;i<10;i++)) ; do mount /dev/vg/lv$i /mnt/lv$i ;done
yum update -y
yum install glusterfs glusterfs-server glusterfs-fuse glusterfs-swift glusterfs-swift-account glusterfs-swift-container glusterfs-swift-object glusterfs-swift-proxy glusterfs-ufo curl gcc git-core memcached python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip python-mock git python-swiftclient -y

#gluster setup
systemctl start glusterd.service
IP=`ip addr show eth0 |grep 'inet ' | awk '{print $2}'| cut -d '/' -f1`
gluster volume create test $IP:/mnt/lv1 $IP:/mnt/lv2 $IP:/mnt/lv3 $IP:/mnt/lv4
gluster volume create test2 $IP:/mnt/lv5 $IP:/mnt/lv6 $IP:/mnt/lv7 $IP:/mnt/lv8

gluster volume start test
gluster volume start test2
cd /etc/swift
mv swift.conf-gluster swift.conf
mv fs.conf-gluster fs.conf
mv proxy-server.conf-gluster proxy-server.conf
mv account-server/1.conf-gluster account-server/1.conf
mv container-server/1.conf-gluster container-server/1.conf

openssl req -new -x509 -nodes -out cert.crt -keyout cert.key

#Configure UFO (edit /etc/swift/proxy-server.conf):
     + add your cert and key to the [DEFAULT] section:
       bind_port = 443
       cert_file = /etc/swift/cert.crt
        key_file = /etc/swift/cert.key
      + add one or more users of the gluster volume to the[filter:tempauth]           section:  
      user_$myvolname_$username=$password .admin
     + add the memcache address to the [filter:cache] section:
        memcache_servers = 127.0.0.1:11211


Generate builders:
cd /etc/swift ; /usr/bin/gluster-swift-gen-builders test ; /usr/bin/gluster-swift-gen-builders test2






[psharma@dhcp193-66 ~]$ curl -v -H 'X-Auth-User: test:tester' -H 'X-Auth-key: testing' -k https://10.65.207.238:443/auth/v1.0
* About to connect() to 10.65.207.238 port 443 (#0)
*   Trying 10.65.207.238... connected
* Connected to 10.65.207.238 (10.65.207.238) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 15 12:08:23 2013 GMT
* 	expire date: Jun 14 12:08:23 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> GET /auth/v1.0 HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-redhat-linux-gnu) libcurl/7.21.7 NSS/3.13.5.0 zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: 10.65.207.238
> Accept: */*
> X-Auth-User: test:tester
> X-Auth-key: testing
> 
< HTTP/1.1 200 OK
< X-Storage-Url: https://10.65.207.238:443/v1/AUTH_test
< X-Storage-Token: AUTH_tk0852185723824b3995872c3e57e7d98d
< X-Auth-Token: AUTH_tk0852185723824b3995872c3e57e7d98d
< Content-Length: 0
< Date: Wed, 15 May 2013 15:53:05 GMT
< 
* Connection #0 to host 10.65.207.238 left intact
* Closing connection #0
[psharma@dhcp193-66 ~]$ curl -v -X PUT -H 'X-Auth-Token: AUTH_tk0852185723824b3995872c3e57e7d98d' https://10.65.207.238:443/v1/AUTH_test/dir1 -k
* About to connect() to 10.65.207.238 port 443 (#0)
*   Trying 10.65.207.238... connected
* Connected to 10.65.207.238 (10.65.207.238) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 15 12:08:23 2013 GMT
* 	expire date: Jun 14 12:08:23 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> PUT /v1/AUTH_test/dir1 HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-redhat-linux-gnu) libcurl/7.21.7 NSS/3.13.5.0 zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: 10.65.207.238
> Accept: */*
> X-Auth-Token: AUTH_tk0852185723824b3995872c3e57e7d98d
> 
< HTTP/1.1 404 Not Found
< Content-Length: 154
< Content-Type: text/html; charset=UTF-8
< Date: Wed, 15 May 2013 15:53:21 GMT
< 
<html>
 <head>
  <title>404 Not Found</title>
 </head>
 <body>
  <h1>404 Not Found</h1>
  The resource could not be found.<br /><br />



 </body>
* Connection #0 to host 10.65.207.238 left intact








[psharma@dhcp193-66 ~]$ curl -v -X GET -H 'X-Auth-Token: AUTH_tk0852185723824b3995872c3e57e7d98d' https://10.65.207.238:443/v1/AUTH_test -k
* About to connect() to 10.65.207.238 port 443 (#0)
*   Trying 10.65.207.238... connected
* Connected to 10.65.207.238 (10.65.207.238) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 15 12:08:23 2013 GMT
* 	expire date: Jun 14 12:08:23 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> GET /v1/AUTH_test HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-redhat-linux-gnu) libcurl/7.21.7 NSS/3.13.5.0 zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: 10.65.207.238
> Accept: */*
> X-Auth-Token: AUTH_tk0852185723824b3995872c3e57e7d98d
> 
< HTTP/1.1 503 Internal Server Error
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Date: Wed, 15 May 2013 15:57:31 GMT
< 
* Connection #0 to host 10.65.207.238 left intact
* Closing connection #0
[psharma@dhcp193-66 ~]$

Comment 1 pushpesh sharma 2013-05-15 10:39:14 UTC
Created attachment 748183 [details]
dmesgs

Comment 2 pushpesh sharma 2013-05-15 11:04:27 UTC
[root@dhcp207-238 swift]# rpm -qa|grep gluster
glusterfs-fuse-3.3.1-14.fc18.x86_64
glusterfs-swift-account-3.3.1-14.fc18.noarch
glusterfs-swift-proxy-3.3.1-14.fc18.noarch
glusterfs-ufo-3.3.1-14.fc18.noarch
glusterfs-server-3.3.1-14.fc18.x86_64
glusterfs-3.3.1-14.fc18.x86_64
glusterfs-swift-3.3.1-14.fc18.noarch
glusterfs-swift-container-3.3.1-14.fc18.noarch
glusterfs-swift-object-3.3.1-14.fc18.noarch



manual mounting and building again does not help.

Comment 3 Junaid 2013-05-16 07:48:53 UTC
[root@dhcp207-232 swift]# cat /etc/issue
Fedora release 18 (Spherical Cow)
Kernel \r on an \m (\l)

[root@dhcp207-232 swift]# df
Filesystem                           1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg-lv6                    10475520   33104  10442416   1% /mnt/lv6
/dev/mapper/vg-lv7                    10475520   33104  10442416   1% /mnt/lv7
/dev/mapper/vg-lv8                    10475520   33104  10442416   1% /mnt/lv8
/dev/mapper/vg-lv9                    10475520   32976  10442544   1% /mnt/lv9

curl  -v  http://10.65.207.232:8080/v1/AUTH_test
* About to connect() to 10.65.207.232 port 8080 (#0)
*   Trying 10.65.207.232...
* connected
* Connected to 10.65.207.232 (10.65.207.232) port 8080 (#0)
> GET /v1/AUTH_test HTTP/1.1
> User-Agent: curl/7.27.0
> Host: 10.65.207.232:8080
> Accept: */*
> 
< HTTP/1.1 204 No Content
< X-Account-Container-Count: 0
< X-Account-Object-Count: 0
< X-Bytes-Used: 0
< X-Timestamp: 1368629841.43600
< X-Object-Count: 0
< X-Account-Bytes-Used: 0
< X-Type: Account
< X-Container-Count: 0
< Accept-Ranges: bytes
< Content-Length: 0
< Date: Thu, 16 May 2013 13:17:26 GMT
< 
* Connection #0 to host 10.65.207.232 left intact
* Closing connection #0

[root@dhcp207-232 swift]# df
Filesystem                           1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg-lv6                    10475520   33104  10442416   1% /mnt/lv6
/dev/mapper/vg-lv7                    10475520   33104  10442416   1% /mnt/lv7
/dev/mapper/vg-lv8                    10475520   33104  10442416   1% /mnt/lv8
/dev/mapper/vg-lv9                    10475520   32976  10442544   1% /mnt/lv9
localhost:test                        41902080  132480  41769600   1% /mnt/gluster-object/test

Please point me to the machine on which this is reproducible.

Comment 4 Junaid 2013-05-16 07:52:23 UTC
Pushpesh, I figure see the problem with your script. The line

Generate builders:
cd /etc/swift ; /usr/bin/gluster-swift-gen-builders test ; /usr/bin/gluster-swift-gen-builders test2

The problem with this line is when you run /usr/bin/gluster-swift-gen-builders test2, it overwrites the pervious builders.

Can you try 

Generate builders:
cd /etc/swift ; /usr/bin/gluster-swift-gen-builders test test2

Comment 5 pushpesh sharma 2013-05-16 13:17:26 UTC
Will verify this again with multiple volumes , once gluster-sift upstream rpms are available.

Comment 6 pushpesh sharma 2013-07-31 07:20:30 UTC
Works as expected with latest, upstream G4S.Documentation is updated to mount manually and run the ring builder.

Comment 9 Niels de Vos 2014-11-27 14:54:25 UTC
The version that this bug has been reported against, does not get any updates from the Gluster Community anymore. Please verify if this report is still valid against a current (3.4, 3.5 or 3.6) release and update the version, or close this bug.

If there has been no update before 9 December 2014, this bug will get automatocally closed.


Note You need to log in before you can comment on or make changes to this bug.