Bug 1328445

Summary: [RHEL-6] gluster-swift: Fix exceptions raised when there are mutliple instances
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Prashanth Pai <ppai>
Component: gluster-swiftAssignee: Prashanth Pai <ppai>
Status: CLOSED ERRATA QA Contact: surabhi <sbhaloth>
Severity: medium Docs Contact:
Priority: medium    
Version: rhgs-3.1CC: asrivast, nlevinki, rcyriac, rhinduja, rhs-bugs, sbhaloth, thiago
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.1.3   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: swiftonfile-1.13.1-8 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1312833 Environment:
Last Closed: 2016-06-23 05:36:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1312833    
Bug Blocks: 1299184    

Comment 2 Prashanth Pai 2016-05-03 09:16:02 UTC
Upstream changes (rhel7) on review. Moving to POST.

Comment 3 Prashanth Pai 2016-05-04 07:43:04 UTC
Upstream change (no distinction between el6 and el7) : http://review.gluster.org/#/c/14118/

Downstream change (el7): https://code.engineering.redhat.com/gerrit/#/c/73583/
Downstream change (el6): https://code.engineering.redhat.com/gerrit/#/c/73591/

Comment 4 surabhi 2016-06-05 12:16:18 UTC
To verify this issue, tried to setup haproxy on rhel6. Haproxy package is not available in rhel6 base channel, took it from brew and installed and did the config settings as required.
Once the settings are configured , starting haproxy fails with following error:

service haproxy restart
Stopping haproxy:                                          [FAILED]
Starting haproxy: [ALERT] 156/114703 (12463) : Starting frontend swift-frontend: cannot bind socket [10.70.47.14:8080]
                                                           [FAILED] 
working on to verify if this is an issue with 2 node haproxy server. Will update the BZ soon.
Same BZ is already verified in RHEL7.

Comment 5 surabhi 2016-06-06 11:29:32 UTC
The issue was with port which was given for haproxy server. the swift and haproxy was trying to bind on the same port. With the help of Prashanth was able to fix that and continued with the test for multi-node swift.
Tried with PUT of 2000 objects from one terminal and 3000 from another in parallel, checked the haproxy.log which shows that the call is forwarded to all the nodes in cluster which were configured.


Jun  6 11:25:38 localhost haproxy[372]: 10.70.47.14:54697 [06/Jun/2016:11:25:38.169] swift-frontend swift-backend/dhcp46-83 0/0/0/99/99 201 273 - - ---- 1/1/0/1/0 0/0 "PUT /v1/AUTH_test/container1/object3 HTTP/1.1"
Jun  6 11:25:38 localhost haproxy[372]: 10.70.47.14:54697 [06/Jun/2016:11:25:38.268] swift-frontend swift-backend/dhcp47-14 0/0/0/91/91 201 273 - - ---- 1/1/0/0/0 0/0 "PUT /v1/AUTH_test/container1/object4 HTTP/1.1"
Jun  6 11:25:38 localhost haproxy[372]: 10.70.47.14:54697 [06/Jun/2016:11:25:38.359] swift-frontend swift-backend/dhcp46-83 0/0/0/82/82 201 273 - - ---- 1/1/0/0/0 0/0 "PUT /v1/AUTH_test/container1/object5 HTTP/1.1"
Jun  6 11:25:38 localhost haproxy[372]: 10.70.47.14:54697 [06/Jun/2016:11:25:38.441] swift-frontend swift-backend/dhcp47-14 0/0/0/85/85 201 273 - - ---- 1/1/0/0/0 0/0 "PUT /v1/AUTH_test/container1/object6 HTTP/1.1"

Verified with  PUT  and there were no exceptions seen.Moving the BZ to verified.

Comment 7 errata-xmlrpc 2016-06-23 05:36:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2016:1246