Bug 1218167
| Summary: | [GlusterFS 3.6.3]: Brick crashed after setting up SSL/TLS in I/O access path with error: "E [socket.c:2495:socket_poller] 0-tcp.gluster-native-volume-3G-1-server: error in polling loop" | |||
|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | ssamanta | |
| Component: | glusterd | Assignee: | Jeff Darcy <jdarcy> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | ||
| Severity: | high | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 3.6.3 | CC: | bugs, mzywusko, rabhat, ueberall | |
| Target Milestone: | --- | Keywords: | Triaged | |
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | glusterfs-v3.6.4 | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1222908 (view as bug list) | Environment: | ||
| Last Closed: | 2016-02-04 15:27:21 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1211643 | |||
| Bug Blocks: | 1216965, 1222908 | |||
|
Description
ssamanta
2015-05-04 10:50:36 UTC
I was unable to reproduce this (in 100 tries) on a Fedora 21 system with the 3.6.3-1 packages from download.gluster.org and OpenSSL 1.0.1j. I notice that you were using OpenSSL 1.0.1e. Before I downgrade my test system, or build a new one, can we please verify that 1.0.1e was the correct OpenSSL version to have on your test system? Also, how exactly were the certificates generated? What SSL "subject" did you use? It's possible that this is a manifestation of a multi-threading issue, which tends to show up in X509_verify_cert. See http://review.gluster.org/#/c/10075/ for details. That would explain the non-deterministic appearance of the bug. Perhaps we need to backport that patch to 3.6? REVIEW: http://review.gluster.org/10591 (socket: use OpenSSL multi-threading interfaces) posted (#1) for review on release-3.6 by Jeff Darcy (jdarcy) After talking with Kaushal earlier I came to know that openssl version to be used is OpenSSL 1.0.1e. I/O data access path from single client(without enabling management SSL/TLS) was working fine with GlusterFS3.6.2. Do we need to use OpenSSL 1.0.1j? I am using fedora-20. Installed Packages Name : openssl Arch : x86_64 Epoch : 1 Version : 1.0.1e Release : 42.fc20 Size : 1.5 M Repo : installed From repo : fedora-updates Summary : Utilities from the general purpose cryptography library with TLS implementation URL : http://www.openssl.org/ License : OpenSSL Description : The OpenSSL toolkit provides support for secure communications between : machines. OpenSSL includes a certificate management tool and shared : libraries which provide various cryptographic algorithms and : protocols. [root@remote-gluster-server ~]# REVIEW: http://review.gluster.org/10591 (socket: use OpenSSL multi-threading interfaces) posted (#2) for review on release-3.6 by Jeff Darcy (jdarcy) This issue is more frequently seen and so marking this as a blocker. I think the OpenSSL version is a red herring. At the time I asked, I was still pretty much in the dark and trying to gather information; I hadn't yet realized that the symptom here closely matches that which http://review.gluster.org/#/c/10075/ had fixed in later versions. I've posted http://review.gluster.org/10591 as a 3.6 backport, and http://review.gluster.org/10617 so that it can pass regression tests (nothing will on 3.6 because of changes to the test machines). They both *have* passed regression tests, and merely await review/merging. This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-v3.6.4, please open a new bug report. glusterfs-v3.6.4 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://www.gluster.org/pipermail/gluster-users/2015-July/022826.html [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user |