Bug 1209432

Summary: Using TLS Identities for Authorization is mandatory, not optional
Product: [Community] GlusterFS Reporter: Ernestas Lukoševičius <ernetas>
Component: transportAssignee: bugs <bugs>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: unspecified    
Version: 3.6.2CC: bugs, ernetas, jdarcy, kaushal, ndevos, rabhat, ueberall
Target Milestone: ---Keywords: Patch, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.7.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-30 12:50:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ernestas Lukoševičius 2015-04-07 11:03:27 UTC
Description of problem:


Version-Release number of selected component (if applicable):
glusterfs 3.6.2, from Ubuntu PPA repositories for Trusty.

How reproducible:


Steps to Reproduce:
1. Setup a volume, add access controls for IP addresses, setup SSL by setting client.ssl and server.ssl to on.
2. Stop volume, restart glusterfs-server and start volume again.
3. The volume will not be possible to mount.

Actual results:
/var/log/glusterfs/bricks/data-gluster.log:
[2015-04-07 10:50:42.308465] E [socket.c:384:ssl_setup_connection] 0-tcp.files-server: SSL connect error
[2015-04-07 10:50:42.308534] E [socket.c:2371:socket_poller] 0-tcp.files-server: server setup failed
[2015-04-07 10:50:43.940634] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:43.945638] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:43.949279] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:43.949332] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files1-20797-2015/04/07-10:50:43:649809-files-client-2-0-0 3.6.2
[2015-04-07 10:50:43.950724] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:43.950744] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files1-20790-2015/04/07-10:50:42:638058-files-client-2-0-0 3.6.2
[2015-04-07 10:50:43.987593] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:43.988160] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:43.988180] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files3-30565-2015/04/07-10:50:38:958036-files-client-2-0-1 3.6.2
[2015-04-07 10:50:44.010998] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:44.011378] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:44.011391] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files3-30572-2015/04/07-10:50:39:978376-files-client-2-0-1 3.6.2
[2015-04-07 10:50:45.882219] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:45.883911] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:45.883934] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files2-25060-2015/04/07-10:50:45:857887-files-client-2-0-0 3.6.2
[2015-04-07 10:50:45.910123] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:50:45.911582] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:50:45.911622] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files2-25053-2015/04/07-10:50:44:846049-files-client-2-0-0 3.6.2
[2015-04-07 10:51:41.157280] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:51:41.157703] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:51:41.157717] E [server-handshake.c:596:server_setvolume] 0-files-server: Cannot authenticate client from files3-30825-2015/04/07-10:51:41:131733-files-client-2-0-0 3.6.2
[2015-04-07 10:51:41.176712] E [socket.c:2486:socket_poller] 0-tcp.files-server: error in polling loop

Note that CN for the certificate here is "gluster". It only works when setting auth.ssl-allow to "gluster" and it doesn't work when this parameter is not set. Working example:

[2015-04-07 10:52:43.229281] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:43.230415] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:43.230432] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:43.230444] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files2-25495-2015/04/07-10:52:37:177801-files-client-2-0-0 (version: 3.6.2)
[2015-04-07 10:52:44.224511] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:44.225950] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:44.225985] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:44.226012] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files2-25488-2015/04/07-10:52:36:172744-files-client-2-0-0 (version: 3.6.2)
[2015-04-07 10:52:44.943023] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:44.945207] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:44.945231] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:44.945243] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files1-21442-2015/04/07-10:52:39:915784-files-client-2-0-0 (version: 3.6.2)
[2015-04-07 10:52:44.946016] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:44.947076] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:44.947089] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:44.947100] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files1-21449-2015/04/07-10:52:40:927783-files-client-2-0-0 (version: 3.6.2)
[2015-04-07 10:52:44.988492] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:44.990729] I [socket.c:379:ssl_setup_connection] 0-tcp.files-server: peer CN = gluster
[2015-04-07 10:52:44.991312] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:44.991337] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:44.991359] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files3-31532-2015/04/07-10:52:30:944772-files-client-2-0-1 (version: 3.6.2)
[2015-04-07 10:52:44.992521] I [login.c:39:gf_auth] 0-auth/login: connecting user name: gluster
[2015-04-07 10:52:44.992544] I [login.c:82:gf_auth] 0-auth/login: allowed user names: gluster
[2015-04-07 10:52:44.992566] I [server-handshake.c:585:server_setvolume] 0-files-server: accepted client from files3-31539-2015/04/07-10:52:31:957776-files-client-2-0-1 (version: 3.6.2)

Expected results:
Judging from http://www.gluster.org/community/documentation/index.php/SSL#Using_TLS_Identities_for_Authorization, I would expect that auth.ssl-allow is optional, not mandatory.

Additional info:
Sorry if I tagged the components (access-control instead of docs) wrong - I'm not sure if this is a feature that lacks documentation or a bug.

Comment 1 Ernestas Lukoševičius 2015-04-07 11:09:55 UTC
Not working:

root@files1:/home/supervisor# gluster volume info
 
Volume Name: files
Type: Replicate
Volume ID: 9730c8ff-0992-4b41-a4a7-76aa3c9ae79b
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: files1:/data/gluster
Brick2: files2:/data/gluster
Brick3: files3:/data/gluster
Options Reconfigured:
auth.allow: 10.80.0.32,10.80.0.33,10.80.0.34
cluster.server-quorum-type: server
storage.owner-uid: 33
storage.owner-gid: 33
client.ssl: on
server.ssl: on
ssl.cipher-list: HIGH:!SSLv2
cluster.server-quorum-ratio: 51%

Working:
root@files1:/home/supervisor# gluster volume info
 
Volume Name: files
Type: Replicate
Volume ID: 9730c8ff-0992-4b41-a4a7-76aa3c9ae79b
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: files1:/data/gluster
Brick2: files2:/data/gluster
Brick3: files3:/data/gluster
Options Reconfigured:
auth.allow: 10.80.0.32,10.80.0.33,10.80.0.34
cluster.server-quorum-type: server
storage.owner-uid: 33
storage.owner-gid: 33
client.ssl: on
server.ssl: on
ssl.cipher-list: HIGH:!SSLv2
auth.ssl-allow: gluster
cluster.server-quorum-ratio: 51%

Comment 2 Niels de Vos 2015-04-07 12:32:04 UTC
There is also this doc:

    https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_ssl.md

I am just not sure if that is correct for 3.6.1 too. Maybe Jeff can give some pointers here.

Comment 3 Jeff Darcy 2015-04-07 13:34:25 UTC
This does affect 3.6 AFAICT.  The patch that introduced SSL/TLS authorization is:

    rpc/auth: allow SSL identity to be used for authorization
    Thu Apr 17 23:21:05 2014 +0000
    caa8a4ea50734378e7e19f70b39a837c58e9d229 (master/release-3.6)

The patch that adds a default "auth.ssl-allow=*" is:

    transport: fix default behavior for SSL authorization
    Tue Jan 6 10:03:49 2015 -0500
    548547b2e41c8e2cf79b929405cf18aecbdedebc (master only)

It should be sufficient to backport 548547b2e41c8e2cf79b929405cf18aecbdedebc.

Comment 4 Niels de Vos 2015-04-07 14:16:47 UTC
Thanks Jeff! Adding this one to the planning for the next 3.6 release.

[Well, failing to find the latest 3.6 tracker bug, I'll let Raghavendra add it.]

Comment 5 Kaushal 2016-08-30 12:50:22 UTC
This bug is being closed as GlusterFS-3.6 is nearing its End-Of-Life and only important security bugs will be fixed. This bug has been fixed in more recent GlusterFS releases. If you still face this bug with the newer GlusterFS versions, please open a new bug.

Comment 6 Red Hat Bugzilla 2023-09-14 02:57:39 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days