Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: I am facing ssl replication problem in fedora-11, I am using openssl to create both clients and server certs and both clients and server certs are signed by same ca, in server its define as in my.cnf file [mysqld] ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem ssl-ca=/etc/mysql/ssl/ca-cert.pem and in clients its define as [client] ssl-ca=/etc/mysql/ssl/ca-cert.pem ssl-key=/etc/mysql/ssl/client-key.pem ssl-cert=/etc/mysql/ssl/client-cert.pem on master i setup slaveuser with ssl like GRANT REPLICATION SLAVE ON *.* TO 'slaveuser'@'abc.com' IDENTIFIED BY 'xxxx' REQUIRE SSL; Now from client when i manually check to connect it gives. [muzi@myhost ~]$ mysql -uslaveuser -h abc.com -p Enter password: ERROR 2026 (HY000): SSL connection error and same error no is shown after setup a replication in show slave status. kindly suggest and give suggestions, also confirm that either this version support ssl or not. Version-Release number of selected component (if applicable):5.1.41-2 How reproducible: Steps to Reproduce: 1. Setup/create both clients and server certs signed by same ca. 2. setup both my.cnf files on clients and servers as mentioned in this thread 3. On server set ssl user for replication 4. mysql -uslaveuser -h abc.com -p Actual results: Expected results: Additional info:
There was an upstream security bug fix that tightened up SSL certificate verification: http://bugs.mysql.com/bug.php?id=47320 The discussion in that report indicates that incorrect self-signed-certificate setups might stop working. I think that's probably what happened to you. You'll need to install the correct public key for your private CA on the client.
(In reply to comment #1) > There was an upstream security bug fix that tightened up SSL certificate > verification: > http://bugs.mysql.com/bug.php?id=47320 > > The discussion in that report indicates that incorrect self-signed-certificate > setups might stop working. > I think that's probably what happened to you. You'll need to install the > correct public key for your private CA on the client. Hmm, here below the description of certs creation , which may be give you more understanding. 1) I setup my local ca on server using openssl x509, common name (CN) is same for my server hostname. like server.abc.com 2) After that i create the server csr using same CN= server.abc.com and signed with same ca. 3) After that i create the client csr using cn = client.abc.com (mysql client machine name) and signed with same server ca, Now for server i have ca-cert.pem (same which-one create for server) server-key.pem server-cert.pem and for clients i have ca-cert.pem (same which one create for above server) client-key.pem client-cert.pem I copy the above 3 files to clients and define in my.cnf like below [client] ssl-ca=/etc/mysql/ca-cert.pem ---> (same ca) ssl-key=/etc/mysql/client-key.pem ssl-cert=/etc/mysql/client-cert.pem and for master [mysqld] ssl-ca=/etc/mysql/ssl/ca-cert.pem --> (same ca) ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem on master i setup slaveuser with ssl like GRANT REPLICATION SLAVE ON *.* TO 'slaveuser'@'abc.com' IDENTIFIED BY 'xxxx' REQUIRE SSL; Now from client when i manually check to connect it gives. [muzi@myhost ~]$ mysql -uslaveuser -h client.abc.com -p Enter password: ERROR 2026 (HY000): SSL connection error Now please suggest where i wrong, or what additional thing is required. All my other servers using same above topology also stop working after upgrade in to new version with same error message.
I did a quick test with a similar config and I was able to SSL connect. You should probably check your certificates first, unless they've just been generated. Try running something like this on every cert.pem file: openssl x509 -noout -subject -issuer -dates -serial -hash -fingerprint -in cert.pem You also don't seem to require client certs, so try using without ssl-ca for mysqld and ssl-{key,cert} for client.
(In reply to comment #3) > I did a quick test with a similar config and I was able to SSL connect. > > You should probably check your certificates first, unless they've just been > generated. Try running something like this on every cert.pem file: > > openssl x509 -noout -subject -issuer -dates -serial -hash -fingerprint -in > cert.pem > > You also don't seem to require client certs, so try using without ssl-ca for > mysqld and ssl-{key,cert} for client. Thanks for the update, here below finger prints for both server and client certs openssl x509 -noout -subject -issuer -dates -serial -hash -fingerprint -in server-cert.pem subject= /C=CA/ST=Berkshire/L=Newbury/O=TEST/OU=mysql/CN=server.abc.com/emailAddress=abc issuer= /C=CA/ST=Berkshire/L=Newbury/O=TEST/OU=mysql/CN=server.abc.com/emailAddress=abc notBefore=Jan 5 15:34:32 2010 GMT notAfter=May 22 15:34:32 2037 GMT serial=A9A414F762E03C1F 55786fae SHA1 Fingerprint=5B:93:52:2E:8F:15:9F:3F:A0:6A:E0:14:C4:ED:4B:3F:46:0C:1C:E1 For client openssl x509 -noout -subject -issuer -dates -serial -hash -fingerprint -in client-cert.pem subject= /C=CA/ST=Berkshire/L=Newbury/O=TEST/OU=mysql/CN=client.abc.com/emailAddress=abc issuer= /C=CA/ST=Berkshire/L=Newbury/O=TEST/OU=mysql/CN=server.abc.com/emailAddress=abc notBefore=Jan 5 15:40:23 2010 GMT notAfter=May 22 15:40:23 2037 GMT serial=A9A414F762E03C20 414288e0 SHA1 Fingerprint=15:38:70:A8:30:1D:4A:19:29:C8:9F:9E:7A:F4:5D:C6:CC:CF:7B:4D Please suggest where i wrong, also you mean i remove the ssl-ca option from both clients and server like in my.cnf as below ? [mysqld] ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem [client] ssl-key=/etc/mysql/ssl/client-key.pem ssl-cert=/etc/mysql/ssl/client-cert.pem
Today i have now verified again by the procedure you described above. on Server i create , local ca-cert.pem and server-cert.pem and server-cert.key files. so in my.cnf of server values are by using on cert files [mysqld] ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem and same ca-cert.pem from which server-cert.pem is singed i copied on client machine. and set client my.cnf as below only define ssl-ca cert from which server certs are singed. [client] ssl-ca=/etc/mysql/ssl/ca-cert.pem after that i setup replication and got the same error again. :(, end results of show slave status on client below Master_SSL_Allowed: Yes Master_SSL_CA_File: /etc/mysql/ssl/ca-cert.pem Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 2026 Last_IO_Error: error connecting to master 'slaveuser.com:3306' - retry-time: 60 retries: 86400 Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.00 sec) also i verified from command same Error 2026. Kindly let me know please where i wrong. as i do according to you described. finger prints of certs i already mentioned.
Keep your tests simple and proceed in steps. It's quite hard to guess your problem from the limited info and not being able to reproduce it. Have you tried some simple tests as running mysql like this right on the database server: mysql --ssl --ssl-ca /etc/mysql/ssl/ca-cert.pem ... If that does not work, does it work with server-cert.pem used instead of ca-cert? When connected, use \s to check if SSL is used. If it works form the same host, try moving to the real client host. Btw, are all your file permissions set properly? Can client read ca-cert?
(In reply to comment #6) > Keep your tests simple and proceed in steps. It's quite hard to guess your > problem from the limited info and not being able to reproduce it. > > Have you tried some simple tests as running mysql like this right on the > database server: > > mysql --ssl --ssl-ca /etc/mysql/ssl/ca-cert.pem ... > > If that does not work, does it work with server-cert.pem used instead of > ca-cert? When connected, use \s to check if SSL is used. If it works form the > same host, try moving to the real client host. > > Btw, are all your file permissions set properly? Can client read ca-cert? Now here below the output, what i did so i think know you may be got the issue i think. on server, i do the following below 1) Removed the old certs 2) Remove the ssl options in my.cnf files like [mysqld] ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem 3) I create the new ca-cert.pem and then check with above command. mysql --ssl --ssl-ca /etc/mysql/ssl/ca-cert.pem so i successfully login to mysql shell. 3) Now i create the server-cert.pem and server-key.pem and sign both files with my local ca-cert.pem which i used above. After that i setup my my.cnf file to use ssl-cert and ssl-key like [mysqld] ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem 4) Now after that i restart and then try to check manually its gives SSL error so i can now remove the ssl option in my.cnf file and restart and then check so it connects successfully but when define options in file its not connect. So this is the problem know, i hope you may be know got what is the root cause in my case.
Aren't you missing an ssl-ca entry in that [mysqld] block? AFAIK both the server and client need access to the same CA certificate.
(In reply to comment #8) > Aren't you missing an ssl-ca entry in that [mysqld] block? AFAIK both the > server and client need access to the same CA certificate. i can try also with it, i can enter below in my.cnf as [mysqld] ssl-ca=/etc/mysql/ssl/ca-cert.pem ssl-key=/etc/mysql/ssl/server-key.pem ssl-cert=/etc/mysql/ssl/server-cert.pem after that i restart mysqld and try to connect on server like mysql -uroot --ssl --ssl-ca /etc/mysql/ssl/ca-cert.pem -p it gives same SSL ERROR 2026 but when i removed the ssl entries in cnf file and then restart and try it connects successfully but with ssl entries defines above it not connect with ssl even on master. so this is the problem, kindly please suggest.
Probably the next thing to ask is exactly how you're generating those certificate files.
(In reply to comment #7) > Now here below the output, what i did so i think know you may be got the issue > i think. Sorry, I completely lost the track. Are you removing / re-creating all those keys / certificates all the time? In your first step 3), which certificate was used to sign server cert? Why do you re-create server key/cert in your second step 3)? Btw, I believe this should help catch certificate issues: openssl verify -purpose sslserver -CAfile ca-cert.pem server-cert.pem (In reply to comment #8) > Aren't you missing an ssl-ca entry in that [mysqld] block? AFAIK both the > server and client need access to the same CA certificate. No, it does work without ssl-ca configured for mysqld. You really only need CA cert if you plan to verify client certificates (and possibly if you need to give it to clients as part of the certificate chain, but that's not this case).
Hello Yes above signature verifing command is so much helpful for me and when i check signatures its fails for server-cert.pem but pass for client-cert.pem, the reason is ca-cert.pem and server-cert.pem use same CN = server.abc.com which is the problem, when i assign different CN to ca-cert and use server-cert with same CN= server.abc.com its gives singature ok. so now i am able to connect with ssl on client :). So i think now in new version of mysql if ca-cert and server-cert use same CN then i think ssl connection is not successful, any way its not the big deal, so i am now happy to ssl, currently i just test the connection with ssl now i am going to implement ssl replication and then update you. Thanks so much for your team.
Hi I am now happy so much to setup ssl based replication successfully, now from my end this thread is resolved now :), I am thanks again for all the participating red hat team. Muzi.