Bug 1795607

Summary: rsyslogd only offers TLSv1.3 when using RELP
Product: Red Hat Enterprise Linux 8 Reporter: Steven Haigh <netwiz>
Component: rsyslogAssignee: Attila Lakatos <alakatos>
Status: CLOSED ERRATA QA Contact: Dalibor Pospíšil <dapospis>
Severity: low Docs Contact:
Priority: low    
Version: 8.0CC: dapospis, netwiz, omoris
Target Milestone: rcKeywords: AutoVerified, Triaged
Target Release: 8.0Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rsyslog-8.2102.0-3.el8 Doc Type: No Doc Update
Doc Text:
If this bug requires documentation, please select an appropriate Doc Type value.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-09 18:04:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1932795    
Bug Blocks:    

Description Steven Haigh 2020-01-28 12:49:52 UTC 
Description of problem:
Transplanting a RELP configuration from CentOS 7 -> CentOS 8, TLS connections are no long allowed from rsyslog clients using RELP.

Version-Release number of selected component (if applicable):
# rpm -qa | grep rsyslog                                                                                                                                                                                                                   
rsyslog-8.37.0-13.el8.x86_64                                                                                                                                                                                                                  rsyslog-doc-8.37.0-13.el8.noarch                                                                                                                                                                                                              
rsyslog-relp-8.37.0-13.el8.x86_64 

How reproducible:
Always

Steps to Reproduce:
1. Configure rsyslog with the following dropin config file: /etc/rsyslog.d/repl.conf

# Provides UDP syslog reception                                                                                                                                          
# for parameters see http://www.rsyslog.com/doc/imudp.html                                                                                                               
module(load="imudp") # needs to be done just once                                                                                                                       
input(type="imudp" port="514")

# Provides TCP syslog reception
# for parameters see http://www.rsyslog.com/doc/imtcp.html
module(load="imtcp") # needs to be done just once
input(type="imtcp" port="514")

module(load="imrelp")
input(type="imrelp" port="2514" tls="on" maxDataSize="10k") 

## Template
$template boot,"/var/log/syslog/ip/%fromhost-ip%/boot.log"
$template cron,"/var/log/syslog/ip/%fromhost-ip%/cron"
$template messages,"/var/log/syslog/ip/%fromhost-ip%/messages"
$template secure,"/var/log/syslog/ip/%fromhost-ip%/secure"
$template maillog,"/var/log/syslog/ip/%fromhost-ip%/maillog"

#kern.*                                                 /dev/console
cron.*                                                  -?cron
mail.*                                                  -?maillog
*.info;mail.none;authpriv.none;cron.none                -?messages
authpriv.*                                              -?secure
local7.*                                                -?boot

2) Configure client with the following in /etc/rsyslog.d/log-forwarder.conf

module(load="omrelp")
action(type="omrelp"
        target="10.1.1.254"
        port="2514"
        tls="on"
        template="RSYSLOG_SyslogProtocol23Format"
        action.reportSuspensionContinuation="on"
        action.resumeRetryCount="-1"
        action.resumeInterval="10"
)

Actual results:
Errors logged to the journal:
rsyslogd[12030]: imrelp[2514]: error 'TLS handshake failed [gnutls error -87: No supported cipher suites have been found.]', object  'lstn 2514: conn to clt 10.1.1.81/10.1.1.81' - input may not work as intended [v8.37.0-13.el8 try http://www.rsyslog.com/e/2353 ]
rsyslogd[12030]: imrelp[2514]: error 'TLS handshake failed [gnutls error -87: No supported cipher suites have been found.]', object  'lstn 2514: conn to clt 10.1.1.93/10.1.1.93' - input may not work as intended [v8.37.0-13.el8 try http://www.rsyslog.com/e/2353 ]
rsyslogd[12030]: imrelp[2514]: error 'TLS handshake failed [gnutls error -87: No supported cipher suites have been found.]', object  'lstn 2514: conn to clt 10.1.1.92/10.1.1.92' - input may not work as intended [v8.37.0-13.el8 try http://www.rsyslog.com/e/2353 ]
rsyslogd[12030]: imrelp[2514]: error 'TLS handshake failed [gnutls error -87: No supported cipher suites have been found.]', object  'lstn 2514: conn to clt 10.1.1.81/10.1.1.81' - input may not work as intended [v8.37.0-13.el8 try http://www.rsyslog.com/e/2353 ]
rsyslogd[12030]: imrelp[2514]: error 'TLS handshake failed [gnutls error -87: No supported cipher suites have been found.]', object  'lstn 2514: conn to clt 10.1.1.91/10.1.1.91' - input may not work as intended [v8.37.0-13.el8 try http://www.rsyslog.com/e/2353 ]

Additional info:

Connecting via OpenSSL s_client shows:
# openssl s_client -connect localhost:2514                                                                                                                                                                                                 
CONNECTED(00000003)                                                                                                                                                                                                                           
write:errno=0                                                                                                                                                                                                                                 
---                                                                                                                                                                                                                                           
no peer certificate available                                                                                                                                                                                                                 
---                                                                                                                                                                                                                                           
No client certificate CA names sent                                                                                                                                                                                                           
Server Temp Key: X25519, 253 bits                                                                                                                                                                                                             
---                                                                                                                                                                                                                                           
SSL handshake has read 133 bytes and written 299 bytes                                                                                                                                                                                        
Verification: OK                                                                                                                                                                                                                              
---                                                                                                                                                                                                                                           
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384                                                                                                                                                                                                
Secure Renegotiation IS NOT supported                                                                                                                                                                                                         
Compression: NONE                                                                                                                                                                                                                             
Expansion: NONE                                                                                                                                                                                                                               
No ALPN negotiated                                                                                                                                                                                                                            
Early data was not sent                                                                                                                                                                                                                       
Verify return code: 0 (ok)                                                                                                                                                                                                                    
---

This configuration seemed to work fine on CentOS 7 - but does not function at all on CentOS 8.
Comment 1 Steven Haigh 2020-01-28 13:06:23 UTC 
Error on the client side:

rsyslogd[517]: omrelp[10.1.1.254:2514]: error 'TLS handshake failed [gnutls error -110: The TLS connection was non-properly terminated.]', object  'conn to srvr 10.1.1.254:2514' - action may not work as inte
nded [v8.1911.0 try https://www.rsyslog.com/e/2353 ]
rsyslogd[517]: omrelp[10.1.1.254:2514]: error 'error opening connection to remote peer', object  'conn to srvr 10.1.1.254:2514' - action may not work as intended [v8.1911.0 try https://www.rsyslog.com/e/2353
 ]
rsyslogd[517]: omrelp[10.1.1.254:2514]: error 'TLS handshake failed [gnutls error -110: The TLS connection was non-properly terminated.]', object  'conn to srvr 10.1.1.254:2514' - action may not work as inte
nded [v8.1911.0 try https://www.rsyslog.com/e/2353 ]
rsyslogd[517]: omrelp[10.1.1.254:2514]: error 'error opening connection to remote peer', object  'conn to srvr 10.1.1.254:2514' - action may not work as intended [v8.1911.0 try https://www.rsyslog.com/e/2353
 ]
Comment 2 Jiří Vymazal 2020-02-13 12:43:53 UTC 
For start it is not recommended to turn on TLS, either trough relp or plain TCP without setting tls.AuthMode (refer to docs on how to set it). While it is stated as not mandatory in docs, in reality not setting it results in it being left  at "none" which does not provide much security over plain TCP/relp and can result in various other problems as the "none" value is there mainly for debugging purposes, not for any production use.

If the above will not solve your problem and same errors persist I would recommend moving this bug against gnutls as the error message flows originally from there.
Comment 3 Steven Haigh 2020-02-14 07:10:58 UTC 
There's a difference between using TLS for Auth and TLS for in-transit encryption.

The client will still check the CN of the server certificate - and we secure access via IP address ranges at a network level.

As a central syslog server, this is more than sufficient.

This is also a regression from EL7.
Comment 4 Daiki Ueno 2020-03-17 13:38:38 UTC 
I tried to reproduce this with a standalone server invocation: GNUTLS_DEBUG_LEVEL=10 /usr/sbin/rsyslogd -d -n

1. Without tls.authMode, the server behaves as if it only accepts TLS 1.3 connection. However, there are indication that the handshake was actually failed:

  1620.257229666:imrelp.c       : imrelp.c: librelp: generic error: ecode 10031, emsg 'TLS handshake failed [gnutls error -43: Error in the certificate.]'

2. With a proper tls.authMode setting (tls.authMode="name" tls.permittedpeer=["..."]), both TLS 1.3 and TLS 1.2 handshakes actually complete as below.

I'm wondering why it looks like TLS 1.3 is accepted in (1); perhaps there might be a missing alert sending logic in rsyslog.

# gnutls-cli --x509cafile ca.pem --x509certfile gnutls/doc/credentials/x509/clicert.pem --x509keyfile gnutls/doc/credentials/x509/clikey.pem --priority NORMAL:-VERS-TLS1.3 localhost -p 20514
Processed 1 CA certificate(s).
Processed 1 client X.509 certificates...
Resolving 'localhost:20514'...
Connecting to '::1:20514'...
- Certificate type: X.509
- Got a certificate list of 1 certificates.
- Certificate[0] info:
 - subject `CN=GnuTLS Test Server (RSA certificate)', issuer `CN=GnuTLS Test CA', serial 0x4de0b4ca, RSA key 2432 bits, signed using RSA-SHA256, activated `2011-05-28 08:39:39 UTC', expires `2038-10-12 08:39:40 UTC', pin-sha256="ZCnc2x+EUztg6ShnEvwtcHxusyXqJ5RJLNCDLc+lVNE="
	Public Key ID:
		sha1:482334530a8931384a5aeacab6d2a6dece1d2b18
		sha256:6429dcdb1f84533b60e9286712fc2d707c6eb325ea2794492cd0832dcfa554d1
	Public Key PIN:
		pin-sha256:ZCnc2x+EUztg6ShnEvwtcHxusyXqJ5RJLNCDLc+lVNE=

- Status: The certificate is trusted. 
- Successfully sent 1 certificate(s) to server.
- Description: (TLS1.2)-(ECDHE-SECP256R1)-(RSA-SHA256)-(AES-256-GCM)
- Session ID: 72:62:28:27:4C:6F:50:47:7F:C6:E3:6D:38:12:D9:2B:95:E4:8C:ED:6A:5C:23:91:2A:85:EF:70:08:92:33:1A
- Options: extended master secret, safe renegotiation,
- Handshake was completed
Comment 5 Daiki Ueno 2020-03-18 05:44:51 UTC 
Anyway I'm moving this back to rsyslog, as this is essentially an rsyslog configuration issue (as Jiri mentioned in comment 2) and we can't do much from GnuTLS side.
Comment 18 errata-xmlrpc 2021-11-09 18:04:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rsyslog and its related dependencies bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4193