RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1370455 - postfix: haproxy proxy SMTPS load balancing doesn't work
Summary: postfix: haproxy proxy SMTPS load balancing doesn't work
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: postfix
Version: 7.2
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: 7.2
Assignee: Jaroslav Škarvada
QA Contact: Alois Mahdal
URL:
Whiteboard:
Depends On:
Blocks: 1420851 1549614
TreeView+ depends on / blocked
 
Reported: 2016-08-26 11:23 UTC by Ramesh Sahoo
Modified: 2020-02-14 17:54 UTC (History)
8 users (show)

Fixed In Version: postfix-2.10.1-7.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-30 09:34:56 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
upstream patch (1.12 KB, patch)
2016-09-15 09:30 UTC, Ondřej Lysoněk 		no flags Details | Diff
haproxy.cfg (3.27 KB, text/plain)
2016-09-16 09:25 UTC, Ramesh Sahoo 		no flags Details
master.cf (6.06 KB, text/plain)
2016-09-16 09:26 UTC, Ramesh Sahoo 		no flags Details
main.cf (26.70 KB, text/plain)
2016-09-16 09:27 UTC, Ramesh Sahoo 		no flags Details
postfix master patch (555 bytes, text/plain)
2016-09-16 11:31 UTC, Ondřej Lysoněk 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:3085 0 None None None 2018-10-30 09:35:13 UTC

Description Ramesh Sahoo 2016-08-26 11:23:09 UTC 
Description of problem:

I am not able to configure HAPROXY for sending proxied SMTP-over-SSL requests to a Postfix daemon on port 465. After setting the parameter "send-proxy" in a Postfix backend and setting "smtpd_upstream_proxy_protocol=haproxy" in the Postfix server, the connection hangs.

The problem is known by Postfix developers and patches are available here:

http://postfix.1071664.n5.nabble.com/smtpd-upstream-proxy-protocol-smtpd-tls-wrappermode-td79550.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815079#5


This is what I get in my setup:

$ openssl s_client -connect localhost:10465
CONNECTED(00000003)
140057306949536:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 247 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

$ tail /var/log/maillog
Aug 10 16:02:03 abc-rhel7 postfix/smtpd[10966]: connect from localhost[127.0.0.1]
Aug 10 16:03:03 abc-rhel7 postfix/smtpd[10966]: SSL_accept error from localhost[127.0.0.1]: lost connection
Aug 10 16:03:03 abc-rhel7 postfix/smtpd[10966]: lost connection after CONNECT from localhost[127.0.0.1]
Aug 10 16:03:03 abc-rhel7 postfix/smtpd[10966]: disconnect from localhost[127.0.0.1]

How reproducible:

haproxy.cf;

# Bug SMTP BEGIN
listen smtp-ssl 127.0.0.1:10465
    mode tcp
    option tcplog
    no option http-server-close
    timeout connect 5s
    server smtp-ssl-localhost 127.0.0.1:465 send-proxy
Comment 4 Ondřej Lysoněk 2016-09-15 09:30:32 UTC 
Created attachment 1201164 [details]
upstream patch

I'm attaching the upstream patch. I tested it and it works.

However, I think that the output of openssl client which you are seeing is caused by misconfiguration, rather than this bug. I encountered it myself as well, while trying to reproduce the bug. Can you post your whole haproxy.cfg, /etc/postfix/master.cf and /etc/postfix/main.cf?
Comment 5 Ramesh Sahoo 2016-09-16 04:21:51 UTC 
I'll attach configuration soon.
Comment 6 Ramesh Sahoo 2016-09-16 09:25:48 UTC 
Created attachment 1201576 [details]
haproxy.cfg

HaProxy configuration
Comment 7 Ramesh Sahoo 2016-09-16 09:26:58 UTC 
Created attachment 1201577 [details]
master.cf

Postfix master.cf
Comment 8 Ramesh Sahoo 2016-09-16 09:27:47 UTC 
Created attachment 1201578 [details]
main.cf

Postfix main.cf
Comment 9 Ramesh Sahoo 2016-09-16 09:28:38 UTC 
All requested configuration files attached.
Comment 10 Ondřej Lysoněk 2016-09-16 11:31:33 UTC 
Created attachment 1201603 [details]
postfix master patch

You need to tell Postfix, where your private key is. Apply the attached patch to /etc/postfix/master.cf (where /etc/postfix/postfix-key.pem is your private key),
restart Postfix and HAProxy, and then you should see the actual buggy behaviour:

After the first connection using openssl s_client, since smtpd started, the connection hangs with the output 'CONNECTED(00000003)'. If you terminate openssl client and try to connect again, it succeeds. The patch eliminates the initial hang after smtpd is started.
Comment 13 Ramesh Sahoo 2016-09-20 03:43:16 UTC 
Hello after I applied the patch, I still the same behavior in RHEL7.2

[root@cls1 haproxy]# openssl s_client -connect localhost:10465
CONNECTED(00000003)
139841599666080:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 247 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
[root@cls1 haproxy]# openssl s_client -connect localhost:465
CONNECTED(00000003)
140684749150112:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 247 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

master.cf;

[root@cls1 postfix]# egrep -v $'^[ \t]*(#|$)'  master.cf
smtp      inet  n       -       n       -       -       smtpd
smtps     inet  n       -       n       -       -       smtpd
   -o smtpd_tls_wrappermode=yes
   -o smtpd_tls_cert_file=/etc/postfix/postfix_cert.pem
   -o smtpd_tls_key_file=/etc/postfix/postfix_key.pem
   -o smtpd_upstream_proxy_protocol=haproxy
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache

[root@cls1 postfix]# netstat -ntlup | egrep 'master|10465'
tcp        0      0 127.0.0.1:465           0.0.0.0:*               LISTEN      3430/master         
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      3430/master         
tcp        0      0 127.0.0.1:10465         0.0.0.0:*               LISTEN      3318/haproxy
Comment 14 Ondřej Lysoněk 2016-09-22 15:53:51 UTC 
When you run

openssl s_client -connect localhost:10465

does it exit immediately, or is there a 30+ seconds delay? Can you try if running the command for the second time changes anything?
Comment 15 Ramesh Sahoo 2016-09-23 05:37:21 UTC 
There is no delay/wait for 30 seconds, I get response immediately within few milliseconds.
Comment 24 Jaroslav Škarvada 2018-05-07 14:35:49 UTC 
The patch works for me.

For reporter: try whether your TLS is working, i.e. disable haproxy by commenting out the following line in your master.cf:
   -o smtpd_upstream_proxy_protocol=haproxy

And then try to connect directly with s_client to the port 465:
$ openssl s_client -connect localhost:465

If it doesn't work, there is probably problem with your TLS certificate or TLS is misconfigured on your machine.

Also for haproxy to work I had to tweak SELinux, e.g.:
# setsebool -P haproxy_connect_any 1
Comment 26 Alois Mahdal 2018-09-27 04:24:51 UTC 
Reviewed the patch and re-run test suite.  No problems found.
Comment 28 errata-xmlrpc 2018-10-30 09:34:56 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3085
Note You need to log in before you can comment on or make changes to this bug.