RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1400632 - With perl-libwww-perl-5.883-3 https_proxy environment variable ceases to be honored by default
Summary: With perl-libwww-perl-5.883-3 https_proxy environment variable ceases to be h...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: perl-libwww-perl
Version: 6.9
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: perl-maint-list
QA Contact: Martin Kyral
Lenka Špačková
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-01 16:41 UTC by Piyush Bhoot
Modified: 2020-05-14 15:27 UTC (History)
5 users (show)

Fixed In Version: perl-libwww-perl-5.833-5.el6
Doc Type: Release Note
Doc Text:
The *LWP::UserAgent* Perl module now correctly handles proxy settings for HTTPS requests The *LWP::UserAgent* Perl module previously did not honor HTTPS proxy environment variables by default. The _perl-libwww-perl_ package version 5.883-3 started using the *IO::Socket::SSL* module instead of the *Net::SSL* module for implementing TLS. Consequently, applications that rely on processing of the `https_proxy` environment variable in the *Net:SSL* module established connections directly to the HTTPS server instead of through the HTTPS proxy server. With this update, the *Net::SSL* module's behavior has been added to the *LWP::UserAgent* module to ensure that the `https_proxy` and `HTTPS_PROXY` environment variables are honored if no `env_proxy` option has been passed to the *LWP::UserAgent* module's `new()` method. Additionally, proxy specifications without a URL schema are now recognized. As a result, applications using the *Net::SSL* module correctly work after switching to the *IO::Socket::SSL* cryptographic back end from the _perl-libwww-perl_ package.
Clone Of:
: 1415093 (view as bug list)
Environment:
Last Closed: 2017-03-21 09:25:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
simple script (1.05 KB, application/x-perl)
2016-12-01 16:41 UTC, Piyush Bhoot
no flags Details
Fix (1.79 KB, patch)
2016-12-02 10:57 UTC, Petr Pisar
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0599 0 normal SHIPPED_LIVE perl-libwww-perl bug fix update 2017-03-21 12:26:06 UTC

Description Piyush Bhoot 2016-12-01 16:41:01 UTC
Created attachment 1226885 [details]
simple script

Description of problem:

With perl-libwww-perl-5.883-2 the $env{https_proxy} variable ceases to be honoured


Version-Release number of selected component (if applicable):
perl-libwww-perl-5.883-2 

How reproducible:
Always

Steps to Reproduce:
1. check soap_test.pl script

Actual results:
Never connects to webproxy

Expected results:
Should connect to web proxy

Additional info:
Verified via Wireshark & tcpdump that the server never attempts to use the web proxy

Comment 1 Petr Pisar 2016-12-02 08:38:57 UTC
The proxy specification is wrong:

$ENV{https_proxy} = "proxy.kcc.com:80"

The variable value must have a protocol:

$ENV{https_proxy} = "http://proxy.kcc.com:80"

as can be verified with LWP that is used underneath the SOAP::Lite:

$ https_proxy=localhost:80 lwp-request -m GET https://localhost:443/
501 Protocol scheme 'localhost' is not supported
$ https_proxy=http://localhost:80 lwp-request -m GET https://localhost:443/
500 Can't connect to localhost:80 (connect: Connection refused)

But you are right that SOAP::Lite somehow causes ignoring the https_proxy environment variable:

$ https_proxy=http://localhost:80/ strace -e connect perl -e 'use SOAP::Lite; SOAP::Lite->proxy(q{https://localhost:443/})->call(q{root})'
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
connect(3, {sa_family=AF_INET6, sin6_port=htons(443), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
connect(3, {sa_family=AF_INET6, sin6_port=htons(443), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 ECONNREFUSED (Connection refused)
500 Can't connect to localhost:443 (connect: Connection refused) at -e line 1
+++ exited with 2 +++

The same issue is with plain HTTP.

Comment 2 Petr Pisar 2016-12-02 09:38:06 UTC
The reason for the failure is complicated:

SOAP::Lite does not bother with HTTP proxies at all. Everything is relied to LWP::UserAgent.

LWP::UserAgent does not honor environment variables by default:

           LWP::UserAgent->new() [...] If the
           "env_proxy" option is passed in with a TRUE value, then proxy
           settings are read from environment variables (see env_proxy()
           method below).

And thus SOAP::Lite applications do not honor environment variables by default too. But one can fix them by adding env_proxy=>1 arguments to SOAP::Lite->proxy() call:

$ https_proxy=http://localhost:80 strace -e connect perl -e 'use SOAP::Lite; SOAP::Lite->proxy(q{https://localhost:443/}, env_proxy=>1)->call(q{root})'
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)
500 Can't connect to localhost:80 (connect: Connection refused) at -e line 1
+++ exited with 2 +++

This was discussed in upstream SOAP::Lite bug report <https://rt.cpan.org/Public/Bug/Display.html?id=58981>.

The reason why lwp-request tool honors the environment is that it explicitly calls env_proxy() on an LWP::UserAgent object.

The reason why your program stopped working after upgrading to perl-libwww-perl-5.883-3 is that the updated package started using IO::Socket::SSL instead of Net::SSL for implementing TLS and Net::SSL unfortunately processes the environment variable on its own and diverges the HTTPS requests to a proxy covertly.

In other words, your application controlled Net::SSL instead of SOAP::Lite (and thus LWP::UserAgent).

You have two options how to fix your application:

(1) The best fix is to add ", env_proxy => 1" to SOAP::Lite->proxy call like this:

my $soap = SOAP::Lite
        -> proxy('https://kcc.service-now.com/change_request.do?SOAP',
                  env_proxy => 1);

(2) You can export PERL_NET_HTTPS_SSL_SOCKET_CLASS=Net::SSL environment variable that will reverts LWP::UserAgent behavior to use Net::SSL instead of the recommended and default IO::Socket::SSL.

Red Hat could modify perl-libwww-perl package to always honor the proxy environment variables. But this modification would be implemented only in Red Hat Enterprise Linux version 6. Not in version 7 or newer.

I will try to amend the perl-libwww-perl package to honor the proxy variables by default to restore compatibility with existing systems, but I strongly recommend you you fix your applications to pass the "env_proxy => 1" arguments as documented in LWP::UserAgent.

Comment 3 Petr Pisar 2016-12-02 10:57:09 UTC
Created attachment 1227272 [details]
Fix

This patch enables processing HTTPS_PROXY and https_proxy variables if env_proxy LWP::UseAgent option is not specified. It also allows to specify the proxy server without a schema.

Comment 4 Petr Pisar 2016-12-02 11:46:53 UTC
How to test:

(1) Use LWP::UserAgent without env_proxy option to connect to an HTTPS server while https_proxy environment variable is set to a HTTP proxy server and verify the TCP connection is opened to the proxy server, not to the HTTPS server. Example:

$ https_proxy=http://localhost:80 perl -e 'use LWP::UserAgent; LWP::UserAgent->new->get(q{https://localhost:443/})'

Before: There it connects to localhost:443.
After: It connects to localhost:80.

(2) Repeat the same with HTTPS_PROXY with the same results as in (1).

(3) Specify https_proxy without the "http://" schema. It should also work.

Please note that this applies to HTTPS proxy. Not to HTTP (http_proxy or HTTP_PROXY) variables. Please also note that it does not apply to LWP::UserAgent->new() invocations that have env_proxy argument.

Comment 10 Piyush Bhoot 2016-12-02 14:40:19 UTC
Thanks for quick and insightful updates over it!

Comment 13 errata-xmlrpc 2017-03-21 09:25:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0599.html

Comment 14 vani 2018-06-12 15:27:49 UTC
Hi All,

We have recently upgraded RHEL OS from 6.7 to 6.9. After upgrade we have observed that service-now incidents are not creating and failing with below error.

Can't connect to rstproxy.rwe.com:8080 (SSL connect attempt failed with unknown errorerror:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol)

Could you please help on this error ?

We have perl-libwww-perl-5.833-5 version.

Comment 15 Petr Pisar 2018-06-13 07:23:03 UTC
(In reply to vani from comment #14)
> We have recently upgraded RHEL OS from 6.7 to 6.9. After upgrade we have
> observed that service-now incidents are not creating and failing with below
> error.
> 
> Can't connect to rstproxy.rwe.com:8080 (SSL connect attempt failed with
> unknown errorerror:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown
> protocol)
> 
> Could you please help on this error ?
> 
Please contact Red Hat support <https://access.redhat.com/support/contact/technicalSupport>.


Note You need to log in before you can comment on or make changes to this bug.