Bug 849177

This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unable to address this
request at this time.

Red Hat invites you to ask your support representative to
propose this request, if appropriate, in the next release of
Red Hat Enterprise Linux.

There's no explicit getsockopt() call in urlgrabber, and I have no idea why it blocks for 63 seconds, although the socket is O_NONBLOCK.

Note that the poll() call has returned {fd=5, revents=POLLERR|POLLHUP}, this indicates that the server has disconnected.  Could you provide a similiar strace of curl, too?  (tcpdump log would be nice, too).

Created attachment 740006 [details]
strace for "curl"

Created attachment 740007 [details]
tcpdump for curl strace

Machine was updated to CentOS 6.4 (yum update) prior to running the newest tests.
Same issue occurs with lengthy delay in the middle...

wget of same URL is still sub-second.


curl is curl-7.19.7-36.el6_4.x86_64

Just to be clear, NOW "curl" also has a 60+ second nap in the middle. So, whatever code changed between 6.3 -> 6.4 affected "curl" in addition to "urlgrabber".

Thanks for the logs.  The FTP server (vsFTPd 2.0.1) does not support extended passive mode (it claims it does, but sends invalid responses).

     0.001410 recvfrom(3, "227 Entering Passive Mode (129,1"..., 16384, 0, NULL, NULL) = 52

According to RFC 2428:

   The first two fields contained in the parenthesis MUST be blank.  The
   third field MUST be the string representation of the TCP port number
   on which the server is listening for a data connection.

strace has truncated the response, and I can't connect to the FTP server, but the reply clearly seems to be bogus.  Curl however parsed port number 57566 somehow, and sent 7 tcp SYN packets that were silently dropped by the server (that's the timeout issue reported).

The curl handling of EPSV responses should be improved, and the client should use PASV immediately.  As a temporary workaround, you can disable it with --disable-epsv option.

In python-urlgrabber, a corresponding setopt(pycurl.FTP_USE_EPSV, 0) could be added to the _set_opts() method, but since that can break other use cases, I don't think it should be made permanent.

According to this http://curl.haxx.se/mail/lib-2011-02/0060.html the problem might not be in the server itself, but a nat/proxy that handles PASV responses fine, but mangles EPSV responses.

Oh, sorry, swapped lines.. "227 Entering Passive Mode (129,1"... was the PASV response.  The problem is with the oher one:

     0.004663 recvfrom(3, "229 Entering Extended Passive Mo"..., 16384, 0, NULL, NULL) = 48

Could you re-run strace with "-s 64", to see the server reply in full?  Thanks.

Created attachment 740437 [details]
strace with more options...

w/ strace -s 64

> recvfrom(3, "229 Entering Extended Passive Mode (|||62284|)\r\n", 16384, 0, NULL, NULL) = 48
> connect(4, {sa_family=AF_INET, sin_port=htons(62284), sin_addr=inet_addr("129.176.212.87")}, 16) = -1 EINPROGRESS (Operation now in progress)
> poll([{fd=4, events=POLLOUT|POLLWRNORM}], 1, 299956) = 1 ([{fd=4, revents=POLLERR|POLLHUP}])

Ok, so the server reply was valid, and curl has correctly parsed it and tried to connect to port 62284.  There's no bug in the client side, neither in curl nor in the urlgrabber.

The connect has timed out (server or network dropping all SYN packets).  There's probably something wrong with the FTP server.

As I wrote before, this may be worked around by disabling EPSV in curl.  Could you confirm that "curl --disable-epsv ftp:///.." helps?

Yes, adding the "--disable-epsv" does resolve the timeout issue. 

So, how does one configure Yum to do this?  

(Remember, my original issue is that the ability to perform yum installs/updates stopped in one of the 6.X updates. Yes, my FTP server is ancient: vsftpd-2.0.1-5.EL4.5.)

I'm afraid you'll have to patch python-urlgrabber, as there's so such option yet, and I don't think we'll add one to support (presumably very few) broken FTP servers.

# patch /usr/lib/python2.7/site-packages/urlgrabber/grabber.py
--- urlgrabber-3.9.1/urlgrabber/grabber.py.old
+++ urlgrabber-3.9.1/urlgrabber/grabber.py
@@ -1159,6 +1159,9 @@ class PyCurlFileObject():
         if not opts:
             opts = self.opts
 
+        # --disable-epsv
+        self.curl_obj.setopt(pycurl.FTP_USE_EPSV, 0)
+
         # keepalives
         if not opts.keepalive:
             self.curl_obj.setopt(pycurl.FORBID_REUSE, 1)

We have run into this same issue on our network. While there may be a limited number of FTP servers that have broken EPSV support, many enterprise firewalls (including Cisco ASRs, various CheckPoint products, and others) do not handle EPSV requests properly (as mentioned in comment #8). I suspect the number of users impacted by this is larger than you may expect. Consider this another request for this feature to be reconsidered, as we are unable to use yum with ftp repos at this point in time.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0839.html