Bug 443033

Summary: ftp install fails because of REST command unsupported by server
Product: [Fedora] Fedora Reporter: Dag <den.mail>
Component: python-urlgrabberAssignee: James Antill <james.antill>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: 9CC: katzj
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-07-14 15:27:16 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dag 2008-04-18 10:42:55 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.8.1.13) Gecko/20080325 Fedora/2.0.0.13-1.fc8 Firefox/2.0.0.13

Description of problem:
When trying to install Fedora 9 Beta or rawhide via "askmethod" then URL:
ftp://ftp.proxad.net/mirrors/fedora.redhat.com/fedora/linux/development/i386/os

install is starting normaly and graphicaly, you are asked to enter several parameters, the partition you disk, then ask to choose packages to install.
At this stage of installation, anaconda downloads successfully every filelist.xml.gz and other repodata files, then displays the packages chooser screen.
then you validate and it resolving dependencies, downloading the repo informations (i.e.: primary.sqlite.bz2).
Then it begins the installation by truing to download the packages...
And it always fails because it can't retrieve the first package (can't open file: corrupted or missing package error).

So I opened Wireshark and captured the FTP sequence:
it appeared that to retrieve packages anaconda issues a REST command. The above server doesn't support REST command.
So anaconda is unable to download any package from a server that doesn't support FTP REST command.

I'm encountering this error since FC3, but never analyzed the network packets until now.

Version-Release number of selected component (if applicable):
anaconda 11.4.0.73

How reproducible:
Always


Steps to Reproduce:
1. boot install with "askmethod" kernel parameter
2. choose 'URL' as the method
3. enter ftp://ftp.proxad.net/mirrors/fedora.redhat.com/fedora/linux/development/i386/os
4. then continue the installation until Anaconda is trying to download the first package (which was eVince for me).

Actual Results:
Anaconda will try to download the package over and over again using the FTP command REST which is unsupported by this server.
Anaconda doesn't handle the "unknown REST command" error, so it can't download the files.

Expected Results:
Anaconda should detect that the FTP server doesn't support the REST command and try to download the packages using the same method it used to download repo informations (i.e.: primary.sqlite.bz2).

Additional info:
here is the captured flow:
No.     Time        Source                Destination           Protocol Info
  23429 931.543501  212.27.60.27          192.168.149.47        FTP      Response: 220 Welcome to ProXad FTP server
  23431 931.544584  192.168.149.47        212.27.60.27          FTP      Request: USER anonymous
  23433 931.598645  212.27.60.27          192.168.149.47        FTP      Response: 331 Please specify the password.
  23434 931.599257  192.168.149.47        212.27.60.27          FTP      Request: PASS anonymous@
  23435 931.640092  212.27.60.27          192.168.149.47        FTP      Response: 230 Login successful.
  23436 931.640977  192.168.149.47        212.27.60.27          FTP      Request: CWD mirrors
  23437 931.689001  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23438 931.689670  192.168.149.47        212.27.60.27          FTP      Request: CWD fedora.redhat.com
  23439 931.731830  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23440 931.732648  192.168.149.47        212.27.60.27          FTP      Request: CWD fedora
  23441 931.775678  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23442 931.777206  192.168.149.47        212.27.60.27          FTP      Request: CWD linux
  23443 931.819343  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23444 931.820102  192.168.149.47        212.27.60.27          FTP      Request: CWD development
  23445 931.868805  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23446 931.870064  192.168.149.47        212.27.60.27          FTP      Request: CWD i386
  23447 931.911142  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23448 931.911785  192.168.149.47        212.27.60.27          FTP      Request: CWD os
  23449 931.961793  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23450 931.962459  192.168.149.47        212.27.60.27          FTP      Request: CWD Packages
  23451 932.003787  212.27.60.27          192.168.149.47        FTP      Response: 250 Directory successfully changed.
  23452 932.004510  192.168.149.47        212.27.60.27          FTP      Request: TYPE I
  23453 932.059447  212.27.60.27          192.168.149.47        FTP      Response: 200 Switching to Binary mode.
  23454 932.060133  192.168.149.47        212.27.60.27          FTP      Request: TYPE A
  23455 932.117169  212.27.60.27          192.168.149.47        FTP      Response: 200 Switching to ASCII mode.
  23456 932.118466  192.168.149.47        212.27.60.27          FTP      Request: PASV
  23457 932.159549  212.27.60.27          192.168.149.47        FTP      Response: 227 Entering Passive Mode (212,27,60,27,242,243)
  23462 932.202682  192.168.149.47        212.27.60.27          FTP      Request: NLST evince-2.22.1.1-1.fc9.i386.rpm
  23463 932.250687  212.27.60.27          192.168.149.47        FTP      Response: 150 Here comes the directory listing.
  23471 932.321168  212.27.60.27          192.168.149.47        FTP      Response: 226 Directory send OK.
  23473 932.325316  192.168.149.47        212.27.60.27          FTP      Request: TYPE I
  23475 932.367299  212.27.60.27          192.168.149.47        FTP      Response: 200 Switching to Binary mode.
  23476 932.368718  192.168.149.47        212.27.60.27          FTP      Request: PASV
  23478 932.417265  212.27.60.27          192.168.149.47        FTP      Response: 227 Entering Passive Mode (212,27,60,27,202,185)
  23484 932.461053  192.168.149.47        212.27.60.27          FTP      Request: REST 440
  23485 932.513485  212.27.60.27          192.168.149.47        FTP      Response: 500 Unknown command.
  23490 932.560879  212.27.60.27          192.168.149.47        FTP      Response: 500 OOPS: 
  23491 932.561207  212.27.60.27          192.168.149.47        FTP      Response: vsf_sysutil_recv_peek: no data
  23494 932.561567  212.27.60.27          192.168.149.47        FTP      Response:

Comment 1 Chris Lumens 2008-04-18 13:54:29 UTC
anaconda's just (indirectly) using what urllib provides, and that's in the
python package.

Comment 2 John Poelstra 2008-04-18 14:51:23 UTC
Re comment #1... is this a bug that should remain open and be fixed or reported
upstream?

Comment 3 Dag 2008-04-29 17:37:36 UTC
Thnaks Chris.
So should I report it in Python bugzilla ?
I set severity to high because it's preventing users from installing Fedora.

Comment 4 Dag 2008-05-04 18:51:47 UTC
I wonder if this is really a bug from the Python urllib library because Anaconda
retrieves the stage2 file from that FTP server that doesn't support the REST
command.

Please, anyone can help here ?

Comment 5 Bug Zapper 2008-05-14 09:37:11 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 8 James Antill 2008-05-15 13:24:02 UTC
 I can't remember why I put this in needinfo, however opening a bug upstream
will likely help it get fixed faster.


Comment 9 Dag 2008-05-23 07:51:19 UTC
I chatted IRC with some Python devs on #python and it seems to be a
Python-urlgrabber issue because there is no trace of any REST command in
Python's urllib.
So I reassign it to the python-urlgrabber component.

Comment 10 James Antill 2008-05-23 14:29:01 UTC
 Ok, maybe I'd meant to ask this before (hence the needinfo :) ... But there is
this code in urlgrabber/byterange:

            except ftplib.error_perm, reason:
                if str(reason)[:3] == '501':
                    # workaround for REST not supported error

...can you see what happens if you add 500 to the above test (which is what
vsftpd seems to be returning).


Comment 11 Bug Zapper 2009-06-10 00:14:38 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 Bug Zapper 2009-07-14 15:27:16 UTC
Fedora 9 changed to end-of-life (EOL) status on 2009-07-10. Fedora 9 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.