Bug 207278 - upgrade failed to get ftp url - path contains /%2F/
upgrade failed to get ftp url - path contains /%2F/
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
rawhide
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Chris Lumens
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-09-20 08:47 EDT by David Timms
Modified: 2008-02-12 07:23 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-02-12 07:23:34 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description David Timms 2006-09-20 08:47:17 EDT
Description of problem:
attempted upgrade (text) mode cant find packages from the ftp server, even
though they are there.

Version-Release number of selected component (if applicable):
rescue iso of 2006-09-19

Steps to Reproduce:
1. boot rescue iso
2. linux (ran in text mode)
3. choose upgrade existing (5.91)
4. go ahead 
  
Actual results:
Error: The file info-4.8-11.1.i386.rpm cannot be opened. This is due to a
missing file or perhaps a corrupt package. Please verify your mirror contains
all required packages, and try using a different one.

If you reboot, your system will be left in an inconsistent state that will
likely require reinstallation.
  Reboot   Retry.

Expected results:
complete upgrade.

Additional info:
Pressing retry returned the same message.
VT3 shows:
linux/core/development/i386/os/Fedora/RPMS/perl-Digest-HMAC-1.01-15.noarch.rpm
failed
WARNING : Try 1/10 for ftp://download.fedora.redhat.com/%2F/pub/fedora/
linux/core/development/i386/os/Fedora/RPMS/info-4.8-11.1.i386.rpm failed
...
WARNING : Try 10/10 for ftp://download.fedora.redhat.com/%2F/pub/fedora/
linux/core/development/i386/os/Fedora/RPMS/info-4.8-11.1.i386.rpm failed
WARNING : Failed to get ftp://download.fedora.redhat.com/%2F/pub/fedora/
linux/core/development/i386/os/Fedora/RPMS/info-4.8-11.1.i386.rpm from mirror

ps. don't know if this is expected when testing with rawhide, but it might be
that the additional characters %2F in the path are causing the problem.
---
just tried command line wget/ftp and found:
[davidt@davidtdesktop fc6]$ wget
ftp://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os/Fedora/RPMS/
--21:47:33-- 
ftp://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os/Fedora/RPMS/
           => `.listing'
Resolving download.fedora.redhat.com... 209.132.176.221, 209.132.176.20,
209.132.176.220, ...
Connecting to download.fedora.redhat.com|209.132.176.221|:21... connected.
Logging in as anonymous ...
Error in server greeting.
Retrying.

--21:47:35-- 
ftp://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os/Fedora/RPMS/
  (try: 2) => `.listing'
Connecting to download.fedora.redhat.com|209.132.176.221|:21... connected.
Logging in as anonymous ...
Error in server greeting.
Retrying.

--21:47:37-- 
ftp://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os/Fedora/RPMS/
  (try: 3) => `.listing'
Connecting to download.fedora.redhat.com|209.132.176.221|:21... connected.
Logging in as anonymous ...
Error in server greeting.
Retrying.

[davidt@davidtdesktop fc6]$ ftp download.fedora.redhat.com Connected to
download.fedora.redhat.com.
421 There are too many connections from your internet address.
ftp>
---
There is only the one machine connecting from my address, so I guess something
is amiss. I waited 1.5 hours and keep gettting the same message albiet with
different package every few tries.
Comment 1 Jeremy Katz 2006-09-20 13:15:55 EDT
What did you enter for the install location?
Comment 2 David Timms 2006-09-20 17:23:35 EDT
site: download.fedora.redhat.com
folder: pub/fedora/linux/core/development/i386
---
This was repeatable this morning with a one day newer: (but probably 1 day old)
1e56a6b85aab92d57dc88adda25ab3ac8a592ad0  FC-development-i386-rescuecd.iso

This time it ended up trying 1-10/10 for
ftp://download.fedora.redhat.com/%2F/pub/fedora/linux/core/
development/i386/os/Fedora/RPMS/libXv-1.0.1-4.1.i386.rpm failed

I checked the folder in firefox (ftp) and I could change folders to the folder
above os, but then attempting to enter into either iso or os folders I get a:
  ~"Too many connected users from your address" dialog.

There is only one machine within my NAT firewall trying to access the ftp site.

Trying again a few minutes later (in firefox) and I was able to read the folder
and download the .rpm listed above without problem (on another machine).

At this stage it returned to the package installation status screen showing 0/0
packages upgraded.

The user interface (text) was able to choose retry after about 5 minutes, and
then gets to cant dl .../info....rpm.
Comment 3 Jeremy Katz 2006-09-21 11:37:32 EDT
Hmm... could be related to keepalive stuff potentially
Comment 4 David Timms 2006-09-27 19:05:10 EDT
With rescue image: dff4bff3236c3c8434fbcda70baa7806b8d0bdfb 
FC-development-i386-rescuecd.iso  2006-09-26, this is repeatable (on same
notebook). Also repeatable when performing new install and formatting all
partitions.
--
On VT3 same messages appear (this time for rpm-libs-4....)
VT2: ftp download.fedora.redhat.com
Trying 209.132.176.20...
Connected to download.fedora.redhat.com (209.132.176.20)
421 There are too many connections from your internet address.
--
I'll repeat tonight on another machine/different mirrors/http.
Comment 5 Gabriel Schulhof 2006-10-24 14:39:25 EDT
Yep - I can confirm this with FC6 as shipped. People can download the CD images
onto a HDD and point the installer at the directory containing them using the
"Local HDD" option, AFAIK.

I'm gonna try that.

HTH
Comment 6 Tim Jackson 2006-12-17 05:36:01 EST
I think the problem here is that anaconda sometimes spuriously suffixes a "./"
to the install URL. I've seen this with HTTP installs. For example, consider the
case where I have an FC6 DVD ISO mounted on the root of a HTTP server (e.g.
http://10.0.0.1/). What do I enter in anaconda?

a) Enter "10.0.0.1" for the "Web site name" and nothing for the "Fedora Core
Directory" -> anaconda won't let me do it, saying "You must enter a directory"

b) Enter "10.0.0.1" for the "Web site name" and "/" for the "Fedora Core
directory" -> anaconda requests files using HTTP requests like this: (from an
Apache log):
GET /./RELEASE-NOTES-en_US.UTF-8.html HTTP/1.1" 404 406 "-" "urlgrabber/2.9.9"

Note the spurious "./". Now, in the case of Apache, it happens to ignore this
but this probably isn't the case for all HTTP/FTP servers and anaconda shouldn't
generate noise like that.

(It seems to me that it would also make more sense just to have a single URL
input instead of the weird split between hostname and path.)
Comment 7 Chris Lumens 2007-08-24 14:06:21 EDT
Is this fixed in F8 test 2?  (Note that this release isn't quite out yet, but I
think we have this issue taken care of.)
Comment 8 David Timms 2007-08-26 17:57:46 EDT
I tried for a giggle with F8t1 rescue iso. This seemed to be going OK, but got
to the same WARNING message on VT3:
23:35:16 WARNING : Try 10/10 for ftp://download.fedora.redhat.com/%2F/pub
/fedora/linux/releases/test/7.90/Fedora/i386/os/Fedora
/libxklavier-3.2-1.fc8.i386.rpm failed
23:35:16 WARNING : Failed to get ftp://download.fedora.redhat.com/%2F/pub
/fedora/linux/releases/test/7.90/Fedora/i386/os/Fedora
/libxklavier-3.2-1.fc8.i386.rpm from mirror 1/1

There is two warning messages at each second, so perhaps it tries every 0.5 seconds.

VT1 {in text mode}:
Welcome to Fedora
 The file libxklavier-3.2-1.fc8.i386.rpm cannot be opened.   
 This is due to a missing file or perhaps a corrupt          
 package.  Please verify your mirror contains all required   
 packages, and try using a different one.                    
                                         tarting 
 If you exit, your system will be left in an inconsistent    
 state that will likely require reinstallation. 
           Reboot                 Retry

In this attempt, a check of the download url using firefox shows the file is
present at the URL, can be downloaded and the rpm extracts with fileroller.

Pressing retry {after 8 hours} allows it to continue {Starting install process}.
It then had 2 misses with libdhcp, but succeeded on third attempt.
Had 2 misses on another file, but proceeded {again without showing a
retry/reboot dialog}.
Then 4 minutes later, 10 misses with libxslt-1.1.21-1.fc8.i386.rpm and the error
dialog.
Went OK for a few minutes and then:
07:48:30/31/31/32/32 5 misses on pap...rpm which is present
07:49:11/12/13/13/14/14 5 + fail on pap...rpm
Retrying within one minute failed, but waiting more than a few minutes then
retry succeeds in getting that file.

It seems to be the mirror {master} is rejecting the download. If that is the
case it is a bad thing, because it could lead to broken install / upgrades on
user machines.
However, the installer might be able to handle the situation better:
- ie not 5x tries only 0.5 second apart.
- actually look at the error message returned in the attempted get.
- provide the error message to the user.
- keeping the connection alive {I imagine wget is run for each new path. If so,
then instead of one connection to the ftp server, there is potentially 1000's of
connections {that might be considered DenialOfService attempt.
====
Last two boot isos 2007-08-25/26 have another error {bug 254681} which makes it
so this is not currently testable.
Comment 9 Chris Lumens 2007-12-18 10:33:51 EST
Well anaconda in rawhide is no longer adding the %2F to URLs it tries to get, so
hopefully the original problem here will go away.  If F9t1 clears this problem
up, we can go ahead and close this bug report out.

As for your specific suggestions:
(1) I don't see any relevant options in the urlgrabber package, so we will have
to implement the waiting in anaconda ourselves.
(2) and (3) are way harder to do than it should be.  There are a whole lot of
layers of code involved here and the real exceptions get interpreted and
re-raised several different times so once we know that an error has occurred, we
really can't figure out what it was.
(4) We actually use urlgrabber, which supports keepalive connections for HTTP
but I don't know that it does for FTP.  This should be a separate bug against
that python module.
Comment 10 Paul Zimmermann 2007-12-18 15:52:06 EST
This bug seems related to {bug 152378} and {bug 237548} (see my
comments at the end of the latter). Does anybody know a mirror which
accepts the spurious /%2F/ (trying desesperately to install F8)?
Comment 11 Chris Lumens 2008-02-08 12:36:28 EST
Is this better in F9 Alpha?
Comment 12 David Timms 2008-02-12 07:23:34 EST
Yes.
Detail:
1. Booted http://mirror.pacific.net.au/ ...
linux/fedora/linux/releases/test/9-Alpha/Fedora/i386/os/images/boot.iso
on vmware-server-1.0.4 on F8 {kernel-2.6.23.14-115.fc8}
2. Entered URL:
ftp://download.fedora.redhat.com/pub/fedora/releases/test/9-Alpha/Fedora/i386/os
Result:
a. It's sitting for a long time at preparing to install {2+hours}.
b. vt shows WARNING try 1/10 of xxxx.rpm failed.
c. seems to do nothing for some minutes, then continues, usually having the same
problem again.
The failed paths no longer contain /%2F/.

Closing this bug resolved as reported {however reporting a fresh bug to keep
other symptoms separate}.

Note You need to log in before you can comment on or make changes to this bug.