Bug 438956 - mirrors.fedoraproject maps + to blank
Summary: mirrors.fedoraproject maps + to blank
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: distribution
Version: rawhide
Hardware: All
OS: Linux
low
low
Target Milestone: ---
Assignee: Matt Domsch
QA Contact: Bill Nottingham
URL: http://mirrors.fedoraproject.org/mirr...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-03-26 06:45 UTC by Alexandre Oliva
Modified: 2014-03-17 03:13 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2008-04-14 03:30:26 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Patch to translate urls (1.45 KB, patch)
2008-03-26 19:06 UTC, Jesse Keating
no flags Details | Diff

Comment 1 Matt Domsch 2008-03-26 15:37:33 UTC
This smells like the old + vs %20 problem again.

Comment 2 Jeroen van Meeuwen 2008-03-26 16:34:40 UTC
Actually + needs to become %2b as %20 is a space character

Comment 3 Jonathan Steffan 2008-03-26 17:32:23 UTC
Fedora Unity solved this with s/+/%2b/ in the jigdo itself. We might want to
solve this at the server level, though.

I have a really quick rewrite map written up, we should implement it.

Comment 4 Jonathan Steffan 2008-03-26 17:40:24 UTC
Rewritemap:
--------------------------------------------------------------------------------
#!/usr/bin/perl
use strict;
$| = 1;
while(<STDIN>) {
        chomp($_);
        my ($url) =
"http://mirrors.fedoraproject.org/mirrorlist?redirect=1&path=pub/fedora/linux/"
. filter($_);
        print $url;
        # Debug
        print "\n";
}

sub filter {
    my($pure) = @_;
    $pure =~ s/([^a-zA-Z0-9_\-. ])/uc sprintf("%%%02x",ord($1))/eg;
    # We need our /
    $pure =~ s/%2F/\//g;
    return $pure;
}
--------------------------------------------------------------------------------

Implement it with:

RewriteEngine On
RewriteMap seo_download prg:/var/www/download.pl
RewriteRule ^/download/(.*)$ ${seo_download:$1} [R,L]

This is testing and running on my dev server.

http://dev.damaestro.us/download/releases/test/9-Beta/Fedora/source/SRPMS/libsigc++20-2.2.0-1.fc9.src.rpm

I would, however, recommend chaining the rewrite with a modified logic (see next
comment.)





Comment 5 Jonathan Steffan 2008-03-26 17:42:19 UTC
Make sure to have a RewriteLock defined also:

http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html#rewritelock

See next comment for the chained rewrite map.

Comment 6 Jonathan Steffan 2008-03-26 17:54:33 UTC
Rewrite map:
--------------------------------------------------------------------------------
#!/usr/bin/perl
use strict;
$| = 1;
while(<STDIN>) {
        chomp($_);
        my ($url) = "/download-chain/" . filter($_);
        print $url;
        # Debug
        print "\n";
}

sub filter {
    my($pure) = @_;
    $pure =~ s/([^a-zA-Z0-9_\-. ])/uc sprintf("%%%02x",ord($1))/eg;
    # We need our /
    $pure =~ s/%2F/\//g;
    return $pure;
}

--------------------------------------------------------------------------------

Implemented with:

RewriteEngine On
RewriteMap seo_download_chain prg:/var/www/download-chain.pl
RewriteRule ^/download-chain/(.*)$ ${seo_download_chain:$1}
RewriteRule ^/download-chain/(.*)$
http://mirrors.fedoraproject.org/mirrorlist?redirect=1&path=pub/fedora/linux/$1
[R,L]

Obviously, you can [R]edirect to anywhere you want.

This is now implemented on my dev server:

http://dev.damaestro.us/download-chain/releases/test/9-Beta/Fedora/source/SRPMS/libsigc++20-2.2.0-1.fc9.src.rpm

Comment 7 Jesse Keating 2008-03-26 19:05:35 UTC
From the uninitated, I would think mirror manager should simply clean the urls
before handing them back to the client.

Looking in the MM code, where it does the mirrorlist, it could just call
urllib.quote(path), which would properly translate + to %2B.

Of course, this breaks down if you actually pass a url with %2B in it to
urllib.quote (silly bugger).

Seems you can unquote safely and then re-quote.

I'm attaching a mirrormanager patch that may fix it, but I don't have a good way
of testing.

Comment 8 Jesse Keating 2008-03-26 19:06:29 UTC
Created attachment 299214 [details]
Patch to translate urls

Comment 9 Matt Domsch 2008-04-14 03:30:26 UTC
Fixed March 27 with commit 28dde0cf56814f7b87b3edfe9d1444df3cadf625 to
mirrormanager.


Note You need to log in before you can comment on or make changes to this bug.