Bug 438956

Summary: mirrors.fedoraproject maps + to blank
Product: [Fedora] Fedora Reporter: Alexandre Oliva <oliva>
Component: distributionAssignee: Matt Domsch <matt_domsch>
Status: CLOSED UPSTREAM QA Contact: Bill Nottingham <notting>
Severity: low Docs Contact:
Priority: low    
Version: rawhideCC: bob, dcantrell, jeffreyt, jonathansteffan, katzj, rvokal, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: http://mirrors.fedoraproject.org/mirrorlist?path=pub/fedora/linux/releases/test/9-Beta/Fedora/source/SRPMS/libsigc++20-2.2.0-1.fc9.src.rpm&redirect=1
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-04-14 03:30:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Patch to translate urls none

Comment 1 Matt Domsch 2008-03-26 15:37:33 UTC
This smells like the old + vs %20 problem again.

Comment 2 Jeroen van Meeuwen 2008-03-26 16:34:40 UTC
Actually + needs to become %2b as %20 is a space character

Comment 3 Jonathan Steffan 2008-03-26 17:32:23 UTC
Fedora Unity solved this with s/+/%2b/ in the jigdo itself. We might want to
solve this at the server level, though.

I have a really quick rewrite map written up, we should implement it.

Comment 4 Jonathan Steffan 2008-03-26 17:40:24 UTC
Rewritemap:
--------------------------------------------------------------------------------
#!/usr/bin/perl
use strict;
$| = 1;
while(<STDIN>) {
        chomp($_);
        my ($url) =
"http://mirrors.fedoraproject.org/mirrorlist?redirect=1&path=pub/fedora/linux/"
. filter($_);
        print $url;
        # Debug
        print "\n";
}

sub filter {
    my($pure) = @_;
    $pure =~ s/([^a-zA-Z0-9_\-. ])/uc sprintf("%%%02x",ord($1))/eg;
    # We need our /
    $pure =~ s/%2F/\//g;
    return $pure;
}
--------------------------------------------------------------------------------

Implement it with:

RewriteEngine On
RewriteMap seo_download prg:/var/www/download.pl
RewriteRule ^/download/(.*)$ ${seo_download:$1} [R,L]

This is testing and running on my dev server.

http://dev.damaestro.us/download/releases/test/9-Beta/Fedora/source/SRPMS/libsigc++20-2.2.0-1.fc9.src.rpm

I would, however, recommend chaining the rewrite with a modified logic (see next
comment.)





Comment 5 Jonathan Steffan 2008-03-26 17:42:19 UTC
Make sure to have a RewriteLock defined also:

http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html#rewritelock

See next comment for the chained rewrite map.

Comment 6 Jonathan Steffan 2008-03-26 17:54:33 UTC
Rewrite map:
--------------------------------------------------------------------------------
#!/usr/bin/perl
use strict;
$| = 1;
while(<STDIN>) {
        chomp($_);
        my ($url) = "/download-chain/" . filter($_);
        print $url;
        # Debug
        print "\n";
}

sub filter {
    my($pure) = @_;
    $pure =~ s/([^a-zA-Z0-9_\-. ])/uc sprintf("%%%02x",ord($1))/eg;
    # We need our /
    $pure =~ s/%2F/\//g;
    return $pure;
}

--------------------------------------------------------------------------------

Implemented with:

RewriteEngine On
RewriteMap seo_download_chain prg:/var/www/download-chain.pl
RewriteRule ^/download-chain/(.*)$ ${seo_download_chain:$1}
RewriteRule ^/download-chain/(.*)$
http://mirrors.fedoraproject.org/mirrorlist?redirect=1&path=pub/fedora/linux/$1
[R,L]

Obviously, you can [R]edirect to anywhere you want.

This is now implemented on my dev server:

http://dev.damaestro.us/download-chain/releases/test/9-Beta/Fedora/source/SRPMS/libsigc++20-2.2.0-1.fc9.src.rpm

Comment 7 Jesse Keating 2008-03-26 19:05:35 UTC
From the uninitated, I would think mirror manager should simply clean the urls
before handing them back to the client.

Looking in the MM code, where it does the mirrorlist, it could just call
urllib.quote(path), which would properly translate + to %2B.

Of course, this breaks down if you actually pass a url with %2B in it to
urllib.quote (silly bugger).

Seems you can unquote safely and then re-quote.

I'm attaching a mirrormanager patch that may fix it, but I don't have a good way
of testing.

Comment 8 Jesse Keating 2008-03-26 19:06:29 UTC
Created attachment 299214 [details]
Patch to translate urls

Comment 9 Matt Domsch 2008-04-14 03:30:26 UTC
Fixed March 27 with commit 28dde0cf56814f7b87b3edfe9d1444df3cadf625 to
mirrormanager.