+++ This bug was initially created as a clone of Bug #143691 +++

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5)
Gecko/20041111 Firefox/1.0

Description of problem:
Neither of the following two examples of baseurl will work with yum:
baseurl=http://joeuser@example.com:mypassword@yum.example.com/repo
baseurl=http://joeuser%40example.com:mypassword@yum.example.com/repo

Both examples fail.  The first gives a traceback (see below) and the
second gives an IO Error (see below).  I would expect the second to
work though.

First example traceback:
Traceback (most recent call last):
  File "/usr/bin/yum", line 8, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 68, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 381, in doCommands
    return self.updatePkgs()
  File "/usr/share/yum-cli/cli.py", line 766, in updatePkgs
    self.doRepoSetup()
  File "/usr/share/yum-cli/cli.py", line 70, in doRepoSetup
    repo.getRepoXML()
  File "/usr/lib/python2.3/site-packages/yum/repos.py", line 465, in
getRepoXML
    result = self.get(relative=remote, local=local, copy_local=1)
  File "/usr/lib/python2.3/site-packages/yum/repos.py", line 443, in get
    checkfunc=checkfunc)
  File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line
414, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line
400, in _mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line
564, in urlgrab
    (url, parts) = self._parse_url(url)
  File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line
666, in _parse_url
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment


Second example error:
http://yum.example.com/repo/repodata/repomd.xml: [Errno 4] IOError:
HTTP Error 401: Authorization Required
Trying other mirror.
Cannot open/read repomd.xml file for repository: example.com
failure: repodata/repomd.xml from example.com: [Errno 256] No more
mirrors to try.



Version-Release number of selected component (if applicable):
yum-2.1.11-3

How reproducible:
Always

Steps to Reproduce:
1. Setup a yum repo that requires authentication
2. Use either form of the example baseurl in the repo config file
3. Run yum
    

Actual Results:  See traceback and error above.

Expected Results:  It should work with one of the example baseurls

Additional info:

--- Additional comment from katzj on 2005-09-21 15:22:32 EDT ---

Is this fixed in newer versions of yum?

--- Additional comment from bjohnson on 2005-09-22 02:42:16 EDT ---

As of yum-2.4.0-3 in rawhide, no, it is still broken.

--- Additional comment from skvidal on 2005-09-25 18:43:14 EDT ---

I'm pretty sure this is all about how urllib deals with proxies with @'s in them.

Jeremy, do you remember a bug like this assigned to python? I thought I did.

--- Additional comment from katzj on 2006-04-19 15:59:27 EDT ---

Based on code inspection, this seems fixed in current python-urlgrabber

--- Additional comment from bjohnson on 2006-04-19 17:09:11 EDT ---

By experience, it's not ;)  This is on FC5 (fully updated).  If you meant
rawhide, I can test that as well.

[bjohnson@localhost ~]$ rpm -q yum python-urlgrabber
yum-2.6.0-1
python-urlgrabber-2.9.8-2
[bjohnson@localhost ~]$

More recent traceback:
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 97, in main
    result, resultmsgs = do()
  File "/usr/share/yum-cli/cli.py", line 481, in doCommands
    return self.updatePkgs()
  File "/usr/share/yum-cli/cli.py", line 957, in updatePkgs
    self.doRepoSetup()
  File "/usr/share/yum-cli/cli.py", line 78, in doRepoSetup
    yum.YumBase.doRepoSetup(self, thisrepo=thisrepo)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 256, in doRepoSetup
    repo.getRepoXML(text=repo)
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 682, in getRepoXML
    cache=self.http_caching == 'all')
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 626, in get
    http_headers=headers,
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 411, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 397, in
_mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 750, in
urlgrab
    (url, parts) = self._parse_url(url)
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 856, in
_parse_url
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment

--- Additional comment from mattdm on 2006-07-10 18:26:38 EDT ---

Fedora Core 3 is now maintained by the Fedora Legacy project for security
updates only. If this problem is a security issue, please reopen and
reassign to the Fedora Legacy product. If it is not a security issue and
hasn't been resolved in the current FC5 updates or in the FC6 test
release, reopen and change the version to match.

Thank you!

--- Additional comment from bjohnson on 2006-07-10 18:55:02 EDT ---

Updated to reflect that it affects FC3-rawhide.

--- Additional comment from mattdm on 2006-07-11 14:27:57 EDT ---

thanks.

--- Additional comment from katzj on 2007-04-25 13:06:36 EDT ---

I can see a way this could still happen, but it'd have to be a URL without a : in it

--- Additional comment from bjohnson on 2007-04-25 14:48:33 EDT ---

(In reply to comment #9)
> I can see a way this could still happen, but it'd have to be a URL without a :
in it

Are you saying that you think it's fixed for all cases except those that are
missing a ":" between login and password?

If so, I'll do some retesting.

--- Additional comment from katzj on 2007-04-25 14:53:38 EDT ---

(In reply to comment #10)
> (In reply to comment #9)
> > I can see a way this could still happen, but it'd have to be a URL without a :
> in it
> 
> Are you saying that you think it's fixed for all cases except those that are
> missing a ":" between login and password?
> 
> If so, I'll do some retesting.

AFAICT -- I don't have a web server set up for easy testing of it, so I'm having
to kind of poke and prod in bizarre ways.  Eventually I'll get around to setting
up a good test environment for it

--- Additional comment from bjohnson on 2007-04-25 16:25:50 EDT ---

Ok, after some quick tests, user@domain still does not work:

# yum update
Loading "installonlyn" plugin
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 82, in main
    base.getOptionsConfig(args)
  File "/usr/share/yum-cli/cli.py", line 156, in getOptionsConfig
    (opts, self.cmds) = self.optparser.setupYumConfig()
  File "/usr/share/yum-cli/cli.py", line 1149, in setupYumConfig
    self.base.setupProgessCallbacks()
  File "/usr/share/yum-cli/output.py", line 322, in setupProgessCallbacks
    self.repos.setProgressBar(TextMeter(fo=sys.stdout))
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 491, in <lambda>
    repos = property(fget=lambda self: self._getRepos(),
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 335, in _getRepos
    repo.setup(self.conf.cache, self.mediagrabber)
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 591, in setup
    self._loadRepoXML(text=self)
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 618, in _loadRepoXML
    cache=self.http_caching == 'all')
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 516, in _getFile
    http_headers=headers,
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 411, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 397, in
_mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 893, in
urlgrab
    (url,parts) = opts.urlparser.parse(url, opts) 
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 666, in parse
    parts = self.process_http(parts)
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 694, in
process_http
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment

user%40domain seems to pass "user%40domain" as the user to auth, which causes a
401 on my web server and a "Cannot open/read repomd.xml file for repository:
test" on the console.

If you need a repo to test against, let me know and you can use mine.

--- Additional comment from fedora-triage-list on 2008-05-13 21:58:42 EDT ---

Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

--- Additional comment from fedora-triage-list on 2009-06-09 18:00:25 EDT ---


This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

--- Additional comment from svidal on 2010-01-13 16:45:47 EST ---

I know this is a long time ago but if anyone involved on this bug still cares I believe we handle this properly in urlgrabber from fedora-12 and rawhide.

testing it to verify would be appreciated.

--- Additional comment from bjohnson on 2010-01-31 14:48:04 EST ---

I'm not able to test with rawhide, but with Fedora-12 it now works with url encoding '@' to '%40'.

That still seems less than optimum.

If I leave '@' alone, I now get this message:

# yum update
Loaded plugins: presto, refresh-packagekit
http://joeuser@example.com:password@yum.example.com/foo/12/x86_64/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 - ""
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: example-foo. Please verify its path and try again
#


The repo definition looks like this:

[example-foo]
name=Example Foo Packages for Fedora $releasever
baseurl=http://joeuser@example.com:password@yum.example.com/foo/$releasever/$basearch
gpgcheck=0
enabled=1

--- Additional comment from mpoole on 2010-03-24 13:48:20 EDT ---

The fix for this bug is in two parts.

Firstly, the user and password information MUST be encoded correctly.

The requirement is spelt out in 3.1 of rfc1738,

  The user name (and password), if present, are followed by a
  commercial at-sign "@". Within the user and password field, any ":",
  "@", or "/" must be encoded.

So that would be

  baseurl=http://user%40example.com:password@yum.example.com/foo/$releasever/$basearch


The second part is then to unquote the user and password fields after they are split from the host portion.

Simply adding

  user = urllib.unquote( user )
  password = urllib.unquote( password )

after the 


  user, password = user_pass.split(':', 1)

in grabber.py  solves the problem.