Bug 143691 - yum does not handle '@' character or '%40' for login when authenticating
yum does not handle '@' character or '%40' for login when authenticating
Status: CLOSED UPSTREAM
Product: Fedora
Classification: Fedora
Component: python-urlgrabber (Show other bugs)
12
noarch Linux
medium Severity medium
: ---
: ---
Assigned To: James Antill
: Reopened
Depends On:
Blocks: 576651
  Show dependency treegraph
 
Reported: 2004-12-24 04:40 EST by Bernard Johnson
Modified: 2014-01-21 17:50 EST (History)
2 users (show)

See Also:
Fixed In Version: fc5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 576651 (view as bug list)
Environment:
Last Closed: 2010-11-04 13:43:17 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Bernard Johnson 2004-12-24 04:40:04 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5)
Gecko/20041111 Firefox/1.0

Description of problem:
Neither of the following two examples of baseurl will work with yum:
baseurl=http://joeuser@example.com:mypassword@yum.example.com/repo
baseurl=http://joeuser%40example.com:mypassword@yum.example.com/repo

Both examples fail.  The first gives a traceback (see below) and the
second gives an IO Error (see below).  I would expect the second to
work though.

First example traceback:
Traceback (most recent call last):
  File "/usr/bin/yum", line 8, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 68, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 381, in doCommands
    return self.updatePkgs()
  File "/usr/share/yum-cli/cli.py", line 766, in updatePkgs
    self.doRepoSetup()
  File "/usr/share/yum-cli/cli.py", line 70, in doRepoSetup
    repo.getRepoXML()
  File "/usr/lib/python2.3/site-packages/yum/repos.py", line 465, in
getRepoXML
    result = self.get(relative=remote, local=local, copy_local=1)
  File "/usr/lib/python2.3/site-packages/yum/repos.py", line 443, in get
    checkfunc=checkfunc)
  File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line
414, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line
400, in _mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line
564, in urlgrab
    (url, parts) = self._parse_url(url)
  File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line
666, in _parse_url
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment


Second example error:
http://yum.example.com/repo/repodata/repomd.xml: [Errno 4] IOError:
HTTP Error 401: Authorization Required
Trying other mirror.
Cannot open/read repomd.xml file for repository: example.com
failure: repodata/repomd.xml from example.com: [Errno 256] No more
mirrors to try.



Version-Release number of selected component (if applicable):
yum-2.1.11-3

How reproducible:
Always

Steps to Reproduce:
1. Setup a yum repo that requires authentication
2. Use either form of the example baseurl in the repo config file
3. Run yum
    

Actual Results:  See traceback and error above.

Expected Results:  It should work with one of the example baseurls

Additional info:
Comment 1 Jeremy Katz 2005-09-21 15:22:32 EDT
Is this fixed in newer versions of yum?
Comment 2 Bernard Johnson 2005-09-22 02:42:16 EDT
As of yum-2.4.0-3 in rawhide, no, it is still broken.
Comment 3 Seth Vidal 2005-09-25 18:43:14 EDT
I'm pretty sure this is all about how urllib deals with proxies with @'s in them.

Jeremy, do you remember a bug like this assigned to python? I thought I did.
Comment 4 Jeremy Katz 2006-04-19 15:59:27 EDT
Based on code inspection, this seems fixed in current python-urlgrabber
Comment 5 Bernard Johnson 2006-04-19 17:09:11 EDT
By experience, it's not ;)  This is on FC5 (fully updated).  If you meant
rawhide, I can test that as well.

[bjohnson@localhost ~]$ rpm -q yum python-urlgrabber
yum-2.6.0-1
python-urlgrabber-2.9.8-2
[bjohnson@localhost ~]$

More recent traceback:
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 97, in main
    result, resultmsgs = do()
  File "/usr/share/yum-cli/cli.py", line 481, in doCommands
    return self.updatePkgs()
  File "/usr/share/yum-cli/cli.py", line 957, in updatePkgs
    self.doRepoSetup()
  File "/usr/share/yum-cli/cli.py", line 78, in doRepoSetup
    yum.YumBase.doRepoSetup(self, thisrepo=thisrepo)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 256, in doRepoSetup
    repo.getRepoXML(text=repo)
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 682, in getRepoXML
    cache=self.http_caching == 'all')
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 626, in get
    http_headers=headers,
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 411, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 397, in
_mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 750, in
urlgrab
    (url, parts) = self._parse_url(url)
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 856, in
_parse_url
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment
Comment 6 Matthew Miller 2006-07-10 18:26:38 EDT
Fedora Core 3 is now maintained by the Fedora Legacy project for security
updates only. If this problem is a security issue, please reopen and
reassign to the Fedora Legacy product. If it is not a security issue and
hasn't been resolved in the current FC5 updates or in the FC6 test
release, reopen and change the version to match.

Thank you!
Comment 7 Bernard Johnson 2006-07-10 18:55:02 EDT
Updated to reflect that it affects FC3-rawhide.
Comment 8 Matthew Miller 2006-07-11 14:27:57 EDT
thanks.
Comment 9 Jeremy Katz 2007-04-25 13:06:36 EDT
I can see a way this could still happen, but it'd have to be a URL without a : in it
Comment 10 Bernard Johnson 2007-04-25 14:48:33 EDT
(In reply to comment #9)
> I can see a way this could still happen, but it'd have to be a URL without a :
in it

Are you saying that you think it's fixed for all cases except those that are
missing a ":" between login and password?

If so, I'll do some retesting.
Comment 11 Jeremy Katz 2007-04-25 14:53:38 EDT
(In reply to comment #10)
> (In reply to comment #9)
> > I can see a way this could still happen, but it'd have to be a URL without a :
> in it
> 
> Are you saying that you think it's fixed for all cases except those that are
> missing a ":" between login and password?
> 
> If so, I'll do some retesting.

AFAICT -- I don't have a web server set up for easy testing of it, so I'm having
to kind of poke and prod in bizarre ways.  Eventually I'll get around to setting
up a good test environment for it
Comment 12 Bernard Johnson 2007-04-25 16:25:50 EDT
Ok, after some quick tests, user@domain still does not work:

# yum update
Loading "installonlyn" plugin
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 82, in main
    base.getOptionsConfig(args)
  File "/usr/share/yum-cli/cli.py", line 156, in getOptionsConfig
    (opts, self.cmds) = self.optparser.setupYumConfig()
  File "/usr/share/yum-cli/cli.py", line 1149, in setupYumConfig
    self.base.setupProgessCallbacks()
  File "/usr/share/yum-cli/output.py", line 322, in setupProgessCallbacks
    self.repos.setProgressBar(TextMeter(fo=sys.stdout))
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 491, in <lambda>
    repos = property(fget=lambda self: self._getRepos(),
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 335, in _getRepos
    repo.setup(self.conf.cache, self.mediagrabber)
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 591, in setup
    self._loadRepoXML(text=self)
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 618, in _loadRepoXML
    cache=self.http_caching == 'all')
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 516, in _getFile
    http_headers=headers,
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 411, in urlgrab
    return self._mirror_try(func, url, kw)
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 397, in
_mirror_try
    return func_ref( *(fullurl,), **kwargs )
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 893, in
urlgrab
    (url,parts) = opts.urlparser.parse(url, opts) 
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 666, in parse
    parts = self.process_http(parts)
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 694, in
process_http
    auth_handler.add_password(None, host, user, password)
UnboundLocalError: local variable 'user' referenced before assignment

user%40domain seems to pass "user%40domain" as the user to auth, which causes a
401 on my web server and a "Cannot open/read repomd.xml file for repository:
test" on the console.

If you need a repo to test against, let me know and you can use mine.
Comment 13 Bug Zapper 2008-05-13 21:58:42 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 14 Bug Zapper 2009-06-09 18:00:25 EDT
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 15 seth vidal 2010-01-13 16:45:47 EST
I know this is a long time ago but if anyone involved on this bug still cares I believe we handle this properly in urlgrabber from fedora-12 and rawhide.

testing it to verify would be appreciated.
Comment 16 Bernard Johnson 2010-01-31 14:48:04 EST
I'm not able to test with rawhide, but with Fedora-12 it now works with url encoding '@' to '%40'.

That still seems less than optimum.

If I leave '@' alone, I now get this message:

# yum update
Loaded plugins: presto, refresh-packagekit
http://joeuser@example.com:password@yum.example.com/foo/12/x86_64/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 - ""
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: example-foo. Please verify its path and try again
#


The repo definition looks like this:

[example-foo]
name=Example Foo Packages for Fedora $releasever
baseurl=http://joeuser@example.com:password@yum.example.com/foo/$releasever/$basearch
gpgcheck=0
enabled=1
Comment 17 Martin Poole 2010-03-24 13:48:20 EDT
The fix for this bug is in two parts.

Firstly, the user and password information MUST be encoded correctly.

The requirement is spelt out in 3.1 of rfc1738,

  The user name (and password), if present, are followed by a
  commercial at-sign "@". Within the user and password field, any ":",
  "@", or "/" must be encoded.

So that would be

  baseurl=http://user%40example.com:password@yum.example.com/foo/$releasever/$basearch


The second part is then to unquote the user and password fields after they are split from the host portion.

Simply adding

  user = urllib.unquote( user )
  password = urllib.unquote( password )

after the 


  user, password = user_pass.split(':', 1)

in grabber.py  solves the problem.
Comment 18 seth vidal 2010-08-30 15:15:49 EDT
okay - I've added a patch to urlgrabber upstream to add a 'username' and a 'password' option. Once that is supported in yum you'll be able to do

[repo]
baseurl=url
username=me@host.com
password=mypassword

and it should work.
Comment 19 Bug Zapper 2010-11-04 08:19:19 EDT
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Note You need to log in before you can comment on or make changes to this bug.