Bug 1323614 - Cannot authenticate: requests.exceptions.MissingSchema: Invalid URL 'stage/auth/token/obtain/': No schema supplied.
Summary: Cannot authenticate: requests.exceptions.MissingSchema: Invalid URL 'stage/au...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: pdc-client
Version: 25
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Stanislav Ochotnicky
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-04 09:01 UTC by Petr Pisar
Modified: 2017-12-12 10:52 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-12 10:52:41 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Petr Pisar 2016-04-04 09:01:29 UTC
I tried pdc-client-0.9.0-1.fc23 like this:

$ pdc rpm list --name perl
Traceback (most recent call last):
  File "/usr/bin/pdc", line 22, in <module>
    runner.run()
  File "/usr/lib/python2.7/site-packages/pdc_client/runner.py", line 101, in run
    self.client = pdc_client.PDCClient(self.args.server, page_size=self.args.page_size)
  File "/usr/lib/python2.7/site-packages/pdc_client/__init__.py", line 153, in __init__
    token = self.obtain_token()
  File "/usr/lib/python2.7/site-packages/pdc_client/__init__.py", line 167, in obtain_token
    return self.auth[end_point]._()['token']
  File "<string>", line 1, in <lambda>
  File "/usr/lib/python2.7/site-packages/beanbag/namespace.py", line 131, in fn
    *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 102, in call
    return self.make_request(path, verb, kwargs, body)
  File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 151, in make_request
    r = self.session.request(verb, path, params=params, data=ebody)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 454, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 388, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 293, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 353, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL 'stage/auth/token/obtain/': No schema supplied. Perhaps you meant http://stage/auth/token/obtain/?

I tried specify server:

$ pdc -s dev rpm list --name perl
Traceback (most recent call last):
  File "/usr/bin/pdc", line 22, in <module>
    runner.run()
  File "/usr/lib/python2.7/site-packages/pdc_client/runner.py", line 101, in run
    self.client = pdc_client.PDCClient(self.args.server, page_size=self.args.page_size)
  File "/usr/lib/python2.7/site-packages/pdc_client/__init__.py", line 153, in __init__
    token = self.obtain_token()
  File "/usr/lib/python2.7/site-packages/pdc_client/__init__.py", line 167, in obtain_token
    return self.auth[end_point]._()['token']
  File "<string>", line 1, in <lambda>
  File "/usr/lib/python2.7/site-packages/beanbag/namespace.py", line 131, in fn
    *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 102, in call
    return self.make_request(path, verb, kwargs, body)
  File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 155, in make_request
    "Bad response code: %d" % (r.status_code,))
beanbag.bbexcept.BeanBagException: Bad response code: 401

It seems broken for me.

Comment 1 Stanislav Ochotnicky 2016-04-04 09:07:48 UTC
I guess we should improve config handling. 

Separately I don't expect that pdc-client can easily authenticate against Fedora instance due to the fact it's not using kerberos and pdc-client does not have FAS-specific support.

If you authenticate against https://pdc.fedoraproject.org/

and then go to:
https://pdc.fedoraproject.org/rest_api/v1/auth/token/obtain/

You will be able to get your token which can then be used in pdc_client invocations. I guess we could add support for tokens in "pdc" client itself

Comment 2 Lubomír Sedlář 2016-04-04 09:16:35 UTC
Unless specified otherwise, pdc client uses stage server. There should be a check if such server is configured.

For Fedora the authentication indeed is not automatic. When you get your token from url mentioned by Stano, you can put it in the config.

Drop this in ~/.config/pdc/client_config.json:

{
  "dev": {
    "token": "PUT TOKEN HERE"
  }
}

Comment 3 Petr Pisar 2016-04-04 10:19:55 UTC
Thanks. It works that way.

Comment 4 Jan Kurik 2016-07-26 05:04:43 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.

Comment 5 chuzhang 2017-08-02 08:00:23 UTC
Is this bug still existing in fedora25 when you input the token? If not, maybe we can close this one.

Comment 6 Petr Pisar 2017-08-02 09:58:25 UTC
(In reply to chuzhang from comment #5)
> Is this bug still existing in fedora25 when you input the token? If not,
> maybe we can close this one.

It worked in comment #3. Now I'm unable to obtain the token:

$ curl --negotiate -u : -H "Accept: application/json"  https://pdc.fedoraproject.org/rest_api/v1/auth/token/obtain/
{"detail":"Authentication credentials were not provided."}

Although I have a Fedora's TGT.

Comment 7 chuzhang 2017-08-03 02:56:57 UTC
@Petr Ok. I got it. But for this problem I am not the proper one to help to answer as this is fedora's instance, not owned by my team. It is using different authentication backend, maybe Ralph Bean's team can help to answer it.

Comment 8 Petr Pisar 2017-08-03 07:41:45 UTC
(In reply to chuzhang from comment #7)
> @Petr Ok. I got it. But for this problem I am not the proper one to help to
> answer as this is fedora's instance, not owned by my team. It is using
> different authentication backend, maybe Ralph Bean's team can help to answer
> it.

That means I cannot test this bug report. Please test it yourself and act accordingly.

Nevertheless, I'm not content with the pdc client behavior. If I remove the ~/.config/pdc/client_config.json file, I get this back trace:

$ pdc -s dev rpm list --name perl
Traceback (most recent call last):
  File "/usr/bin/pdc", line 22, in <module>
    runner.run()
  File "/usr/lib/python3.5/site-packages/pdc_client/runner.py", line 159, in run
    ssl_verify=ssl_verify)
  File "/usr/lib/python3.5/site-packages/pdc_client/__init__.py", line 153, in __init__
    token = self.obtain_token()
  File "/usr/lib/python3.5/site-packages/pdc_client/__init__.py", line 167, in obtain_token
    return self.auth[end_point]._()['token']
  File "/usr/lib/python3.5/site-packages/beanbag/namespace.py", line 131, in fn
    *args, **kwargs)
  File "/usr/lib/python3.5/site-packages/beanbag/url_v1.py", line 102, in call
    return self.make_request(path, verb, kwargs, body)
  File "/usr/lib/python3.5/site-packages/beanbag/url_v1.py", line 151, in make_request
    r = self.session.request(verb, path, params=params, data=ebody)
  File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 461, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 394, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/lib/python3.5/site-packages/requests/models.py", line 295, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3.5/site-packages/requests/models.py", line 355, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL 'dev/auth/token/obtain/': No schema supplied. Perhaps you meant http://dev/auth/token/obtain/?

There is no word about missing a token in the configuration file. It's simply user unfriendly. In ideal world, the client should be able to perform the Kerberos authentication. If you do not plan to implement it, then please make the error message obvious that a user needs to obtain and store the token in the configuration file manually.

Comment 9 chuzhang 2017-08-03 09:55:01 UTC
(In reply to Petr Pisar from comment #8)
> (In reply to chuzhang from comment #7)
> > @Petr Ok. I got it. But for this problem I am not the proper one to help to
> > answer as this is fedora's instance, not owned by my team. It is using
> > different authentication backend, maybe Ralph Bean's team can help to answer
> > it.
> 
> That means I cannot test this bug report. Please test it yourself and act
> accordingly.

I was assuming you had already resolved the token issue in pdc_client usage, you can get token and configure it in config file just as Stano mentions.
And your problem is you can not use curl to obtain the token, I am assuming curl --negotiate is for Kerberos authentication(our internal server is using Kerberos), but you were getting token from a fedora server which is not using Kerberos authentication backend. So if you want to resolve this issue you can reach out to our fedora team mates.

> 
> Nevertheless, I'm not content with the pdc client behavior. If I remove the
> ~/.config/pdc/client_config.json file, I get this back trace:
> 
> $ pdc -s dev rpm list --name perl
> Traceback (most recent call last):
>   File "/usr/bin/pdc", line 22, in <module>
>     runner.run()
>   File "/usr/lib/python3.5/site-packages/pdc_client/runner.py", line 159, in
> run
>     ssl_verify=ssl_verify)
>   File "/usr/lib/python3.5/site-packages/pdc_client/__init__.py", line 153,
> in __init__
>     token = self.obtain_token()
>   File "/usr/lib/python3.5/site-packages/pdc_client/__init__.py", line 167,
> in obtain_token
>     return self.auth[end_point]._()['token']
>   File "/usr/lib/python3.5/site-packages/beanbag/namespace.py", line 131, in
> fn
>     *args, **kwargs)
>   File "/usr/lib/python3.5/site-packages/beanbag/url_v1.py", line 102, in
> call
>     return self.make_request(path, verb, kwargs, body)
>   File "/usr/lib/python3.5/site-packages/beanbag/url_v1.py", line 151, in
> make_request
>     r = self.session.request(verb, path, params=params, data=ebody)
>   File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 461, in
> request
>     prep = self.prepare_request(req)
>   File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 394, in
> prepare_request
>     hooks=merge_hooks(request.hooks, self.hooks),
>   File "/usr/lib/python3.5/site-packages/requests/models.py", line 295, in
> prepare
>     self.prepare_url(url, params)
>   File "/usr/lib/python3.5/site-packages/requests/models.py", line 355, in
> prepare_url
>     raise MissingSchema(error)
> requests.exceptions.MissingSchema: Invalid URL 'dev/auth/token/obtain/': No
> schema supplied. Perhaps you meant http://dev/auth/token/obtain/?
> 
> There is no word about missing a token in the configuration file. It's
> simply user unfriendly. In ideal world, the client should be able to perform
> the Kerberos authentication. If you do not plan to implement it, then please
> make the error message obvious that a user needs to obtain and store the
> token in the configuration file manually.

Actually even if we do not configure token in the file, we can still access PDC internal server(which uses Kerberos backend) if you already running kinit. So we haven't emphasis that you should configure token in config file. Even we haven't emphasis the existence for the configure file. 
But for your case, it is using FAS, maybe we need to tell the user there is the configuration file and you should configure token in this file. Is that what you want us to improve or make the tool user friendly? Please correct me if my understanding is wrong.

Comment 10 Petr Pisar 2017-08-03 12:00:34 UTC
(In reply to chuzhang from comment #9)
> I was assuming you had already resolved the token issue in pdc_client usage,
> you can get token and configure it in config file just as Stano mentions.

It worked when when I wrote the comment #3. I don't remember how I did it a year ago. I used the curl instructions on the web interface and it does not work for me now. I used the token I saw in my web browser and it also does not work now.

> And your problem is you can not use curl to obtain the token, I am assuming
> curl --negotiate is for Kerberos authentication(our internal server is using
> Kerberos), but you were getting token from a fedora server which is not
> using Kerberos authentication backend. So if you want to resolve this issue
> you can reach out to our fedora team mates.
> 
I think I know what you talk about. Module Build Service accesses Fedora's PDC and it uses (or used) some strange way for obtaining the token (running a local HTTP server, spawning a web browser to follow strange HTTP redirection path full of JavaScript client side code). I totally disagree with this approach and I don't see any interest in debugging this issue. I'm sorry.

> Actually even if we do not configure token in the file, we can still access
> PDC internal server(which uses Kerberos backend) if you already running
> kinit.

What is PDC internal server URL? Stock /etc/pdc.d/fedora.json delivered with pdc-client-1.2.0-3.fc25.noarch does not know it. Moreover I expect that stock Fedora package configuration does work with Fedora instance. Instead I get a meaningless back trace.

> But for your case, it is using FAS, maybe we need to tell the user there is
> the configuration file and you should configure token in this file. Is that
> what you want us to improve or make the tool user friendly?

If this is the right way how to use the pdc client against Fedora instance, then yes, document the procedure.

Comment 11 chuzhang 2017-08-09 08:10:40 UTC
(In reply to Petr Pisar from comment #10)
> (In reply to chuzhang from comment #9)
> > I was assuming you had already resolved the token issue in pdc_client usage,
> > you can get token and configure it in config file just as Stano mentions.
> 
> It worked when when I wrote the comment #3. I don't remember how I did it a
> year ago. I used the curl instructions on the web interface and it does not
> work for me now. I used the token I saw in my web browser and it also does
> not work now.
This is fedora instance, so i don't have much knowledge for why it happens, sorry about that.
> 
> > And your problem is you can not use curl to obtain the token, I am assuming
> > curl --negotiate is for Kerberos authentication(our internal server is using
> > Kerberos), but you were getting token from a fedora server which is not
> > using Kerberos authentication backend. So if you want to resolve this issue
> > you can reach out to our fedora team mates.
> > 
> I think I know what you talk about. Module Build Service accesses Fedora's
> PDC and it uses (or used) some strange way for obtaining the token (running
> a local HTTP server, spawning a web browser to follow strange HTTP
> redirection path full of JavaScript client side code). I totally disagree
> with this approach and I don't see any interest in debugging this issue. I'm
> sorry.
> 
> > Actually even if we do not configure token in the file, we can still access
> > PDC internal server(which uses Kerberos backend) if you already running
> > kinit.
> 
> What is PDC internal server URL? Stock /etc/pdc.d/fedora.json delivered with
> pdc-client-1.2.0-3.fc25.noarch does not know it. Moreover I expect that
> stock Fedora package configuration does work with Fedora instance. Instead I
> get a meaningless back trace.
> 
These are PDC's internal stage and product server
https://pdc.stage.engineering.redhat.com
https://pdc.engineering.redhat.com
At least, you can read them.

> > But for your case, it is using FAS, maybe we need to tell the user there is
> > the configuration file and you should configure token in this file. Is that
> > what you want us to improve or make the tool user friendly?
> 
> If this is the right way how to use the pdc client against Fedora instance,
> then yes, document the procedure.
Will add a new task for it.

I am still curious about which instance you should choose, if the data you needed can be found in PDC internal server, surely I can help, but if it is in fedora instance, as what i said, we should ask fedora team mates.

Comment 12 Fedora End Of Life 2017-11-16 18:46:45 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 13 Fedora End Of Life 2017-12-12 10:52:41 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.