Bug 1266407

Summary: IPA (external users) not able to authenticate using hammer CLI: invalid user / SSO failed
Product: Red Hat Satellite Reporter: Sebastian Hetze <shetze>
Component: Users & RolesAssignee: Oleh Fedorenko <ofedoren>
Status: CLOSED ERRATA QA Contact: Lukáš Hellebrandt <lhellebr>
Severity: high Docs Contact:
Priority: high    
Version: 6.1.2CC: ahumbe, alazik, anrussel, apatel, aruzicka, baptiste.agasse, bdm, bkearney, dhawke, dleroux, ebeaudoi, egolov, erinn.looneytriggs, fperalta, howey.vernon, inecas, info, jalviso, jpazdziora, jsenkyri, karnsing, kevbrown, kkinge, ktordeur, lhellebr, matthew.lesieur, mhulan, mirko.schmidt, momran, mtenheuv, nils.ketelsen, Nishit.J.Patel, oezr, ofedoren, patalber, pcreech, pierre-yves.goubet, rdesouza, rheron, ricardo.arguello, rob.verduijn, sadas, satellite6-bugs, sebastian.haeutle, shetze, sigbjorn, striker, suarora, tbrisker, thadzhie, th, tonay, tonflo, tquill, wpinheir
Target Milestone: 6.13.0Keywords: FieldEngineering, PrioBumpGSS, PrioBumpPM, Reopened
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
URL: http://projects.theforeman.org/issues/11317
Whiteboard:
Fixed In Version: rubygem-hammer_cli_foreman-3.5.1, rubygem-apipie-bindings-0.6.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-03 13:20:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sebastian Hetze 2015-09-25 08:23:27 UTC
Description of problem:

IPA users are not able to authenticate via hammer CLI.
Foreman gives
[I] invalid user
[W] SSO failed

Steps to Reproduce:
1. Configure IPA backend for authentication.
2. Login via Web-UI, assing roles to IPA user (this works)
3. Try something like "hammer organization list"

Actual results:

Invalid username or password

Expected results:

---|----------------|----------------|------------
ID | NAME           | LABEL          | DESCRIPTION
---|----------------|----------------|------------
1  | Default Org    | Default_Org    |            
---|----------------|----------------|------------

Additional info:

Comment 3 Jan Pazdziora 2016-01-05 12:50:23 UTC
I'm not sure what the [I] and [W] in comment 0 mean exactly but Kerberos authentication RFEs are in bug 1181394 (server side) and bug 1264161 (hammer CLI).

Per bug 1181394 comment 5, the external authentication is not handled by API at this point at all so if part of this use-case is password-based authentication, we can track that use-case here.

Comment 4 Jan Pazdziora 2016-01-05 12:51:18 UTC
(In reply to Sebastian Hetze from comment #0)
> 3. Try something like "hammer organization list"
> 
> Actual results:
> 
> Invalid username or password

Will hammer prompt for the login and password or just fail with this error message right away?

Comment 5 Sebastian Hetze 2016-01-05 13:20:11 UTC
[I] and [W] are information and warning messages from /var/log/foreman/production.log

This is still easy to reproduce with Sat 6.1.5

I use .hammer/cli_config.yml for authentication, so no prompt here.

However, if I remove the cli_config.yml hammer prompts for username and password (and gives "Invalid username or password" message for external users).

As you see this is clear text password authentication and has nothing to do with the Kerberos RFE.

Comment 6 Tony 2016-02-04 08:42:27 UTC
Still same issue on Satellite 6.1.6

Comment 9 Ivan Necas 2016-04-26 16:12:57 UTC
I don't think it's realistic to get the changed needed into the code
in 6.2 timeframe.

Here is a solution I propose as a workaround:

```
IPA_SERVER=ipa.example_com

cat <<EOF > /etc/httpd/conf.d/05-foreman-ssl.d/api_ext_auth.conf
<Location /api>
  AuthType Basic
  AuthName "private area"
  AuthBasicProvider PAM
  AuthPAMService $IPA_SERVER
  Require valid-user
</Location>
EOF
service httpd restart
foreman-rake config -- -k authorize_login_delegation_api -v true
```

This will configure the apache to perform the idm integration calls for api requests. The downside of this solution is that in this case, other auth methods (such as internal users) will not work (that's one of the reasons I don't think it would be accepted upstream).

I propose documenting this as a KCS and postponing the bug to 6.3 to have a proper solution together with https://bugzilla.redhat.com/show_bug.cgi?id=1181394

Comment 13 Marek Hulan 2016-09-08 14:25:15 UTC
Just to clarify, I think this issue is not about using kerberos for authentication, as Jan said there are already separate BZs for kerberos authentication.

This BZ is about logging in users which auth source is not internal nor "EXTERNAL", but native LDAP-based one through hammer. It works fine for me with Satellite 6.2.0 and IPA.

Sebastian or Tony, could you please verify with the newer version? In case you still experience the issue, could you change the log level to debug and send a production.log while user tries to login via hammer?

Comment 14 Jan Pazdziora 2016-09-08 14:38:10 UTC
Actually, my understanding is this is exactly about authenticating with Kerberos, but not on WebUI but with hammer.

Comment 15 Marek Hulan 2016-09-08 15:11:22 UTC
I think in comment 5 it's said by the reporter that it has nothing to do with the Kerberos. Hammer should be able to take username and password and do authentication against LDAP auth source. If the auth source is "EXTERNAL" (not LDAP), there's no LDAP server to ask and yes, as a different authentication mechanism we could use Kerberos ticket, but that's a different RFE.

Comment 16 Sebastian Hetze 2016-09-08 15:18:06 UTC
The problem persists with current Sat6.2.1.

Authentication is using IPA as documented in https://access.redhat.com/documentation/en/red-hat-satellite/6.2/paged/server-administration-guide/92-using-identity-management

Changed config.log_level = :debug in
/usr/share/foreman/config/environments/production.rb
and restarted httpd

This is the output in /var/log/foreman/production.log:

2016-09-08 16:56:08 [app] [I] Started GET "/katello/api/organizations" for 192.168.1.10 at 2016-09-08 16:56:08 +0200
2016-09-08 16:56:08 [app] [I] Processing by Katello::Api::V2::OrganizationsController#index as JSON
2016-09-08 16:56:08 [app] [I]   Parameters: {"api_version"=>"v2", "organization"=>{}}
2016-09-08 16:56:08 [app] [W] SSO failed
2016-09-08 16:56:08 [app] [I]   Rendered api/v2/errors/unauthorized.json.rabl within api/v2/layouts/error_layout (1.1ms)
2016-09-08 16:56:08 [app] [I] Filter chain halted as :authorize rendered or redirected
2016-09-08 16:56:08 [app] [I] Completed 401 Unauthorized in 36ms (Views: 4.7ms | ActiveRecord: 2.7ms)

Comment 17 Sebastian Hetze 2016-09-08 15:40:00 UTC
(In reply to Jan Pazdziora from comment #14)
> Actually, my understanding is this is exactly about authenticating with
> Kerberos, but not on WebUI but with hammer.

While it would be nice to have hammer re-use an existing Kerberos ticket obtained previously with IPA authentication, this is not the issue here.

The expectation is, that authentication with hammer works for external IPA users like it does with the WebUI.

Comment 18 Marek Hulan 2016-09-09 08:18:35 UTC
Ok, thanks Sebastian for clarification. I thought you have configured the external authentication using LDAP auth source which would have worked in your case for both WebUI and CLI since you don't use Kerberos tickets.

Comment 19 Bryan Kearney 2016-10-17 20:26:07 UTC
Marek, so is this a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1264161

Comment 20 Marek Hulan 2016-10-18 06:33:42 UTC
Not entirely, this BZ requests support on server side to talk to kerberos and keeping hammer asking for username and password. The authentication would happen through negotiation.

BZ 1264161 asks for using existing kerberos ticket obtained from kinit instead of username and password.

Comment 22 Jan Pazdziora 2016-10-18 06:38:47 UTC
(In reply to Bryan Kearney from comment #19)
> Marek, so is this a dupe of
> https://bugzilla.redhat.com/show_bug.cgi?id=1264161

It might or might not be.

My current understanding is that the request here is to be able to login + password for external users (for example users from IPA/IdM), with no Kerberos involved. The solution would likely involve PAM setup on the server as shown in comment 9, but ideally in such a way that falling back is possible.

For that fall back, some refactoring on the client might be needed and that refactoring might be similar to what bug 1264161 might need.

It shouldn't be treated as duplicate because the test plan / QA effort will be different. The solution however might be very close to / part of the one for that bug.

Comment 23 Bryan Kearney 2016-11-29 15:03:52 UTC
Upstream bug assigned to tstrachota

Comment 26 Brad Buckingham 2017-06-14 12:28:26 UTC
*** Bug 1331689 has been marked as a duplicate of this bug. ***

Comment 28 Brad Buckingham 2018-02-20 18:36:15 UTC
*** Bug 1546718 has been marked as a duplicate of this bug. ***

Comment 30 Ivan Necas 2018-04-05 07:23:06 UTC
*** Bug 1376654 has been marked as a duplicate of this bug. ***

Comment 35 Daniel Leroux 2019-10-10 18:26:40 UTC
Hello Team,

Does anybody know if a fix is in the plans for Sat 6.7?

Comment 40 Bryan Kearney 2020-06-09 15:01:01 UTC
The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Red Hat Technical Support or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 41 Francisco Peralta 2020-06-10 09:03:47 UTC
I think that with more than 20 attached customer cases, this issue should still be considered as to be resolved and prioritized high.
Is there something that could be done to support backporting it from upstream or what are your plans to implement it in a future release?

Thanks in advance for letting us know,
 Cisco.

Comment 42 Bryan Kearney 2020-07-06 14:40:35 UTC
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.

Comment 43 Bryan Kearney 2020-07-06 14:43:27 UTC
Mistakenly closed, this will be coming up in 7.0

Comment 45 Ondřej Ezr 2021-01-05 19:17:21 UTC
I've started to implement #1264161. This should be most likely duplicate even though it is not strictly the same, I believe it covers most requirements - using the kerberos tickets negotiation.

Comment 52 Richard Barrott 2022-02-08 03:08:31 UTC
Hi team, 
any updates on this BZ?  Is it actually a duplicate of 1264161?

Comment 62 Tim Quill 2022-06-09 23:29:48 UTC
Hi Team, 

Do we have a targeted version for this fix? The customer requesting this has asked for an update. 

regards, 

Tim

Comment 77 Lukáš Hellebrandt 2023-01-12 12:19:40 UTC
Hi Adam,

Please take a look at the comments once again - I think your reproducer from comment 75 is wrong in this point: "1) Set up IdM/IPA as LDAP authentication source in Satellite"

I think this BZ is not about auth not working with IdM added as an LDAP external source (that's only mentioned here as a workaround); it's rather about auth not working after following docs article "Configuring External Authentication Using Red Hat Identity Management": https://access.redhat.com/documentation/en-us/red_hat_satellite/6.7/html/administering_red_hat_satellite/chap-red_hat_satellite-administering_red_hat_satellite-configuring_external_authentication#sect-Red_Hat_Satellite-Administering_Red_Hat_Satellite-Configuring_External_Authentication-Using_Identity_Management

I.e. not when adding IdM as a source explicitly but as a part of "satellite-installer --foreman-ipa-authentication=true" workflow.

Do you agree?

Comment 78 Adam Ruzicka 2023-01-12 13:48:24 UTC
Hi,
I can't say I do. Comment #71 states that the scope of this bz should be defined by comments #58 and #59, which talk about having IdM set up as LDAP authentication source in Satellite.

I admit that comments in this BZ point in all sorts of directions, including the one you're describing in #77, but as far as I understand, that would be better covered by whttps://bugzilla.redhat.com/show_bug.cgi?id=1264161

Comment 79 Lukáš Hellebrandt 2023-01-13 14:40:03 UTC
My observations on Sat 6.13:

1) When I add an IdM instance as an authentication resource of type LDAP [0] (Administer -> Authentication Sources -> LDAP -> Create), this works except case (*). I can login in WebUI as well as in Hammer, using username/password from that LDAP. A user is successfully created and I can run `hammer -uuser -ppassword task list`.

2) When I add IdM as an external authentication source, using `satellite-installer --foreman-ipa-authentication=true` workflow [1], only authentication in WebUI works. I can login using either username/password or Kerberos ticket in the WebUI. I can't login using username/password in Hammer.

(*) When I setup both 1) and 2) against the same IdM instance (as a workaround for this bug - to be able to use IdM according to documentation in WebUI and to setup that same IdM as an LDAP source so it works in Hammer), I get the following behavior: When I first login using 2), the user is created and bound to resource 2). When I try to login using the same user in Hammer, i.e. through source 1), the login fails. That's not surprising because it's already bound to another source and Satellite doesn't know it's effectively the same one.

[0] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html/administering_red_hat_satellite/configuring_external_authentication_admin#Using_LDAP_admin
[1] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.12/html/installing_satellite_server_in_a_connected_network_environment/configuring_external_authentication_satellite#Using_FreeIPA_satellite

Comment 80 Lukáš Hellebrandt 2023-01-13 14:51:25 UTC
Looking at the OP and comments here, I believe this BZ is about using username/password external authentication with IdM. Setting up IdM is described in [0] where it is explicitly stated that "External users can now log in to Satellite using their Red Hat Identity Management credentials. They can now choose to either log in to Satellite Server directly using their username and password or take advantage of the configured Kerberos single sign-on and obtain a ticket on their client machine and be logged in automatically.".
For the purpose of this BZ, let's ignore Kerberos since it's been repeatedly stated that this is about username/password authentication.

Looking at comments 0 and 25 and also OP of bug 1331689 which is a duplicate of this bug, I think it's clear that the customer meant using external auth resource (IdM) rather than just using it as a manually added LDAP source (I'm still talking about username/password, [0] doesn't imply Kerberos).

Comment 31 explicitly states that LDAP external source is only workaround.

Looking at titles of most of the attached customer cases (I don't have access to details), I also think this is not about LDAP but about workflow described in [0].

Furthermore, in WebUI, after applying [0], both Kerberos ticket login and username/password login work so I would expect the same in Hammer.

Finally, case (*) from my above comment makes it impossible to use IdM according to documentation, at least in WebUI, while also using that same IdM as an LDAP source to workaround the issue in Hammer.

=> As a QE, looking from the customer's perspective (rather than perspective of some of us who may have misunderstood the issue and this BZ's intention), I think I must FAIL this BZ because it doesn't fix the original issue and wouldn't meet customers expectations.

[0] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.12/html/installing_satellite_server_in_a_connected_network_environment/configuring_external_authentication_satellite#Using_FreeIPA_satellite

Comment 84 Bryan Kearney 2023-01-27 16:00:24 UTC
Upstream bug assigned to ofedoren

Comment 85 Bryan Kearney 2023-01-27 16:00:30 UTC
Upstream bug assigned to ofedoren

Comment 86 Adam Ruzicka 2023-02-21 17:56:28 UTC
Everything should be finally ready here.
- https://github.com/theforeman/puppet-foreman/pull/1108
- https://github.com/theforeman/hammer-cli-foreman/pull/609
- https://github.com/Apipie/apipie-bindings/pull/82

apipie-bindings-0.6.0 containing the fixes is already released in upstream, puppet changes will most likely need to be cherry-picked, I'll defer to Oleh how he wants to deal with hammer-cli-foreman.

Comment 87 Oleh Fedorenko 2023-02-22 10:39:46 UTC
I am going to release hammer 3.5.1 which eventually should lend with Foreman 3.5.x.

Comment 90 Lukáš Hellebrandt 2023-03-07 14:44:14 UTC
Setting this back to ON_QA, a new hammer subcommand `hammer auth login basic-external` has been added.

Comment 91 Lukáš Hellebrandt 2023-03-08 10:34:19 UTC
Verified with Sat 6.13.0 snap 13.0.

# rpm -q rubygem-apipie-bindings
rubygem-apipie-bindings-0.6.0-1.el8sat.noarch
# rpm -q rubygem-hammer_cli_foreman-3.5.1
rubygem-hammer_cli_foreman-3.5.1-1.el8sat.noarch
# satellite-installer --help | grep ipa-authentication
    --foreman-ipa-authentication  Enable configuration for external authentication via IPA (current: true)
    --foreman-ipa-authentication-api  Enable configuration for external authentication via IPA for API (current: true)
# kinit <username>
Password for <username>@<DOMAIN>:
# hammer auth login negotiate
Successfully authenticated using negotiate auth, using the KEYRING principal.
# hammer auth logout
Logged out.
# hammer auth login basic
[Foreman] Username: <username>
[Foreman] Password for <username>: 
Unable to authenticate user <username>
Invalid username or password.
# NOW, USE THE NEW SUBCOMMAND BASIC-EXTERNAL:
# hammer auth login --help | grep external
 basic-external                Authenticate against external source (IPA/PAM) with credentials
# hammer auth login basic-external
[Foreman] Username: <username>
[Foreman] Password for <username>: 
Successfully logged in as '<username>'.
# NOW, ADD SOME ROLES TO THE NEWLY CREATED USER, E.G. IN WEBUI
# hammer organization create --name testorg
Organization created.
# hammer organization list
---|----------------------|----------------------|-------------|---------------------
ID | TITLE                | NAME                 | DESCRIPTION | LABEL               
---|----------------------|----------------------|-------------|---------------------
1  | Default Organization | Default Organization |             | Default_Organization
3  | testorg              | testorg              |             | testorg             
---|----------------------|----------------------|-------------|---------------------

Results: Functionality works, it's possible to get authenticated using IdM-stored username/password in Hammer
Flaws:
1) You need to enable sessions and specifically use login command, credentials saved in ~/.hammer/cli.modules.d/foreman.yml won't work
2) You need to specifically use a new subcommand basic-external, there is no one way to get authenticated with both local and external credentials, as in WebUI
3) Although the new subcommand is shown in Hammer help, documentation is missing
4) You shouldn't have an IdM also added as an LDAP authentication source. If you do and attempt to login as the same user one time using LDAP and then (after enabling IdM through installer) using IdM, e.g. Kerberos ticket (kinit <user>; hammer auth login negotiate), you get: "Failed to log in external user: Username '<username>' already exists in a different authentication source".
This is not documented and should be. Also, for migrating from one to the other, you can just remove all the automatically created users to start again.

Comment 92 Lukáš Hellebrandt 2023-03-08 10:35:48 UTC
@alazik please consider points 3) and 4) for documentation

Comment 93 Adam Lazik 2023-03-08 10:44:07 UTC
Thank you, will do!

I also added doc tracker to links.

Comment 97 errata-xmlrpc 2023-05-03 13:20:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097

Comment 98 Red Hat Bugzilla 2023-09-18 00:11:51 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days