Bug 1627764 - master static pod failed to start when LDAP auth is set
Summary: master static pod failed to start when LDAP auth is set
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.10.z
Assignee: Vadim Rutkovsky
QA Contact: Johnny Liu
URL:
Whiteboard:
: 1632982 1633301 1636122 1641083 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-11 12:16 UTC by Johnny Liu
Modified: 2019-04-24 07:32 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: insecure flag was not used for LDAP auth setup Consequence: installation failed since CA for LDAP auth is optional Fix: LDAP auth doesn't use CA if insecure is specified Result: installation proceeds
Clone Of:
Environment:
Last Closed: 2019-04-24 07:32:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
installation log with inventory file embedded (943.62 KB, text/plain)
2018-09-11 12:16 UTC, Johnny Liu
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3662731 0 None None None 2018-10-22 23:07:47 UTC
Red Hat Product Errata RHSA-2018:2709 0 None None None 2018-11-11 16:39:46 UTC

Description Johnny Liu 2018-09-11 12:16:28 UTC
Created attachment 1482345 [details]
installation log with inventory file embedded

Description of problem:

Version-Release number of the following components:
openshift-ansible-3.10.45-1.git.0.5aef941.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. Enable LDAP auth for a fresh install
openshift_master_identity_providers=[{'name': 'LDAP_auth', 'login': 'true', 'challenge': 'true', 'kind': 'LDAPPasswordIdentityProvider', 'attributes': {'id': ['dn'], 'email': ['mail'], 'name': ['uid'], 'preferredUsername': ['uid']}, 'bindDN': '', 'bindPassword': '', 'insecure': 'true', 'url': 'ldap://10.66.147.104:389/ou=People,dc=my-domain,dc=com?uid'}]
2.
3.

Actual results:
master static pod fail to start.
master log:
W0911 11:29:43.451158       1 start_master.go:270] Warning: oauthConfig.identityProvider[0].provider.insecure: Invalid value: true: validating passwords over an insecure connection could allow them to be intercepted, master start will continue.
Invalid MasterConfig /etc/origin/master/master-config.yaml
  oauthConfig.identityProvider[0].provider.ca: Invalid value: "/etc/origin/master/LDAP_auth_ldap_ca.crt": Cannot specify a ca with insecure=true

oauthConfig.identityProvider[0].provider.ca is set to "/etc/origin/master/LDAP_auth_ldap_ca.crt", which conflicts with insecure=true setting, even /etc/origin/master/LDAP_auth_ldap_ca.crt is not existing on the master.

Expected results:
installation should PASS.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Vadim Rutkovsky 2018-09-11 13:08:41 UTC
PR https://github.com/openshift/openshift-ansible/pull/9940 would fix that

Comment 2 Gaoyun Pei 2018-09-26 05:36:52 UTC
All the three kinds of identityProviders involved in https://github.com/openshift/openshift-ansible/pull/9731/ should have similar issue.


Even user didn't specify LDAP/request header/OpenID CA certificate via openshift_master_<identityProviders>_ca or openshift_master_<identityProviders>_ca_file, installer will always set 
ca: /etc/origin/master/<name>_<identityProviders>_ca.crt
in master-config.yaml, which would break the installation.
 

I'm agree with what we did in PR https://github.com/openshift/openshift-ansible/pull/9731/, the path of CA file on masters not allowed to be customized is a better choice.

But I think we should set CA for identityProviders automatically only when these CA certificates were specified in inventory file. Customer could choose not to use CA certificate when using these three kinds of identityProviders.

Comment 3 Vadim Rutkovsky 2018-09-26 09:19:44 UTC
(In reply to Gaoyun Pei from comment #2)
> Even user didn't specify LDAP/request header/OpenID CA certificate via
> openshift_master_<identityProviders>_ca or
> openshift_master_<identityProviders>_ca_file, installer will always set 
> ca: /etc/origin/master/<name>_<identityProviders>_ca.crt
> in master-config.yaml, which would break the installation.

...  

> But I think we should set CA for identityProviders automatically only when
> these CA certificates were specified in inventory file. Customer could
> choose not to use CA certificate when using these three kinds of
> identityProviders.

This file is expected to be present on the host if any of these providers is set. 
Filed https://bugzilla.redhat.com/show_bug.cgi?id=1633137 to describe the new behaviour in the docs

Comment 4 Johnny Liu 2018-09-27 01:53:18 UTC
*** Bug 1633301 has been marked as a duplicate of this bug. ***

Comment 6 Eric Jones 2018-10-03 18:28:10 UTC
Hi Vadim,
Do you mean that the user has to provide a CA or that it will use the default CA on the masters?

I ask because customer case 02191718 has tried using insecure=true and Not providing a CA and still saw the errors listed here.

Comment 7 Vadim Rutkovsky 2018-10-04 17:11:29 UTC
*** Bug 1636122 has been marked as a duplicate of this bug. ***

Comment 8 Vadim Rutkovsky 2018-10-04 17:13:57 UTC
(In reply to Eric Jones from comment #6)
> Hi Vadim,
> Do you mean that the user has to provide a CA or that it will use the
> default CA on the masters?

The use has to provide the CA now, yes. Master's CA won't be used in api server container

> I ask because customer case 02191718 has tried using insecure=true and Not
> providing a CA and still saw the errors listed here.


`insecure=true` is a special case, CA won't be required there. This is implemented in https://github.com/openshift/openshift-ansible/pull/9940

Comment 9 Eric Jones 2018-10-04 21:09:44 UTC
Hi Vadim,

Thanks for responding, so as that code was only merged a few days ago, and is still on official QA for proper errata can you confirm that before that code the CA was still required despite `insecure=true`?

If so, then the best workaround for this issue, till this code releases, is to manually setup LDAP post install, right?

Comment 10 Vadim Rutkovsky 2018-10-04 21:31:45 UTC
(In reply to Eric Jones from comment #9)
> Hi Vadim,
> 
> Thanks for responding, so as that code was only merged a few days ago, and
> is still on official QA for proper errata can you confirm that before that
> code the CA was still required despite `insecure=true`?

Yes, correct. Currently released packages require any CA bundle to specified

> If so, then the best workaround for this issue, till this code releases, is
> to manually setup LDAP post install, right?

Another option is to have openshift_master_ldap_ca_file=/etc/pki/tls/certs/ca-bundle.crt - that would use localhost's CA bundle. Since insecure=true is set it won't actualy be used for cert validation

Comment 11 Vadim Rutkovsky 2018-10-05 08:46:26 UTC
Fix is available in openshift-ansible-3.10.52-1

Comment 12 Eric Jones 2018-10-05 20:19:51 UTC
Note (as this is a public bug):

openshift-ansible-3.10.52-1 is not available at this time.

https://access.redhat.com/downloads/content/rhel---7/x86_64/7093/openshift-ansible/3.10.47-1.git.0.95bc2d2.el7_5/noarch/fd431d51/package

Comment 13 Johnny Liu 2018-10-10 08:39:02 UTC
Verified this bug with openshift v3.10.51 + openshift-ansible-3.10.53-1.git.0.ba2c2ec.el7.noarch, and PASS.

openshift_master_identity_providers=[{'name': 'LDAP_auth', 'login': 'true', 'challenge': 'true', 'kind': 'LDAPPasswordIdentityProvider', 'attributes': {'id': ['dn'], 'email': ['mail'], 'name': ['uid'], 'preferredUsername': ['uid']}, 'bindDN': '', 'bindPassword': '', 'ca': '', 'insecure': 'true', 'url': 'ldap://x.x.x.x:389/ou=People,dc=my-domain,dc=com?uid'}]


Installation is completed successfully.
# cat /etc/origin/master/master-config.yaml
<--snip-->
  identityProviders:
  - challenge: true
    login: true
    mappingMethod: claim
    name: LDAP_auth
    provider:
      apiVersion: v1
      attributes:
        email:
        - mail
        id:
        - dn
        name:
        - uid
        preferredUsername:
        - uid
      bindDN: ''
      bindPassword: ''
      ca: ''
      insecure: true
      kind: LDAPPasswordIdentityProvider
      url: ldap://x.x.x.x:389/ou=People,dc=my-domain,dc=com?uid
<--snip-->

Comment 14 hgomes 2018-10-22 23:08:27 UTC
Added KCS to track this issue.
https://access.redhat.com/solutions/3662731

Comment 15 hgomes 2018-10-22 23:12:50 UTC
This is happening for OCP 3.11 too.

Workaround from https://bugzilla.redhat.com/show_bug.cgi?id=1627764#c13 works.

Comment 16 Alberto Gonzalez de Dios 2018-10-23 08:36:11 UTC
Issue reproduced in my lab with latest packages:

[root@workstation-311 ansible]# rpm -qa | grep openshift-ansible
openshift-ansible-roles-3.11.16-1.git.0.4ac6f81.el7.noarch
openshift-ansible-playbooks-3.11.16-1.git.0.4ac6f81.el7.noarch
openshift-ansible-3.11.16-1.git.0.4ac6f81.el7.noarch
openshift-ansible-docs-3.11.16-1.git.0.4ac6f81.el7.noarch

[root@workstation-311 ansible]# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                                                repo name                                                                                                             status
!rhel-7-server-ansible-2.6-rpms/x86_64                                                 Red Hat Ansible Engine 2.6 RPMs for Red Hat Enterprise Linux 7 Server                                                     11
!rhel-7-server-extras-rpms/x86_64                                                      Red Hat Enterprise Linux 7 Server - Extras (RPMs)                                                                        923
!rhel-7-server-ose-3.11-rpms/x86_64                                                    Red Hat OpenShift Container Platform 3.11 (RPMs)                                                                         228
!rhel-7-server-rpms/7Server/x86_64                                                     Red Hat Enterprise Linux 7 Server (RPMs)                                                                              21.065

[root@sa3-master-0 ~]# docker logs $(docker ps -a --format '{{.ID}}' -f name=^/.*controllers_master-controllers.*$) 2>&1  | tail -2
Invalid MasterConfig /etc/origin/master/master-config.yaml
  oauthConfig.identityProvider[1].provider.ca: Invalid value: "/etc/origin/master/alber_ldap_ldap_ca.crt": could not read file: stat /etc/origin/master/alber_ldap_ldap_ca.crt: no such file or directory

Comment 17 Alberto Gonzalez de Dios 2018-10-23 13:17:12 UTC
*** Bug 1641083 has been marked as a duplicate of this bug. ***

Comment 21 errata-xmlrpc 2018-11-11 16:39:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2709

Comment 22 Vadim Rutkovsky 2018-11-29 20:26:38 UTC
*** Bug 1632982 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.