Bug 1327697

Summary: schannel connections fail to the DC because netlogon credentials are not updated correctly
Product: Red Hat Enterprise Linux 6 Reporter: Orion Poplawski <orion>
Component: sambaAssignee: Andreas Schneider <asn>
Status: CLOSED ERRATA QA Contact: Robin Hack <rhack>
Severity: high Docs Contact:
Priority: high    
Version: 6.7CC: afarley, alden.1, asn, baumanmo, bloch, Colin.Simpson, gdeschner, grundblom, jarrpa, kesquires, lmiksik, lpol, magnus.moren, marvin24, maryinman, metze, michal.bruncko, mkolaja, mvermaes, oirraza, rhack, rich, salmy, sbose, ssn, thomas.jarosch, tscherf, vorpal
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: samba-3.6.23-34.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1329523 1329960 1329961 1329962 1329964 1329966 1329967 (view as bug list) Environment:
Last Closed: 2017-03-21 10:16:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1269194, 1329523, 1329960, 1329961, 1329962, 1329964, 1329966, 1329967    
Attachments:
Description Flags
abrt report none

Description Orion Poplawski 2016-04-15 16:08:53 UTC 
Description of problem:

smbclient connections with passwords fail with:

# smbclient -U USER -L earth
Enter USER's password:
Got challenge flags:
Got NTLMSSP neg_flags=0x60898215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_TARGET_INFO
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
NTLMSSP: Set final flags:
Got NTLMSSP neg_flags=0x60088215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
NTLMSSP Sign/Seal - Initialising with flags:
Got NTLMSSP neg_flags=0x60088215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
SPNEGO login failed: Access denied
session setup failed: NT_STATUS_ACCESS_DENIED

Server:
[2016/04/15 09:55:38.693594,  4] libsmb/namequery_dc.c:146(ads_dc_name)
  ads_dc_name: using server='AD-OFFICE01.AD.DOMAIN.COM' IP=X.X.X.X
[2016/04/15 09:55:38.693841,  3] libsmb/cliconnect.c:3183(cli_start_connection)
  Connecting to host=AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.693938,  3] lib/util_sock.c:766(open_socket_out_send)
  Connecting to X.X.X.X at port 445
[2016/04/15 09:55:38.694697,  5] lib/util_sock.c:165(print_socket_options)
  Socket options:
        SO_KEEPALIVE = 0
        SO_REUSEADDR = 0
        SO_BROADCAST = 0
        TCP_NODELAY = 1
        TCP_KEEPCNT = 9
        TCP_KEEPIDLE = 7200
        TCP_KEEPINTVL = 75
        IPTOS_LOWDELAY = 0
        IPTOS_THROUGHPUT = 0
        SO_REUSEPORT = 0
        SO_SNDBUF = 16384
        SO_RCVBUF = 16384
        SO_SNDLOWAT = 1
        SO_RCVLOWAT = 1
        SO_SNDTIMEO = 0
        SO_RCVTIMEO = 0
        TCP_QUICKACK = 1
[2016/04/15 09:55:38.698405,  5] rpc_client/cli_pipe.c:1998(rpc_pipe_bind_send)
  Bind RPC Pipe: host AD-OFFICE01.AD.DOMAIN.COM auth_type 0, auth_level 1
[2016/04/15 09:55:38.698527,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.699272,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 52
[2016/04/15 09:55:38.699390,  5] rpc_client/cli_pipe.c:1866(check_bind_response)
  check_bind_response: accepted!
[2016/04/15 09:55:38.699546,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.700384,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 20
[2016/04/15 09:55:38.700602,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.702070,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 24
[2016/04/15 09:55:38.702184,  5] rpc_client/cli_netlogon.c:148(rpccli_netlogon_setup_creds)
  rpccli_netlogon_setup_creds: server AD-OFFICE01.AD.DOMAIN.COM credential chain established.
[2016/04/15 09:55:38.702742,  5] rpc_client/cli_pipe.c:1998(rpc_pipe_bind_send)
  Bind RPC Pipe: host AD-OFFICE01.AD.DOMAIN.COM auth_type 68, auth_level 6
[2016/04/15 09:55:38.702841,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.703622,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 72
[2016/04/15 09:55:38.703712,  5] rpc_client/cli_pipe.c:1866(check_bind_response)
  check_bind_response: accepted!
[2016/04/15 09:55:38.703762,  5] ../libcli/auth/credentials.c:108(netlogon_creds_step)
        seed        48c92a03:99ed1e33
[2016/04/15 09:55:38.703809,  5] ../libcli/auth/credentials.c:113(netlogon_creds_step)
        seed+time   9fda38ff:99ed1e33
[2016/04/15 09:55:38.703904,  5] ../libcli/auth/credentials.c:118(netlogon_creds_step)
        CLIENT      51d94044:5a7df281
[2016/04/15 09:55:38.703950,  5] ../libcli/auth/credentials.c:124(netlogon_creds_step)
        seed+time+1 9fda3900:99ed1e33
[2016/04/15 09:55:38.704026,  5] ../libcli/auth/credentials.c:129(netlogon_creds_step)
        SERVER      20492bd2:2705c8c6
[2016/04/15 09:55:38.704135,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.704992,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 104
[2016/04/15 09:55:38.705664,  5] ../libcli/auth/credentials.c:108(netlogon_creds_step)
        seed        48c92a03:99ed1e33
[2016/04/15 09:55:38.705733,  5] ../libcli/auth/credentials.c:113(netlogon_creds_step)
        seed+time   9fda38ff:99ed1e33
[2016/04/15 09:55:38.705817,  5] ../libcli/auth/credentials.c:118(netlogon_creds_step)
        CLIENT      51d94044:5a7df281
[2016/04/15 09:55:38.705866,  5] ../libcli/auth/credentials.c:124(netlogon_creds_step)
        seed+time+1 9fda3900:99ed1e33
[2016/04/15 09:55:38.705948,  5] ../libcli/auth/credentials.c:129(netlogon_creds_step)
        SERVER      20492bd2:2705c8c6
[2016/04/15 09:55:38.706077,  5] rpc_client/cli_pipe.c:842(rpc_api_pipe_send)
  rpc_api_pipe: host AD-OFFICE01.AD.DOMAIN.COM
[2016/04/15 09:55:38.706936,  5] rpc_client/cli_pipe.c:98(rpc_read_send)
  rpc_read_send: data_to_read: 104
[2016/04/15 09:55:38.707088,  2] ../libcli/auth/credentials.c:289(netlogon_creds_client_check)
  credentials check failed
[2016/04/15 09:55:38.707145,  0] rpc_client/cli_netlogon.c:623(rpccli_netlogon_sam_network_logon)
  rpccli_netlogon_sam_network_logon: credentials chain check failed
[2016/04/15 09:55:38.708373,  0] auth/auth_domain.c:331(domain_client_validate)
  domain_client_validate: unable to validate password for user USER in domain DOMAIN to Domain controller AD-OFFICE01.AD.DOMAIN.COM. Error was NT_STATUS_ACCESS_DENIED.
[2016/04/15 09:55:38.709297,  5] auth/auth.c:281(check_ntlm_password)
  check_ntlm_password: winbind authentication for user [USER] FAILED with error NT_STATUS_ACCESS_DENIED
[2016/04/15 09:55:38.709370,  2] auth/auth.c:330(check_ntlm_password)
  check_ntlm_password:  Authentication for user [USER] -> [USER] FAILED with error NT_STATUS_ACCESS_DENIED
[2016/04/15 09:55:38.709439,  3] smbd/error.c:81(error_packet_set)
  error packet at smbd/sesssetup.c(124) cmd=115 (SMBsesssetupX) NT_STATUS_ACCESS_DENIED



Version-Release number of selected component (if applicable):
samba-3.6.23-30.el6_7.x86_64
samba-4.2.10-6.el7_2.x86_64
Comment 2 Orion Poplawski 2016-04-15 16:12:32 UTC 
Starting winbind fixes this.  Apologies.
Comment 3 Orion Poplawski 2016-04-15 16:23:03 UTC 
Actually, it doesn't.  I do get farther:

# smbclient -U USER -L SERVER
Enter USER's  password:
Domain=[DOMAIN] OS=[Unix] Server=[Samba 3.6.23-30.el6_7]

        Sharename       Type      Comment
        ---------       ----      -------
        homes           Disk      Home Directories
        netlogon        Disk      Network Logon Service
        web             Disk      NWRA Boulder web documents
        ftp             Disk      NWRA Boulder ftp file repository
        data            Disk      Data drives
        home            Disk      Home dir
        local           Disk      Local dir
        scans           Disk      Scanned Documents
        IPC$            IPC       IPC Service (NWRA Boulder Network Server)
        user            Disk      Home Directories
session setup failed: NT_STATUS_ACCESS_DENIED
NetBIOS over TCP disabled -- no workgroup available

Then I get an abrt crash report for winbind on the server

# smbclient -U USER -L SERVER
Enter USER's password:
session setup failed: NT_STATUS_ACCESS_DENIED
Comment 4 Orion Poplawski 2016-04-15 16:26:51 UTC 
Created attachment 1147713 [details]
abrt report
Comment 5 Andreas Schneider 2016-04-15 16:40:56 UTC 
We are already aware of the issue and working on it.
Comment 6 Glen Rundblom 2016-04-19 14:34:45 UTC 
I am also having this issue with Scientific Linux 6.7 and upgraded last Wednesday across our systems. I found that the problem lies in the netbios name resolution (nmb). If the user specifies the fully qualified domain name of the active directory, they are granted access, otherwise if they use the netbios name of the domain, I will see the "NT_STATUS_ACCESS_DENIED" message. 

I also found that domain joined windows boxes are able to access the samba shares, but I believe that is because they have a current kerberos token.

So, ad.domain.local\username - works
localdomain\username - does not work

I hope this helps,
-Glen
Comment 7 Glen Rundblom 2016-04-19 15:53:17 UTC 
(In reply to Glen Rundblom from comment #6)
> I am also having this issue with Scientific Linux 6.7 and upgraded last
> Wednesday across our systems. I found that the problem lies in the netbios
> name resolution (nmb). If the user specifies the fully qualified domain name
> of the active directory, they are granted access, otherwise if they use the
> netbios name of the domain, I will see the "NT_STATUS_ACCESS_DENIED"
> message. 
> 
> I also found that domain joined windows boxes are able to access the samba
> shares, but I believe that is because they have a current kerberos token.
> 
> So, ad.domain.local\username - works
> localdomain\username - does not work
> 
> I hope this helps,
> -Glen

Update: I found that specifying the fully qualified domain name was actually engaging in a kerberos ticket and granting access, I found on an ubuntu machine that installing krb5-user allowed nautilus to get a kerberos ticket, and then allow access (it was not working without it), so I suppose that is the piece that apple/mac clients and Windows clients were doing that I did not notice yet..
Comment 19 BugMasta 2016-05-24 04:41:53 UTC 
Pathetic.
Comment 23 errata-xmlrpc 2017-03-21 10:16:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0662.html