Bug 1970168 - ipa-trust-add fails with "not enough quota"
Summary: ipa-trust-add fails with "not enough quota"
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: freeipa
Version: 34
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: IPA Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1982211 1982212
TreeView+ depends on / blocked
 
Reported: 2021-06-10 00:53 UTC by Robbie Harwood
Modified: 2021-10-29 16:49 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1982211 1982212 (view as bug list)
Environment:
Last Closed: 2021-10-29 16:49:28 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Fedora Pagure freeipa issue 8923 0 None None None 2021-07-14 14:42:55 UTC
Red Hat Issue Tracker FREEIPA-6955 0 None None None 2021-09-28 09:19:06 UTC

Description Robbie Harwood 2021-06-10 00:53:48 UTC
[root@ipaserver ~]# echo "vagrant" | ipa trust-add --type=ad ad.test --admin vagrant --password --two-way=true
ipa: ERROR: CIFS server communication error: code "3221225495", message "{Not Enough Quota} Not enough virtual memory or paging file quota is available to complete the specified operation." (both may be "None")
[root@ipaserver ~]# 

Injecting a stack print, the callsite is:

  File "/usr/share/ipa/wsgi.py", line 59, in application
    return api.Backend.wsgi_dispatch(environ, start_response)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 296, in __call__
    return self.route(environ, start_response)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 308, in route
    return app(environ, start_response)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 917, in __call__
    response = super(jsonserver_session, self).__call__(environ, start_response)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 517, in __call__
    response = super(jsonserver, self).__call__(environ, start_response)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 473, in __call__
    response = self.wsgi_execute(environ)
  File "/usr/lib/python3.9/site-packages/ipaserver/rpcserver.py", line 400, in wsgi_execute
    result = command(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 471, in __call__
    return self.__do_call(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 499, in __do_call
    ret = self.run(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 821, in run
    return self.execute(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipaserver/plugins/trust.py", line 759, in execute
    full_join = self.validate_options(*keys, **options)
  File "/usr/lib/python3.9/site-packages/ipaserver/plugins/trust.py", line 868, in validate_options
    self.trustinstance = ipaserver.dcerpc.TrustDomainJoins(self.api)
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 1742, in __init__
    self.__populate_local_domain()
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 1756, in __populate_local_domain
    ld.retrieve(FQDN)
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 992, in retrieve
    self.init_lsa_pipe(remote_host)
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 891, in init_lsa_pipe
    self._pipe = self.__gen_lsa_connection(binding)                                                                                                                                                                                                                                                                            
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 870, in __gen_lsa_connection
    raise assess_dcerpc_error(e)
  File "/usr/lib/python3.9/site-packages/ipaserver/dcerpc.py", line 179, in assess_dcerpc_error
    traceback.print_stack(file=f)

I've previously captured an strace, but it doesn't seem helpful: https://rharwood.fedorapeople.org/strace

I observe no network traffic on any interface, other than: DNS, STP, SSH, LDAP, and HTTPS (on 443).

Tarball of debug logs with debug level 50: https://rharwood.fedorapeople.org/var_log_samba.tar.gz

Should you reproduce this locally, I have a repo: https://github.com/frozencemetery/ad-testing/ (vagrant up ipaserver, vagrant ssh ipaserver, sudo ./install_trust.sh).

I'm personally at a loss as to how to debug this further.  I don't seem to be able to find where the lsa stuff comes from, and no matter which smbd I attach to with gdb (even all of them), I can't seem to catch the connection.  Is it possible that there is no connection, somehow?

Comment 1 Andreas Schneider 2021-07-14 10:03:57 UTC
Error loading module '/usr/lib64/samba/idmap/sss.so': /usr/lib64/samba/idmap/sss.so: cannot open shared object file: No such file or directory

looks like idmap_sss is not installed and it can't allocate IDs ...

Comment 2 Alexander Bokovoy 2021-07-14 10:05:02 UTC
sssd-winbind-idmap is missing in IPA dependencies, it looks like.

Comment 3 Florence Blanc-Renaud 2021-07-15 14:29:31 UTC
Fixed upstream
master:
https://pagure.io/freeipa/c/24afb10c30577be6092ab699fd7f6eeef9fa62b2

Comment 4 Florence Blanc-Renaud 2021-07-15 16:23:10 UTC
Fixed upstream
ipa-4-9:
https://pagure.io/freeipa/c/1a4f459b81bc77cdf233b65f41d0f76dbb5f2fce

Comment 5 Robbie Harwood 2021-07-15 18:11:05 UTC
I appreciate you looking into this issue.  However, adding the package doesn't fix the issue for me:

[root@ipaserver vagrant]# rpm -qv sssd-winbind-idmap
sssd-winbind-idmap-2.5.2-1.fc34.x86_64
[root@ipaserver vagrant]# kinit admin
Password for admin: 
[root@ipaserver vagrant]# ipa trust-add --type=ad ad.test --admin vagrant --password
Active Directory domain administrator's password: 
ipa: ERROR: CIFS server communication error: code "3221225495", message "{Not Enough Quota} Not enough virtual memory or paging file quota is available to complete the specified operation." (both may be "None")
[root@ipaserver vagrant]# 

Samba logs at level 50: https://rharwood.fedorapeople.org/samba2.tar.gz

I see some things in log.winbindd-idmap - looks like there was trouble allocating a GID, maybe?

Comment 6 Alexander Bokovoy 2021-07-15 18:41:55 UTC
It should not allocate any GID, that's intentional, sss idmap module is read-only.
I'll look at the logs tomorrow.

Comment 7 Robbie Harwood 2021-08-03 19:32:49 UTC
Andreas pointed out that my logs aren't actually at the right level, so here's another attempt: http://rharwood.fedorapeople.org/samba3.tar.gz

Comment 8 Andreas Schneider 2021-08-11 15:41:33 UTC
Can someone from IPA Team please look into this. I dunno where this error is coming from.

When I looked into logs with Robbie we at least that winbind is unable to allocate gids, however this is how Samba is set up. The smb.conf (net conf list) has

idmap config * : range = 0-0

so winbind wont allocate any gid.

Comment 9 Rob Crittenden 2021-08-11 19:24:52 UTC
This configuration appears to be by design: https://github.com/freeipa/freeipa/blob/master/doc/designs/adtrust/samba-domain-member.md

Comment 10 Florence Blanc-Renaud 2021-08-12 14:55:25 UTC
Robbie,
in the config steps found in https://github.com/frozencemetery/ad-testing/ I don't see the forwarders config.
You can try to add the following steps:
- on ipa server: ipa dnsforwardzone-add ad.test --forwarder 192.168.3.2 --forward-policy only
- on ipa server: also add the ipaserver address in /etc/hosts (192.168.3.3 ipaserver.ipa.test) to force using this address (the vagrant image contains 2 IP addresses and I'm afraid this may confuse the system).

Comment 11 Robbie Harwood 2021-08-12 20:35:47 UTC
Thanks for taking a look.  I've made the suggested changes (repo updated accordingly).

The error is different now:

# echo vagrant | ipa trust-add --type=ad ad.test --admin vagrant --password
ipa: ERROR: Cannot find specified domain or server name
# 

I'm unsure what it's trying to resolve - the system can resolve adserver.ad.test just fine (it pings).  Happy to provide more logs but I'm not sure what would be useful: there doesn't appear to be anything new in /var/log and an strace of the `ipa trust-add` didn't show any resolution failures.

Comment 12 Florence Blanc-Renaud 2021-08-13 10:26:21 UTC
Hi Robbie,
in my env it's working if I modify the ipa-server-install command and add --ip-address 192.168.3.3 (in order to force binding only to this address).

Comment 13 Florence Blanc-Renaud 2021-09-28 09:18:21 UTC
@rharwood 
Did you manage to solve the issue with my proposal from #c12?

Comment 14 Robbie Harwood 2021-10-29 16:49:28 UTC
Your proposal allowed it to get further, thanks.


Note You need to log in before you can comment on or make changes to this bug.