Bug 1797719
| Summary: | ipa trust-add fetch domain failure when selinux enforcing | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Scott Poore <spoore> |
| Component: | freeipa | Assignee: | IPA Maintainers <ipa-maint> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | medium | Docs Contact: | |
| Priority: | high | ||
| Version: | 32 | CC: | abokovoy, contribs, dwalsh, fcami, frenaud, ftrivino, grepl.miroslav, ipa-maint, jcholast, jhrozek, lvrabec, mhjacks, plautrba, pvoborni, rcritten, rharwood, ssorce, toms, twoerner, vmojzis, zpytela |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | freeipa-4.9.2-2.fc33 freeipa-4.9.2-2.fc32 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-02-24 20:40:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi Scott, The AVC denials are the common dontaudited ones so likely not related. Could you also check journal for USER_AVC entries? Did this started to happen with some particular version of ipa, kerberos, samba, or other packages? I see this:
[root@fedora-0 ~]# journalctl -xel --since 09:27:48 |grep AVC
Feb 04 09:27:49 fedora-0.example.test audit[1676]: AVC avc: denied { noatsecure } for pid=1676 comm="oddjobd" scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0
Feb 04 09:27:49 fedora-0.example.test audit[1676]: AVC avc: denied { rlimitinh } for pid=1676 comm="com.redhat.idm." scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0
Feb 04 09:27:49 fedora-0.example.test audit[1676]: AVC avc: denied { siginh } for pid=1676 comm="com.redhat.idm." scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0
Feb 04 09:27:51 fedora-0.example.test audit[1679]: AVC avc: denied { read } for pid=1679 comm="ipa-getkeytab" name="config" dev="dm-0" ino=397052 scontext=system_u:system_r:ipa_helper_t:s0 tcontext=system_u:object_r:selinux_config_t:s0 tclass=file permissive=0
The last one there being more interesting.
[root@fedora-0 ~]# find / -inum 397052
/etc/selinux/config
But, I'm not seeing that every time. In fact, the first time I ran it today was the only time I saw it. I just reran trust-del/trust-add about 10 times and didn't see it.
Is there some other way to check logs that would help? or set higher log level?
Scott, The fourth AVC is also a common dontaudited one. I am afraid we cannot get more out of audit. Is it possible to make the ipa command more verbose instead? In SELinux permissive, there are no AVC reported either? This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle. Changing version to 32. Zdenek,
I don't think I can see any USER_AVC entries in the journal. All I see are the same AVCs I listed above. Is there another way I could search for journal USER_AVCs or other selinux messages?
When I run it in permissive mode, I see the same AVCs also and no more. Anything else I can check?
As for more verbose/debug output from ipa command:
[root@fedora-0 ~]# SINCE=$(date +%H:%M:%S); ipa trust-del adcs.test; sleep 1; echo Secret123|ipa trust-add adcs.test --admin Administrator --password
ipa: ERROR: adcs.test: trust not found
ipa: ERROR: error on server 'fedora-0.example.test': Fetching domains from trusted forest failed. See details in the error_log
[root@fedora-0 ~]# SINCE=$(date +%H:%M:%S); ipa trust-del adcs.test; sleep 1; echo Secret123|ipa -vvv trust-add adcs.test --admin Administrator --password
-------------------------
Deleted trust "adcs.test"
-------------------------
ipa: INFO: Request: {
"id": 0,
"method": "trust_add/1",
"params": [
[
"adcs.test"
],
{
"realm_admin": "Administrator",
"realm_passwd": "Secret123",
"version": "2.235"
}
]
}
send: b'POST /ipa/session/json HTTP/1.1\r\nHost: fedora-0.example.test\r\nAccept-Encoding: gzip\r\nAccept-Language: en-us\r\nReferer: https://fedora-0.example.test/ipa/xml\r\nCookie: ipa_session=MagBearerToken=%2bJeNk3TLstzkWoJ4Y5aoeOVhe8KSGSZ%2fiI%2bYTkPhfo3jzsqByeHnCnpAmAPhcANpkwsKLf5gBAzmDvk5zsLfPSZ%2bHEZbK7z7bZ0itgFrr1qiYin1CmXCCDxnd0TuDosQFHYY%2bdctuspONFMmyAoOdXIZeyXHF09hrk07WvkLIwU1SOgSaGXJ51jTEjOFPvXZ;\r\nAccept-Encoding: gzip\r\nUser-Agent: Python-xmlrpc/3.8\r\nContent-Type: application/json\r\nContent-Length: 248\r\n\r\n'
send: b'{\n "id": 0,\n "method": "trust_add/1",\n "params": [\n [\n "adcs.test"\n ],\n {\n "realm_admin": "Administrator",\n "realm_passwd": "Secret123",\n "version": "2.235"\n }\n ]\n}'
reply: 'HTTP/1.1 200 Success\r\n'
header: Date: Wed, 12 Feb 2020 17:08:58 GMT
header: Server: Apache/2.4.41 (Fedora) OpenSSL/1.1.1d mod_wsgi/4.6.8 Python/3.8 mod_auth_gssapi/1.6.1
header: X-Frame-Options: DENY
header: Content-Security-Policy: frame-ancestors 'none'
header: Cache-Control: no-cache, private
header: Vary: Accept-Encoding
header: Content-Encoding: gzip
header: Transfer-Encoding: chunked
header: Content-Type: application/json; charset=utf-8
body: b'{"result": null, "error": {"code": 906, "message": "error on server \'fedora-0.example.test\': Fetching domains from trusted forest failed. See details in the error_log", "data": {"server": "fedora-0.example.test", "error": "Fetching domains from trusted forest failed. See details in the error_log"}, "name": "ServerCommandError"}, "id": 0, "principal": "admin", "version": "4.8.4"}'
ipa: INFO: Response: {
"error": {
"code": 906,
"data": {
"error": "Fetching domains from trusted forest failed. See details in the error_log",
"server": "fedora-0.example.test"
},
"message": "error on server 'fedora-0.example.test': Fetching domains from trusted forest failed. See details in the error_log",
"name": "ServerCommandError"
},
"id": 0,
"principal": "admin",
"result": null,
"version": "4.8.4"
}
ipa: ERROR: error on server 'fedora-0.example.test': Fetching domains from trusted forest failed. See details in the error_log
And for info if needed later (httpd/error_log):
[Wed Feb 12 12:09:02.571411 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ipa: ERROR: Helper fetch_domains was called for forest adcs.test, return code is 1
[Wed Feb 12 12:09:02.572235 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ipa: ERROR: Standard output from the helper:
[Wed Feb 12 12:09:02.572257 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ---
[Wed Feb 12 12:09:02.572268 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422]
[Wed Feb 12 12:09:02.572332 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ipa: ERROR: Error output from the helper:
[Wed Feb 12 12:09:02.572345 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] Traceback (most recent call last):
[Wed Feb 12 12:09:02.572356 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/lib/python3.8/site-packages/ipaserver/dcerpc.py", line 847, in __gen_lsa_connection
[Wed Feb 12 12:09:02.572367 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] result = lsa.lsarpc(binding, self.parm, self.creds)
[Wed Feb 12 12:09:02.572377 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] samba.NTSTATUSError: (3221225485, 'An invalid parameter was passed to a service or function.')
[Wed Feb 12 12:09:02.572388 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422]
[Wed Feb 12 12:09:02.572398 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] During handling of the above exception, another exception occurred:
[Wed Feb 12 12:09:02.572409 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422]
[Wed Feb 12 12:09:02.572437 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] Traceback (most recent call last):
[Wed Feb 12 12:09:02.572448 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/libexec/ipa/oddjob/com.redhat.idm.trust-fetch-domains", line 313, in <module>
[Wed Feb 12 12:09:02.572458 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] domains = dcerpc.fetch_domains(
[Wed Feb 12 12:09:02.572468 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/lib/python3.8/site-packages/ipaserver/dcerpc.py", line 1519, in fetch_domains
[Wed Feb 12 12:09:02.572478 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] domains = communicate(td)
[Wed Feb 12 12:09:02.572489 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/lib/python3.8/site-packages/ipaserver/dcerpc.py", line 1479, in communicate
[Wed Feb 12 12:09:02.572499 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] td.init_lsa_pipe(td.info['dc'])
[Wed Feb 12 12:09:02.572509 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/lib/python3.8/site-packages/ipaserver/dcerpc.py", line 871, in init_lsa_pipe
[Wed Feb 12 12:09:02.572520 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] self._pipe = self.__gen_lsa_connection(binding)
[Wed Feb 12 12:09:02.572549 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] File "/usr/lib/python3.8/site-packages/ipaserver/dcerpc.py", line 850, in __gen_lsa_connection
[Wed Feb 12 12:09:02.572575 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] raise assess_dcerpc_error(e)
[Wed Feb 12 12:09:02.572586 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ipalib.errors.RemoteRetrieveError: CIFS server communication error: code "3221225485", message "An invalid parameter was passed to a service or function." (both may be "None")
[Wed Feb 12 12:09:02.572596 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] --
[Wed Feb 12 12:09:02.572607 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422]
[Wed Feb 12 12:09:02.572852 2020] [wsgi:error] [pid 937:tid 139775378171648] [remote 192.168.122.100:33422] ipa: INFO: [jsonserver_session] admin: trust_add/1('adcs.test', realm_admin='Administrator', realm_passwd='********', version='2.235'): ServerCommandError
Scott, This is a list of steps to do: ================== # remove syscall auditing suppression auditctl -d never,task # enable full path auditing auditctl -w /etc/shadow -p w # rebuild the policy with temporarily removing dontaudit rules semodule -DB # possibly make the system permissive or not, you can get different results setenforce 0 ================== and then after reproducing check both the audit log and journal. # changing back to the previous state setenforce 1 semodule -B auditctl -W /etc/shadow -p w Off the ipa logs, is it possible to isolate the library/system call which fails? Please let me know if additional information is required. I have the same issue on a fresh install of freeipa on Fedora 32. ipa trust-fetch-domains fails after adding trust with ipa trust-add. After setting selinux to permissive, it works immediately without issues. Please let me know where I can find the logs that are of interest to you. With SELinux set to permissive, no AVC's appear when running ipa trust-fetch-domains. Interestingly enough, I cannot get the problem to occur after enabling enforcing mode in SELinux again. [root@nis01 toms]# setenforce 1 [root@nis01 toms]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: permissive Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 32 [root@nis01 toms]# ipa trust-fetch-domains Realm name: sdktest.local ---------------------------------------------------------------------------------------- List of trust domains successfully refreshed. Use trustdomain-find command to list them. ---------------------------------------------------------------------------------------- ---------------------------- Number of entries returned 0 ---------------------------- [root@nis01 toms]# ipa trust-del sdktest.local ----------------------------- Deleted trust "sdktest.local" ----------------------------- [root@nis01 toms]# ipa trust ipa: ERROR: unknown command 'trust' [root@nis01 toms]# ipa trust-find ---------------- 0 trusts matched ---------------- ---------------------------- Number of entries returned 0 ---------------------------- [root@nis01 toms]# ipa trust-add sdktest.local Active Directory domain administrator: Administrator Active Directory domain administrator's password: ------------------------------------------------------ Added Active Directory trust for realm "sdktest.local" ------------------------------------------------------ Realm name: sdktest.local Domain NetBIOS name: SDKTEST Domain Security Identifier: S-1-5-21-1506927830-4214915980-971779718 Trust direction: Trusting forest Trust type: Active Directory domain Trust status: Established and verified [root@nis01 toms]# ipa trust-fetch-domains sdktest.local ---------------------------------------------------------------------------------------- List of trust domains successfully refreshed. Use trustdomain-find command to list them. ---------------------------------------------------------------------------------------- ---------------------------- Number of entries returned 0 ---------------------------- To double check, I ended up deleting the trust again, setting selinux to enforcing in the config, rebooting, and adding the trust again. I am unable to reproduce the issue after succesfully adding the trust once. I have not tried this with alternate domains. François, Will this bz be resolved in freeipa-selinux? If yes, the component should change. Zdenek, I've changed the component for now and will let you know soon. Fixed upstream:
master:
add58fb selinux: modify policy to allow one-way trust
ipa-4-8:
https://pagure.io/freeipa/c/40b69f2739590a3c954b76e88e557a277129819f
ipa-4-9:
https://pagure.io/freeipa/c/952b6bdcceda9f460e17075404084f1f3ddb5eaa
FEDORA-2021-9ead482997 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2021-9ead482997 FEDORA-2021-a53fdfcaeb has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-a53fdfcaeb FEDORA-2021-a53fdfcaeb has been pushed to the Fedora 33 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-a53fdfcaeb` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-a53fdfcaeb See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2021-9ead482997 has been pushed to the Fedora 32 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-9ead482997` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-9ead482997 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2021-a53fdfcaeb has been pushed to the Fedora 33 stable repository. If problem still persists, please make note of it in this bug report. FEDORA-2021-9ead482997 has been pushed to the Fedora 32 stable repository. If problem still persists, please make note of it in this bug report. |
Description of problem: when I run trust-add with SELinux in enforcing mode, I get an error: [root@fedora-0 ~]# echo Secret123| ipa trust-add adcs.test --admin Administrator --password ipa: ERROR: error on server 'fedora-0.example.test': Fetching domains from trusted forest failed. See details in the error_log From error_log: [Mon Feb 03 11:44:42.544594 2020] [wsgi:error] [pid 6831:tid 140492134995712] [remote 192.168.122.100:47994] ipalib.errors.RemoteRetrieveError: CIFS server communication error: code "3221225485", message "An invalid parameter was passed to a service or function." (both may be "None") But, I don't see an AVC. [root@fedora-0 httpd]# ausearch -m avc -ts recent When I disable dontaudit, I see: # ausearch -m avc -ts recent ---- time->Mon Feb 3 11:52:25 2020 type=AVC msg=audit(1580748745.535:428): avc: denied { noatsecure } for pid=7604 comm="oddjobd" scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0 ---- time->Mon Feb 3 11:52:25 2020 type=AVC msg=audit(1580748745.536:429): avc: denied { rlimitinh } for pid=7604 comm="com.redhat.idm." scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0 ---- time->Mon Feb 3 11:52:25 2020 type=AVC msg=audit(1580748745.536:430): avc: denied { siginh } for pid=7604 comm="com.redhat.idm." scontext=system_u:system_r:oddjob_t:s0-s0:c0.c1023 tcontext=system_u:system_r:ipa_helper_t:s0 tclass=process permissive=0 Version-Release number of selected component (if applicable): # rpm -q freeipa-server krb5-server samba selinux-policy freeipa-server-4.8.4-6.fc32.x86_64 krb5-server-1.18-0.beta2.1.fc32.x86_64 samba-4.12.0-0.1.rc1.fc32.x86_64 selinux-policy-3.14.5-21.fc32.noarch How reproducible: Unknown but, I can consistently reproduce by deleting/adding trust when SELinux in enforcing mode. Steps to Reproduce: 1. ipa-server-install 2. add dns forwarder and conditional forwarder on AD server if necessary 3. kinit admin 4. ipa trust-add ad.domain --admin Administrator --password Actual results: Errors as above. If I put SELinux in Permissive mode, it works. Expected results: Works in Enforcing mode Additional info: