Bug 1464115

Summary: ipa-dnskeysyncd AVCs during openqa freeipa tests
Product: [Fedora] Fedora Reporter: Menanteau Guy <menantea>
Component: selinux-policyAssignee: Lukas Vrabec <lvrabec>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: abokovoy, dan, dominick.grift, dwalsh, hannsj_uhl, ipa-maint, jcholast, jhrozek, lsm5, lvrabec, mbasti, menantea, mgrepl, normand, plautrba, pmoore, pvoborni, rcritten, slaznick, ssekidde, ssorce, tkrizek
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: selinux-policy-3.13.1-260.14.fc26 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-15 20:11:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1071880, 1430830    
Attachments:
Description Flags
summary of journal log
none
summary of audit log
none
audit log line analysing thru sealert
none
krb4kdc log when kinit problem none

Description Menanteau Guy 2017-06-22 12:57:00 UTC 
Created attachment 1290703 [details]
summary of journal log

During openqa tests exercising different freeipa scenari, I have got intermittent problems with "kinit" command on ppc64le arch. This problems occurs on f26 Beta-1.4 version and last nightly composes.

for example I got ipa-client-install failed when I run command line "realm join" or when I run "join domain" thru  cockpit. It seems the problem oocurs always durring "kinit" command. Note that I got problem also when "kinit" is executed in command line later (outside of ipa-client-install when this one succeed).

In journal and audit log, I found traces about ipa-dnskeysyncd permission problem. see attached summary logs.

I tried to disable selinux (setenforce 0) and all tests passed without any errors. I didn't yet investigate what specific selinux ipa_dnskey policy I have to disable (https://www.mankier.com/8/ipa_dnskey_selinux) but what is puzzling me is that it is an intermittent problem and sometimes even with selinux policy enabled, kinit performs correctly and there is no problem on ipa_dnskey.

failing openqa tests:
server_role_deploy_domain_controller
server_realmd_join_kickstart 
realmd_join_sssd 
realmd_join_cockpit
Comment 1 Menanteau Guy 2017-06-22 12:57:38 UTC 
Created attachment 1290704 [details]
summary of audit log
Comment 2 Menanteau Guy 2017-06-22 12:59:59 UTC 
Created attachment 1290705 [details]
audit log line analysing thru sealert
Comment 3 Martin Bašti 2017-06-22 14:40:06 UTC 
Hello,

ipa-dnskeysyncd is unrelated to kinit, please provide more information about failing kinit in separate BZ (try KRB5_TRACE=/dev/stderr kinit) when issue occurs.

changing component to SELinux as this BZ is related only to AVCs

Thank you
Comment 4 Menanteau Guy 2017-06-23 10:43:32 UTC 
I ran tests with enforcing enabled and got the problem again.
kerberos traces are as follow:

KRB5_TRACE=/dev/stderr kinit admin
[9981] 14982122358.540844: Getting initial credentials for admin
[9981] 14982122358.542648: Sending request (173 bytes) to DOMAIN.LOCAL
[9981] 14982122358.542910: Initiating TCP connection to stream 10.0.2.100:88
[9981] 14982122358.543224: Sending TCP request to stream 10.0.2.100:88
[9981] 14982122368.553502: Sending initial UDP request to dgram 10.0.2.100:88
[9981] 14982122371.556012: Sending retry UDP request to dgram 10.0.2.100:88
[9981] 14982122376.559790: Sending retry UDP request to dgram 10.0.2.100:88
[9981] 14982122385.566022: Terminating TCP connection to stream 10.0.2.100:88
kinit: Cannot contact any KDC for realm 'DOMAIN.LOCAL' while getting initial credentials

note that this kinit run on the server itself, means that TCP and UDP traffic try to be established on the machine itself (ipadd 10.0.2.100) but there is no answer...
Comment 5 Martin Bašti 2017-06-23 10:53:53 UTC 
Do you have any KDC errors on server? :
less /var/log/krb5kdc.log
journalctl -u krb5kdc
systemctl status krb5kdc
Comment 6 Menanteau Guy 2017-06-26 13:57:50 UTC 
Created attachment 1291985 [details]
krb4kdc log when kinit problem
Comment 7 Menanteau Guy 2017-06-26 13:58:45 UTC 
journalctl -u krb5kdc
-- Logs begin at Sun 2017-06-25 14:22:00 EDT, end at Mon 2017-06-26 09:31:44 EDT. --
Jun 26 09:25:16 ipa001.domain.local systemd[1]: Starting Kerberos 5 KDC...
Jun 26 09:25:16 ipa001.domain.local systemd[1]: krb5kdc.service: PID file /var/run/krb5kdc.pid not readable (yet?) after start: No such file or directory
Jun 26 09:25:16 ipa001.domain.local systemd[1]: Started Kerberos 5 KDC.
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Stopping Kerberos 5 KDC...
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Stopped Kerberos 5 KDC.
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Starting Kerberos 5 KDC...
Jun 26 09:27:35 ipa001.domain.local systemd[1]: krb5kdc.service: PID file /var/run/krb5kdc.pid not readable (yet?) after start: No such file or directory
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Started Kerberos 5 KDC.

systemctl status krb5kdc
krb5dc.service - Kerberos 5 KDC
   Loaded: loaded (/usr/lib/systemd/krb5kdc.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2017-06-26 09:27:35 EDT; 2min 44s ago
 Main PID: 9142 (krb5kdc)
     Tasks: 1 (limit: 4915)
    CGroup: /system.slice/krb5kdc.service
               9142 /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid

Jun 26 09:27:35 ipa001.domain.local systemd[1]: Stopped Kerberos 5 KDC.
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Starting Kerberos 5 KDC.
Jun 26 09:27:35 ipa001.domain.local systemd[1]: krb5kdc.service: PID file /var/run/krb5kdc.pid not readable (yet?) after...rectory
Jun 26 09:27:35 ipa001.domain.local systemd[1]: Started Kerberos 5 KDC.
Comment 8 Menanteau Guy 2017-06-27 13:53:22 UTC 
I don't see explicit traces pointing on a problem for krb5kdc. Traces about krb5kdc.pid file not readable are still there when the problem does not occur ("enforcing" disabled).
Comment 9 Lukas Vrabec 2017-06-27 14:27:12 UTC 
Menanteau, 

You mean SELinux is in permissive mode? 

Attach output:
# sestatus


Thanks,
Lukas.
Comment 10 Menanteau Guy 2017-06-27 14:54:16 UTC 
Yes, as I said in the description of the bug, I never had problem when I run "setenforce 0" before running ipa or kerberos commands.
Comment 11 Lukas Vrabec 2017-06-27 15:59:26 UTC 
Moving to FreeIPA.This is issue is not related to SELinux.
Comment 12 Martin Bašti 2017-06-27 16:43:25 UTC 
It consists of two bugs

1)
This is SELinux

type=AVC msg=audit(1497726952.793:735): avc:  denied  { execute_no_trans } for  pid=9992 comm="ipa-dnskeysyncd" path="/usr/libexec/ipa/ipa-dnskeysync-replica" dev="dm-0" ino=5833743 scontext=system_u:system_r:ipa_dnskey_t:s0 tcontext=system_u:object_r:ipa_dnskey_exec_t:s0 tclass=file permissive=0

2)
The kinit issues are not selinux.
Comment 13 Menanteau Guy 2017-06-28 07:54:59 UTC 
Hum, perhaps I didn't try enough, but so far, I didn't see the kinit problem when selinux is disabled. I will run a longer serie of tests with selinux disable to see if kinit fails sometime.
Comment 14 Standa Laznicka 2017-08-30 08:13:47 UTC 
Comment 13 shows it is actually a SELinux issue.
Comment 15 Michel Normand 2017-09-06 09:09:01 UTC 
no more failure with last compose 20170903.
Comment 16 Michel Normand 2017-09-06 09:10:45 UTC 
(In reply to Michel Normand from comment #15)
> no more failure with last compose 20170903.

oups I closed the wrong bug, sorry, I re-open it
Comment 17 Fedora Update System 2017-10-26 12:32:08 UTC 
selinux-policy-3.13.1-260.14.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d312739a4e
Comment 18 Fedora Update System 2017-11-15 20:11:27 UTC 
selinux-policy-3.13.1-260.14.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
Comment 19 Menanteau Guy 2019-10-22 09:07:24 UTC 
comment to clear the needinfo flag