Bug 1438724
| Summary: | ipa-server-install fails with Error: Unable to set admin password Command | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Varun Mylaraiah <mvarun> |
| Component: | 389-ds-base | Assignee: | mreynolds |
| Status: | CLOSED DUPLICATE | QA Contact: | Viktor Ashirov <vashirov> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | abokovoy, jpazdziora, ksiddiqu, mreynolds, mvarun, nkinder, pvoborni, rcritten, rmeggins, tbordaz, tscherf |
| Target Milestone: | rc | Keywords: | Regression, Reopened, TestBlocker |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | 389-ds-base-1.3.6.1-7.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-05-20 13:51:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
389 DS version [root@cloud-qe-02 ~]# rpm -qa 389* 389-ds-base-1.3.6.1-6.el7.x86_64 389-ds-base-libs-1.3.6.1-6.el7.x86_64 Why is this a 389 bug? (In reply to Petr Vobornik from comment #5) > Why is this a 389 bug? I changed component to 389-ds because with previous 389-ds build (389-ds-base-1.3.6.1-5.el7.x86_64) IPA install successful and failing with build 389-ds-base-1.3.6.1-6.el7.x86_64 Regarding DS: - there is a crash (I do not know if a core was dumped) type=ANOM_ABEND msg=audit(1491290744.757:224): auid=4294967295 uid=389 gid=389 ses=4294967295 pid=2007 comm="ns-slapd" reason="memory violation" sig=11 - There are cos errors (I do not know if they are related to the crash) ERR - cos-plugin - cos_cache_query_attr - cos attribute krbPwdPolicyReference failed schema check on dn: cn=dns,dc=testrelm,dc=test Investigating the crash
- nsslapd crash did not dump a core
because of machine/sysconfig config it did not allow core dump
[root@cloud-qe-02 ~]# cat /etc/sysconfig/dirsrv.systemd
[Service]
LimitNOFILE=8192
[root@cloud-qe-02 ~]# tail /etc/sysconfig/dirsrv
# there is a problem and fail to start.
# If using systemd, omit the "; export STARTPID_TIME" at the end.
#STARTPID_TIME=10 ; export STARTPID_TIME
# How many seconds to wait for the pid file to show up before we assume there
# is a problem and fail to start.
# If using systemd, omit the "; export PID_TIME" at the end.
#PID_TIME=600 ; export PID_TIME
KRB5CCNAME=/tmp/krb5cc_389
KRB5_KTNAME=/etc/dirsrv/ds.keytab
[root@cloud-qe-02 ~]# sysctl -a |grep suid
fs.suid_dumpable = 0
Is it possible to try to reproduce with http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes
- cos error messages are tracked with https://bugzilla.redhat.com/show_bug.cgi?id=1437492
Is it reproducible ? If yes could it be reproduced with those settings http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes The crash is reproducible
(gdb) where
#0 0x00007fbd027f03b7 in op_shared_allow_pw_change (pb=pb@entry=0x5634932b8d00, mod=0x5634933d6c20,
old_pw=old_pw@entry=0x7fbcb97794d8, smods=smods@entry=0x7fbcb97794e0) at ldap/servers/slapd/modify.c:1241
#1 0x00007fbd027f223f in modify_internal_pb (pb=pb@entry=0x5634932b8d00) at ldap/servers/slapd/modify.c:573
#2 0x00007fbd027f2c93 in slapi_modify_internal_pb (pb=pb@entry=0x5634932b8d00) at ldap/servers/slapd/modify.c:454
#3 0x00007fbcf50f9d10 in ipapwd_apply_mods (dn=0x5634932f8e40 "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test",
mods=0x5634933d66c0) at common.c:950
#4 0x00007fbcf50fa3f9 in ipapwd_SetPassword (krbcfg=krbcfg@entry=0x5634932c8d50, data=data@entry=0x7fbcb97797c0, is_krb=1)
at common.c:866
#5 0x00007fbcf510078f in ipapwd_chpwop (krbcfg=0x5634932c8d50, pb=0x7fbcb9779a50) at ipa_pwd_extop.c:577
#6 ipapwd_extop (pb=0x7fbcb9779a50) at ipa_pwd_extop.c:1770
#7 0x000056348c5cb451 in do_extended (pb=pb@entry=0x7fbcb9779a50) at ldap/servers/slapd/extendop.c:348
#8 0x000056348c5c45e8 in connection_dispatch_operation (pb=0x7fbcb9779a50, op=0x56348f6dfa00, conn=0x563492edf240)
at ldap/servers/slapd/connection.c:688
#9 connection_threadmain () at ldap/servers/slapd/connection.c:1772
#10 0x00007fbd00b699bb in _pt_root (arg=0x563492e0ca20) at ../../../nspr/pr/src/pthreads/ptthread.c:216
#11 0x00007fbd00509e25 in start_thread (arg=0x7fbcb977a700) at pthread_create.c:308
#12 0x00007fbcffdeb34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
It is due to the fix https://pagure.io/389-ds-base/issue/49039.
In case of internal update, the connection is NULL and the fix does not test its value before derefencing it.
Need to open a new ticket
This bug is a 389-ds bug tracked with https://pagure.io/389-ds-base/issue/49210 I can not reproduce the crash using an internal extended operation. So this is specific to how the IPA plugin is setting the password. The problem is that I can not find these source files (ipa_pwd_extop.c & common.c) in the latest version of the freeipa source code. Where can I find these files? (In reply to mreynolds from comment #12) > I can not reproduce the crash using an internal extended operation. So this > is specific to how the IPA plugin is setting the password. The problem is > that I can not find these source files (ipa_pwd_extop.c & common.c) in the > latest version of the freeipa source code. > > Where can I find these files? I found them - nevermind... This was a regression in 1394899 - which is now fixed *** This bug has been marked as a duplicate of bug 1394899 *** Sorry, this is *not* a duplicate. If anything, it is a result of improper fix in bug 1394899. Crash: ------------ Thread 1 (Thread 0x7efda9a47700 (LWP 17128)): #0 0x00007efdd83ca3b7 in op_shared_allow_pw_change (pb=pb@entry=0x55977c703900, mod=0x55977d219a60, old_pw=old_pw@entry=0x7efda9a464d8, smods=smods@entry=0x7efda9a464e0) at ldap/servers/slapd/modify.c:1241 isroot = 0 internal_op = 32 repl_op = 0 pwresponse_req = 0 res = <optimized out> dn = 0x55977cab5b00 "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test" errtxt = 0x0 sdn = {flag = 6 '\006', udn = 0x55977cab5b00 "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test", dn = 0x55977cab5ac0 "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test", ndn = 0x55977cab5bc0 "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test", ndn_len = 50} e = 0x0 pwpolicy = 0x55977c7fc050 rc = 0 values = 0x0 operation = 0x55977b95fee0 proxy_err = <optimized out> proxydn = 0x0 proxystr = 0x0 errtext = 0x0 #1 0x00007efdd83cc23f in modify_internal_pb (pb=0x55977c703900) at ldap/servers/slapd/modify.c:573 controls = 0x0 pwpolicy_ctrl = 0 op = 0x0 opresult = 0 normalized_mods = 0x55977cb9b060 mods = 0x55977cab4700 mod = 0x55977cb9b078 smods = {mods = 0x55977cab4700, num_elements = 5, num_mods = 4, iterator = 0, free_mods = 1} pw_change = 0 old_pw = 0x0 #2 0x00007efdcacd3d10 in ipapwd_apply_mods () from /usr/lib64/dirsrv/plugins/libipa_pwd_extop.so No symbol table info available. #3 0x00007efdcacd43f9 in ipapwd_SetPassword () from /usr/lib64/dirsrv/plugins/libipa_pwd_extop.so No symbol table info available. #4 0x00007efdcacda78f in ipapwd_extop () from /usr/lib64/dirsrv/plugins/libipa_pwd_extop.so No symbol table info available. #5 0x0000559778b35451 in do_extended (pb=pb@entry=0x7efda9a46a50) at ldap/servers/slapd/extendop.c:348 extoid = 0x55977d243940 "1.3.6.1.4.1.4203.1.11.1" errmsg = <optimized out> extval = {bv_len = 65, bv_val = 0x55977ca350c0 "0?\200\062uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test\202\tSecret123"} p = 0x55977b2e0840 lderr = <optimized out> rc = 2 len = 65 tag = <optimized out> name = 0x2 <Address 0x2 out of bounds> ---------------- line 1241 is if (!internal_op) { note that internal_op is integer and is not 0 here, so we should skip the whole if body, not crash. Looking at the disassembled code in the backtrace report, I can see this: 0x00007efdd83ca38d <+333>: mov %r15,%rsi 0x00007efdd83ca390 <+336>: mov %rbx,%rdi 0x00007efdd83ca393 <+339>: mov %rax,0x10(%rsp) 0x00007efdd83ca398 <+344>: callq 0x7efdd83827b0 <proxyauth_get_dn@plt> 0x00007efdd83ca39d <+349>: test %eax,%eax 0x00007efdd83ca39f <+351>: jne 0x7efdd83ca768 <op_shared_allow_pw_change+1320> 0x00007efdd83ca3a5 <+357>: mov 0xc(%rsp),%r10d 0x00007efdd83ca3aa <+362>: test %r10d,%r10d 0x00007efdd83ca3ad <+365>: je 0x7efdd83ca500 <op_shared_allow_pw_change+704> 0x00007efdd83ca3b3 <+371>: mov 0x8(%rbx),%rax => 0x00007efdd83ca3b7 <+375>: mov 0xc4(%rax),%eax 0x00007efdd83ca3bd <+381>: test %eax,%eax 0x00007efdd83ca3bf <+383>: jne 0x7efdd83ca3da <op_shared_allow_pw_change+410> 0x00007efdd83ca3c1 <+385>: mov 0x10(%r12),%rdx 0x00007efdd83ca3c6 <+390>: mov %rbp,%rsi 0x00007efdd83ca3c9 <+393>: mov %rbx,%rdi 0x00007efdd83ca3cc <+396>: callq 0x7efdd8380650 <check_pw_minage@plt> 0x00007efdd83ca3d1 <+401>: cmp $0x1,%eax We just load value 'internal_op' from the stack but somehow this operation crashes. I do not see how "if (!internal_op)" would cause a crash - unless there is some other memory corruption going on. This code has been in place for years without issue. Do you have a reproducible test case? Can you rerun this test under valgrind and provide the valgrind report? Also what version of 389-ds-base are you using? (In reply to mreynolds from comment #16) > I do not see how "if (!internal_op)" would cause a crash - unless there is > some other memory corruption going on. This code has been in place for > years without issue. > > Do you have a reproducible test case? Can you rerun this test under > valgrind and provide the valgrind report? > > Also what version of 389-ds-base are you using? Actually it is gdb "mistake" that is showing the wrong line. The crash occurred actually on the next block where it was not tested that pb->pb_conn was valid: /* check if password is within password minimum age; error result is sent directly from check_pw_minage */ if (!pb->pb_conn->c_needpw && check_pw_minage(pb, &sdn, mod->mod_bvalues) == 1) { (In reply to thierry bordaz from comment #18) > (In reply to mreynolds from comment #16) > > I do not see how "if (!internal_op)" would cause a crash - unless there is > > some other memory corruption going on. This code has been in place for > > years without issue. > > > > Do you have a reproducible test case? Can you rerun this test under > > valgrind and provide the valgrind report? > > > > Also what version of 389-ds-base are you using? > > Actually it is gdb "mistake" that is showing the wrong line. > The crash occurred actually on the next block where it was not tested that > pb->pb_conn was valid: > > /* check if password is within password minimum age; > error result is sent directly from check_pw_minage */ > if (!pb->pb_conn->c_needpw && > check_pw_minage(pb, &sdn, mod->mod_bvalues) == 1) > { Ah nice catch Thierry! I was going to do another el7 build today anyway so I'll get this fix in. Actually this is fixed upstream already - it just wasn't picked up in the last RHEL build. This will be in included in today's build... So this is a duplicate actually, closing. Please use 1394899 to track the issue/crash fix. *** This bug has been marked as a duplicate of bug 1394899 *** *** This bug has been marked as a duplicate of bug 1394899 *** |
Description of problem: ipa-server-install fails with Error: Unable to set admin password Command Version-Release number of selected component (if applicable): ipa-server-4.5.0-4.el7.x86_64 How reproducible: 100% Steps to Reproduce: [root@cloud-qe-02 ~]# ipa-server-install --setup-dns --forwarder=10.16.36.29 --reverse-zone=34.19.10.in-addr.arpa. --allow-zone-overlap --hostname=cloud-qe-02.testrelm.test -r TESTRELM.TEST -n testrelm.test -p <XXXXX> -a <XXXXX> --ip-address=10.19.34.27 The log file for this installation can be found in /var/log/ipaserver-install.log ============================================================================== This program will set up the IPA Server. This includes: * Configure a stand-alone CA (dogtag) for certificate management * Configure the Network Time Daemon (ntpd) * Create and configure an instance of Directory Server * Create and configure a Kerberos Key Distribution Center (KDC) * Configure Apache (httpd) * Configure DNS (bind) * Configure the KDC to enable PKINIT To accept the default shown in brackets, press the Enter key. WARNING: conflicting time&date synchronization service 'chronyd' will be disabled in favor of ntpd Warning: skipping DNS resolution of host cloud-qe-02.testrelm.test Checking DNS domain testrelm.test., please wait ... Checking DNS forwarders, please wait ... Using reverse zone(s) 34.19.10.in-addr.arpa. The IPA Master Server will be configured with: Hostname: cloud-qe-02.testrelm.test IP address(es): 10.19.34.27 Domain name: testrelm.test Realm name: TESTRELM.TEST BIND DNS server will be configured to serve IPA domain with: Forwarders: 10.16.36.29 Forward policy: only Reverse zone(s): 34.19.10.in-addr.arpa. Continue to configure the system with these values? [no]: yes The following operations may take some minutes to complete. Please wait until the prompt is returned. Adding [10.19.34.27 cloud-qe-02.testrelm.test] to your /etc/hosts file Configuring NTP daemon (ntpd) [1/4]: stopping ntpd [2/4]: writing configuration [3/4]: configuring ntpd to start on boot [4/4]: starting ntpd Done configuring NTP daemon (ntpd). Configuring directory server (dirsrv). Estimated time: 30 seconds [1/47]: creating directory server user [2/47]: creating directory server instance [3/47]: enabling ldapi [4/47]: configure autobind for root [5/47]: stopping directory server [6/47]: updating configuration in dse.ldif [7/47]: starting directory server [8/47]: adding default schema [9/47]: enabling memberof plugin [10/47]: enabling winsync plugin [11/47]: configuring replication version plugin [12/47]: enabling IPA enrollment plugin [13/47]: configuring uniqueness plugin [14/47]: configuring uuid plugin [15/47]: configuring modrdn plugin [16/47]: configuring DNS plugin [17/47]: enabling entryUSN plugin [18/47]: configuring lockout plugin [19/47]: configuring topology plugin [20/47]: creating indices [21/47]: enabling referential integrity plugin [22/47]: configuring certmap.conf [23/47]: configure new location for managed entries [24/47]: configure dirsrv ccache [25/47]: enabling SASL mapping fallback [26/47]: restarting directory server [27/47]: adding sasl mappings to the directory [28/47]: adding default layout [29/47]: adding delegation layout [30/47]: creating container for managed entries [31/47]: configuring user private groups [32/47]: configuring netgroups from hostgroups [33/47]: creating default Sudo bind user [34/47]: creating default Auto Member layout [35/47]: adding range check plugin [36/47]: creating default HBAC rule allow_all [37/47]: adding entries for topology management [38/47]: initializing group membership [39/47]: adding master entry [40/47]: initializing domain level [41/47]: configuring Posix uid/gid generation [42/47]: adding replication acis [43/47]: enabling compatibility plugin [44/47]: activating sidgen plugin [45/47]: activating extdom plugin [46/47]: tuning directory server [47/47]: configuring directory to start on boot Done configuring directory server (dirsrv). Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes [1/30]: creating certificate server user [2/30]: configuring certificate server instance [3/30]: exporting Dogtag certificate store pin [4/30]: stopping certificate server instance to update CS.cfg [5/30]: backing up CS.cfg [6/30]: disabling nonces [7/30]: set up CRL publishing [8/30]: enable PKIX certificate path discovery and validation [9/30]: starting certificate server instance [10/30]: configure certmonger for renewals [11/30]: requesting RA certificate from CA [12/30]: setting up signing cert profile [13/30]: setting audit signing renewal to 2 years [14/30]: restarting certificate server [15/30]: publishing the CA certificate [16/30]: adding RA agent as a trusted user [17/30]: authorizing RA to modify profiles [18/30]: authorizing RA to manage lightweight CAs [19/30]: Ensure lightweight CAs container exists [20/30]: configure certificate renewals [21/30]: configure Server-Cert certificate renewal [22/30]: Configure HTTP to proxy connections [23/30]: restarting certificate server [24/30]: migrating certificate profiles to LDAP [25/30]: importing IPA certificate profiles [26/30]: adding default CA ACL [27/30]: adding 'ipa' CA entry [28/30]: updating IPA configuration [29/30]: enabling CA instance [30/30]: configuring certmonger renewal for lightweight CAs Done configuring certificate server (pki-tomcatd). Configuring directory server (dirsrv) [1/3]: configuring TLS for DS instance [2/3]: restarting directory server [3/3]: adding CA certificate entry Done configuring directory server (dirsrv). Configuring Kerberos KDC (krb5kdc) [1/10]: adding kerberos container to the directory [2/10]: configuring KDC [3/10]: initialize kerberos container WARNING: Your system is running out of entropy, you may experience long delays [4/10]: adding default ACIs [5/10]: creating a keytab for the directory [6/10]: creating a keytab for the machine [7/10]: adding the password extension to the directory [8/10]: creating anonymous principal [9/10]: starting the KDC [10/10]: configuring KDC to start on boot Done configuring Kerberos KDC (krb5kdc). Configuring kadmin [1/2]: starting kadmin [2/2]: configuring kadmin to start on boot Done configuring kadmin. Restarting directory server to enable password extension plugin Configuring ipa-otpd [1/2]: starting ipa-otpd [2/2]: configuring ipa-otpd to start on boot Done configuring ipa-otpd. Configuring ipa-custodia [1/5]: Generating ipa-custodia config file [2/5]: Making sure custodia container exists [3/5]: Generating ipa-custodia keys [4/5]: starting ipa-custodia [5/5]: configuring ipa-custodia to start on boot Done configuring ipa-custodia. Configuring the web interface (httpd) [1/22]: setting mod_nss port to 443 [2/22]: setting mod_nss cipher suite [3/22]: setting mod_nss protocol list to TLSv1.0 - TLSv1.2 [4/22]: setting mod_nss password file [5/22]: enabling mod_nss renegotiate [6/22]: adding URL rewriting rules [7/22]: configuring httpd [8/22]: setting up httpd keytab [9/22]: retrieving anonymous keytab [10/22]: configuring Gssproxy [11/22]: setting up ssl [12/22]: configure certmonger for renewals [13/22]: importing CA certificates from LDAP [14/22]: publish CA cert [15/22]: clean up any existing httpd ccaches [16/22]: configuring SELinux for httpd [17/22]: create KDC proxy user [18/22]: create KDC proxy config [19/22]: enable KDC proxy [20/22]: restarting httpd [21/22]: configuring httpd to start on boot [22/22]: enabling oddjobd Done configuring the web interface (httpd). Configuring Kerberos KDC (krb5kdc) [1/1]: installing X509 Certificate for PKINIT Done configuring Kerberos KDC (krb5kdc). Applying LDAP updates Upgrading IPA:. Estimated time: 1 minute 30 seconds [1/9]: stopping directory server [2/9]: saving configuration [3/9]: disabling listeners [4/9]: enabling DS global lock [5/9]: starting directory server [6/9]: upgrading server [7/9]: stopping directory server [8/9]: restoring configuration [9/9]: starting directory server Done. Restarting the KDC Configuring DNS (named) [1/12]: generating rndc key file WARNING: Your system is running out of entropy, you may experience long delays [2/12]: adding DNS container [3/12]: setting up our zone [4/12]: setting up reverse zone [5/12]: setting up our own record [6/12]: setting up records for other masters [7/12]: adding NS record to the zones [8/12]: setting up kerberos principal [9/12]: setting up named.conf [10/12]: setting up server configuration [11/12]: configuring named to start on boot [12/12]: changing resolv.conf to point to ourselves Done configuring DNS (named). Restarting the web server to pick up resolv.conf changes Configuring DNS key synchronization service (ipa-dnskeysyncd) [1/7]: checking status [2/7]: setting up bind-dyndb-ldap working directory [3/7]: setting up kerberos principal [4/7]: setting up SoftHSM [5/7]: adding DNSSEC containers [6/7]: creating replica keys [7/7]: configuring ipa-dnskeysyncd to start on boot Done configuring DNS key synchronization service (ipa-dnskeysyncd). Restarting ipa-dnskeysyncd Restarting named Updating DNS system records Unable to set admin password Command '/usr/bin/ldappasswd -h cloud-qe-02.testrelm.test -ZZ -x -D cn=Directory Manager -y /var/lib/ipa/tmptMrGaZ -T /var/lib/ipa/tmpzeYQN9 uid=admin,cn=users,cn=accounts,dc=testrelm,dc=test' returned non-zero exit status 255 Configuring client side components Using existing certificate '/etc/ipa/ca.crt'. Skip cloud-qe-02.testrelm.test: LDAP server is not responding, unable to verify if this is an IPA server Failed to verify that cloud-qe-02.testrelm.test is an IPA Server. This may mean that the remote server is not up or is not reachable due to network or firewall settings. Please make sure the following ports are opened in the firewall settings: TCP: 80, 88, 389 UDP: 88 (at least one of TCP/UDP ports 88 has to be open) Also note that following ports are necessary for ipa-client working properly after enrollment: TCP: 464 UDP: 464, 123 (if NTP enabled) The ipa-client-install command failed. See /var/log/ipaclient-install.log for more information ipa.ipapython.install.cli.install_tool(CompatServerMasterInstall): ERROR Configuration of client side components failed! ipa.ipapython.install.cli.install_tool(CompatServerMasterInstall): ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information Additional info: [root@cloud-qe-02 ~]# getenforce Disabled I have attached machine info in comment1