Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1375656

Summary:	ipa-replica-install with 4.3.2-2.fc24 fails with CCacheError: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (39756044): Credential cache is empty
Product:	Red Hat Enterprise Linux 7	Reporter:	Petr Vobornik <pvoborni>
Component:	ipa	Assignee:	IPA Maintainers <ipa-maint>
Status:	CLOSED WORKSFORME	QA Contact:	Kaleem <ksiddiqu>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	7.3	CC:	abokovoy, extras-qa, ipa-maint, jcholast, jhrozek, jpazdziora, mbabinsk, mkosek, pvoborni, rcritten, ssorce
Target Milestone:	rc	Keywords:	Regression
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:	Cause: When `ipa-replica-install` is run with `--setup-ca` and `setup-dns` options without existing ccache present(without ipa client installed first). Consequence: Replica installation on domain level 1 without client installed first will fail. Workaround (if any): Run `ipa-client-install` first, kinit as privileged user (or use OTP) and run `ipa-replica-install`. Result: Replica is installed on domain level 1.	Story Points:	---
Clone Of:	1373883	Environment:
Last Closed:	2016-09-13 16:37:42 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1373883
Bug Blocks:

Description Petr Vobornik 2016-09-13 15:33:01 UTC

+++ This bug was initially created as a clone of Bug #1373883 +++

Description of problem:

ipa-replica-install with 4.3.2-2.fc24 fails with

  [22/24]: Restart HTTP server to pick up changes
  [23/24]: enabling CA instance
  [24/24]: Updating DNS CA records
  [error] CCacheError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
ipa.ipapython.install.cli.install_tool(Replica): ERROR    Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

Version-Release number of selected component (if applicable):

freeipa-server-4.3.2-2.fc24

How reproducible:

Deterministic for ipa-replica-install runs that reach this far.

Steps to Reproduce:
1. Have 4.3.2-2.fc24 master.
2. Have another Fedora 24 machine with freeipa-server-4.3.2-2.fc24 bits installed.
3. Run /usr/sbin/ipa-replica-install -U --setup-ca --setup-dns --forwarder=10.11.12.13 --ip-address=10.11.12.50 -P admin -w TheVery5ecretPa55word

Actual results:

  [14/24]: importing CA chain to RA certificate database
  [15/24]: fixing RA database permissions
  [16/24]: setting up signing cert profile
  [17/24]: setting audit signing renewal to 2 years
  [18/24]: configure certificate renewals
  [19/24]: configure Server-Cert certificate renewal
  [20/24]: Configure HTTP to proxy connections
  [21/24]: updating IPA configuration
  [22/24]: Restart HTTP server to pick up changes
  [23/24]: enabling CA instance
  [24/24]: Updating DNS CA records
  [error] CCacheError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
ipa.ipapython.install.cli.install_tool(Replica): ERROR    Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

The log ends with

2016-09-07T09:59:32Z DEBUG   duration: 1 seconds
2016-09-07T09:59:32Z DEBUG   [23/24]: enabling CA instance
2016-09-07T09:59:32Z DEBUG Starting external process
2016-09-07T09:59:32Z DEBUG args=/bin/systemctl disable pki-tomcatd.target
2016-09-07T09:59:32Z DEBUG Process finished, return code=0
2016-09-07T09:59:32Z DEBUG stdout=
2016-09-07T09:59:32Z DEBUG stderr=Removed symlink /etc/systemd/system/multi-user.target.wants/pki-tomcatd.target.

2016-09-07T09:59:32Z DEBUG   duration: 0 seconds
2016-09-07T09:59:32Z DEBUG   [24/24]: Updating DNS CA records
2016-09-07T09:59:32Z DEBUG flushing ldapi://%2fvar%2frun%2fslapd-EXAMPLE-TEST.socket from SchemaCache
2016-09-07T09:59:32Z DEBUG retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-EXAMPLE-TEST.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f888caeffc8>
2016-09-07T09:59:32Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2016-09-07T09:59:32Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index'
2016-09-07T09:59:32Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 447, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 437, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1299, in __update_ca_records
    bind.add_ipa_ca_dns_records(api.env.host, api.env.domain)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 1086, in add_ipa_ca_dns_records
    self.api.Backend.ldap2.connect(autobind=True)
  File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, in connect
    conn = self.create_connection(*args, **kw)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 199, in create_connection
    principal = krb_utils.get_principal(ccache_name=ccache)
  File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line 185, in get_principal
    raise errors.CCacheError(message=unicode(e))
CCacheError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty

2016-09-07T09:59:32Z DEBUG   [error] CCacheError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
2016-09-07T09:59:32Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, in run
    cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, in run
    self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, in execute
    for nothing in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 586, in _configure
    next(executor)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install
    for nothing in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1687, in main
    promote(self)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 377, in decorated
    func(installer)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1484, in promote
    ca_cert_bundle=ca_data)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1371, in configure_replica
    self.start_creation(runtime=210)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 447, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 437, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1299, in __update_ca_records
    bind.add_ipa_ca_dns_records(api.env.host, api.env.domain)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 1086, in add_ipa_ca_dns_records
    self.api.Backend.ldap2.connect(autobind=True)
  File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, in connect
    conn = self.create_connection(*args, **kw)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 199, in create_connection
    principal = krb_utils.get_principal(ccache_name=ccache)
  File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line 185, in get_principal
    raise errors.CCacheError(message=unicode(e))

2016-09-07T09:59:32Z DEBUG The ipa-replica-install command failed, exception: CCacheError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
2016-09-07T09:59:32Z ERROR Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (39756044): Credential cache is empty
2016-09-07T09:59:32Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Expected results:

No error, replica set up.

Additional info:

--- Additional comment from Jan Pazdziora on 2016-09-07 13:19:34 CEST ---

Reproducer beaker jobs:

https://beaker.engineering.redhat.com/jobs/1490407
https://beaker.engineering.redhat.com/jobs/1490408
https://beaker.engineering.redhat.com/jobs/1490410

--- Additional comment from Martin Babinsky on 2016-09-12 14:43:57 CEST ---

I was able to reproduce this bug locally. It happens when `ipa-replica-install` is run with `--setup-ca` and `setup-dns` options without existing ccache present.

A workaround is to first run `ipa-client-install`, kinit as privileged user (or use OTP) and run `ipa-replica-install`.

--- Additional comment from Martin Babinsky on 2016-09-12 14:49:14 CEST ---

Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6299

Comment 3 Martin Babinsky 2016-09-13 16:37:42 UTC

The issue is actually not present in RHEL 7.3 builds due to refactoring of DNS record reation/update done during implementation of DNS Locations feature.

Verified to work using ipa-server-4.4.0-10.el7.x86_64