Bug 1898459

Summary: ipa-server-upgrade fails without 'admin' user
Product: [Fedora] Fedora Reporter: Rob Foehl <rwf>
Component: freeipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 33CC: abokovoy, contribs, fcami, ftrivino, ipa-maint, jcholast, jhrozek, mhjacks, pvoborni, rcritten, ssorce, twoerner
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-05 07:43:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
admin user recovery LDAP update file none

Description Rob Foehl 2020-11-17 08:15:06 UTC
Description of problem:

Fedora 31 / FreeIPA 4.8.6 server upgraded to Fedora 33, post-upgrade reboot ipa-server-upgrade process fails when trying to add the root@REALM principal alias to 'admin' if that user does not exist.


Version-Release number of selected component (if applicable):

4.8.10-6.fc33


How reproducible:

100%

Actual results:

add_admin_root_alias() function from https://pagure.io/freeipa/c/65c2736bd20ffb9d98769e71d905f71d1a4d857e fails if the 'admin' user does not exist, as follows:

2020-11-17T02:12:31Z DEBUG raw: user_add_principal('admin', ('root',), version='2.239')
2020-11-17T02:12:31Z DEBUG user_add_principal('admin', (ipapython.kerberos.Principal('root'),), all=False, raw=False, version='2.239', no_members=False)
2020-11-17T02:12:31Z DEBUG raw: trust_find('', sizelimit=0, version='2.239')
2020-11-17T02:12:31Z DEBUG trust_find(None, sizelimit=0, all=False, raw=False, version='2.239', pkey_only=False)
2020-11-17T02:12:31Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2020-11-17T02:12:31Z DEBUG   File "/usr/lib/python3.9/site-packages/ipapython/admintool.py", line 180, in execute
    return_value = self.run()
  File "/usr/lib/python3.9/site-packages/ipaserver/install/ipa_server_upgrade.py", line 54, in run
    server.upgrade()
  File "/usr/lib/python3.9/site-packages/ipaserver/install/server/upgrade.py", line 1847, in upgrade
    upgrade_configuration()
  File "/usr/lib/python3.9/site-packages/ipaserver/install/server/upgrade.py", line 1729, in upgrade_configuration
    add_admin_root_alias()
  File "/usr/lib/python3.9/site-packages/ipaserver/install/server/upgrade.py", line 1340, in add_admin_root_alias
    api.Command.user_add_principal("admin", rootprinc)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 471, in __call__
    return self.__do_call(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 499, in __do_call
    ret = self.run(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipalib/frontend.py", line 821, in run
    return self.execute(*args, **options)
  File "/usr/lib/python3.9/site-packages/ipaserver/plugins/baseldap.py", line 2388, in execute
    entry_attrs.dn = callback(
  File "/usr/lib/python3.9/site-packages/ipaserver/plugins/baseuser.py", line 772, in pre_callback
    ensure_krbcanonicalname_set(ldap, entry_attrs)
  File "/usr/lib/python3.9/site-packages/ipalib/util.py", line 1164, in ensure_krbcanonicalname_set
    old_entry = ldap.get_entry(
  File "/usr/lib/python3.9/site-packages/ipapython/ipaldap.py", line 1633, in get_entry
    entries = self.get_entries(
  File "/usr/lib/python3.9/site-packages/ipapython/ipaldap.py", line 1444, in get_entries
    entries, truncated = self.find_entries(
  File "/usr/lib/python3.9/site-packages/ipapython/ipaldap.py", line 1585, in find_entries
    break
  File "/usr/lib64/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3.9/site-packages/ipapython/ipaldap.py", line 1088, in error_handler
    raise errors.NotFound(reason=arg_desc or 'no such entry')

2020-11-17T02:12:31Z DEBUG The ipa-server-upgrade command failed, exception: NotFound: no such entry

This happens regardless of whether the root principal alias has been manually added to another IPA user (in the admins group, although that shouldn't matter here).  It would appear that the code handles a duplicate entry exception if the alias exists elsewhere, but here the user_add_principal call is throwing NotFound.


Expected results:

The duplicate entry test for another user works and there is no dependency on a specific user named 'admin'.  If neither are available, perhaps failing with an error to the effect that a root@REALM alias must be added by hand.

Comment 1 Alexander Bokovoy 2020-11-17 08:19:33 UTC
We do not support installations without admin user and admins group. They must exist.

Comment 2 Rob Foehl 2020-11-17 08:41:37 UTC
I'm not sure where to start with that, given the motivation here...  Either leaving 'admin' around is a bad idea for the exact same reason, or this change isn't necessary.

Comment 3 Alexander Bokovoy 2020-11-17 08:54:50 UTC
I would point you to https://bugzilla.redhat.com/show_bug.cgi?id=1789205 but this bug is private. Let me copy the relevant fragments of it:

------
Customer executed ipa-adtrust-install and it was successful but it throws a warning message. But cutomer has a valid principal of IPA administrator user. He excuted kinit with an IPA administrator user before ipa-adtrust-install. 

~~~~~~~~
WARNING: you MUST re-kinit admin user before using 'ipa trust-*' commands
family in order to re-generate Kerberos tickets to include AD-specific
~~~~~~~~

But he is not using admin user. They have deleted an admin user and using a user which is in the group of administrators in IPA to perform this. I don't think this should be a problem. Am I correct ? 


And he tried to create the trust again and that fails. I am attaching those outputs also. They tried with two administrator users. Attaching both the outputs.

-----
Deleting pre-defined administrative user and group is certainly not supported. 
If they are not using those, one can disable 'admin' user with 'ipa user-disable admin' command.

ipa-adtrust-install expects both of them exist and specifically admin user to exist before modifying the admins group.
-----


This all resulted in updating the documentation to provide the information about admin/admins:

User admin is required to exist: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/linux_domain_identity_authentication_and_policy_guide/index#managing-users-life-cycle
Group admins is required to exist: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/linux_domain_identity_authentication_and_policy_guide/index#ugroups-intro-default

Comment 4 Alexander Bokovoy 2020-11-17 09:17:52 UTC
The second part of it was how to recover admin user / admins group:

------

It is a bit complicated. Attached is an update file that needs to be modified before applying.

Please run on the initial master 

# ipa idrange-show EXAMPLE.COM_id_range

where EXAMPLE.COM is replaced by your realm name.

It should display an ID range assigned to this master. Something like

  Range name: EXAMPLE.COM_id_range
  First Posix ID of the range: 1536000000
  Number of IDs in the range: 200000
  First RID of the corresponding RID range: 1000
  First RID of the secondary RID range: 100000000
  Range type: local domain range

Take the value from 'First Posix ID of the range' and replace string VALUE in the attached file with it. E.g., replace

default: uidNumber: VALUE

with

default: uidNumber: 1536000000

and the same for gidNumber.

After that, apply the update file:

# ipa-ldap-updater ./80-admin-user.update

This should create an admin user with the correct uidNumber and SID.

After that, you need to add 'admin' user to admins group with

# ipa group-add-member admins --users=admin

and, finally, re-run ipa-adtrust-install

# ipa-adtrust-install --add-sids

Then verify that admins group has a SID ending with -512, like before.

-------

We also created https://github.com/freeipa/freeipa-healthcheck/issues/120 to track the issue through ipa-healthcheck. There is a rule now that should highlight the fact that admin/admins are missing. The ticket for healthcheck is still open as tests aren't merged yet.

Comment 5 Alexander Bokovoy 2020-11-17 09:18:26 UTC
Created attachment 1730052 [details]
admin user recovery LDAP update file

Comment 6 Alexander Bokovoy 2020-11-17 09:19:59 UTC
Also, domain SID in the 80-admin-user.update should be updated to correspond to IPA domain SID.

Comment 7 Rob Foehl 2020-11-17 10:29:53 UTC
Thanks for the pointer on how to recover that user -- I'd already tried to find that without any luck.

Still problematic having it around, though.  In any case, I'll have to see whether I have the stamina for another round of FreeIPA misadventures tomorrow.

Comment 8 Ben Cotton 2021-11-04 17:14:31 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 François Cami 2021-11-05 07:43:37 UTC
Closing as NOTABUG as the behavior is expected.