Bug 1980356
| Summary: | reinstalling samba client causes winbindd coredump | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Sergey Orlov <sorlov> |
| Component: | ipa | Assignee: | Florence Blanc-Renaud <frenaud> |
| Status: | CLOSED ERRATA | QA Contact: | ipa-qe <ipa-qe> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 9.0 | CC: | abokovoy, aboscatt, amore, asn, dkarpele, frenaud, gdeschner, jarrpa, ksiddiqu, myusuf, pvoborni, rcritten, ssidhaye, tscherf |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | sync-to-jira | ||
| Fixed In Version: | ipa-4.9.8-1.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-05-17 12:44:18 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2027125 | ||
| Deadline: | 2021-11-08 | ||
Upstream ticket: https://pagure.io/freeipa/issue/8687 Can you please try if it works with packages from: https://kojihub.stream.rdu2.redhat.com/koji/taskinfo?taskID=471537 Andreas, Would it be possible for you to push the fix into Fedora's updates-testing if not done already please? We can reproduce the problem easily there. François Hello Andreas Sorry for long delay, I have just returned from PTO I tried to download packages from https://kojihub.stream.rdu2.redhat.com/koji/taskinfo?taskID=471537 but I am constantly getting "403 Frobidden". What smb.conf is used in this case? It looks like it fails to allocate a uid/gid. Can we get a full backtrace? Here is smb.conf after running "ipa-client-samba":
[global]
# Limit number of forked processes to avoid SMBLoris attack
max smbd processes = 1000
# Use dedicated Samba keytab. The key there must be synchronized
# with Samba tdb databases or nothing will work
dedicated keytab file = FILE:/etc/samba/samba.keytab
kerberos method = dedicated keytab
# Set up logging per machine and Samba process
log file = /var/log/samba/log.%m
log level = 1
# We force 'member server' role to allow winbind automatically
# discover what is supported by the domain controller side
server role = member server
realm = TESTRELM.TEST
netbios name = CLIENT
workgroup = TESTRELM
# Local writable range for IDs not coming from IPA or trusted domains
idmap config * : range = 0 - 0
idmap config * : backend = tdb
idmap config TESTRELM : range = 1973000000 - 1973199999
idmap config TESTRELM : backend = sss
# Default homes share
[homes]
read only = no
To get more details you can reproduce the issue using following procedure: - Get 2 VMs with RHEL9 or other affected OS (RHEL8.5 or Fedora). 1MT works just fine - On the first machine (the IPA master) execute: hostnamectl set-hostname master.testrelm.test systemctl stop firewalld dnf install -y *ipa-server-dns *ipa-server-trust-ad ipa-server-install -a Secret123 -p Secret123 --setup-dns --auto-forwarders -n testrelm.test -U -r TESTRELM.TEST echo Secret123 | kinit admin ipa-adtrust-install -U - On the second machine (IPA client) execute: hostnamectl set-hostname client.testrelm.test echo "<MASTER_IP_ADDRESS> master.testrelm.test" >> /etc/hosts dnf install -y *ipa-client *ipa-client-samba ipa-client-install -p admin -w Secret123 --domain testrelm.test -U --server master.testrelm.test ipa-client-samba -U echo Secret123 | kinit admin ipa dnsrecord-add testrelm.test client --a-ip-address <CLIENT_IP_ADDRESS> ipa-client-samba -U --server master.testrelm.test systemctl enable --now smb winbind ipa-client-samba --uninstall -U ipa-client-samba -U --server master.testrelm.test systemctl enable --now smb winbind winbind is asserting here on purpose. It looks like IPA has been stopped. So it tries to allocate an id for the system user (root) and fails. This means ipa needs to cleanup files and databases correctly. Moving back to ipa, the fix needs to be done in ipa-client-samba uninstaller. Upstream PR in review https://github.com/freeipa/freeipa/pull/6056 Fixed upstream master: https://pagure.io/freeipa/c/91334b271a929aca8d77c143db005f358cb7ea4a https://pagure.io/freeipa/c/bbe554acf4d164d8d3847f343395ba28abac0392 Fixed upstream ipa-4-9: https://pagure.io/freeipa/c/82eaa2eac454aed75a498d2c6ccd9e921f9c8a89 https://pagure.io/freeipa/c/6302769b83af75f267c76fe6f854d5b42b6b80f5 Included with the rebase to 4.9.8 A) Tested in permissive mode reason: https://bugzilla.redhat.com/show_bug.cgi?id=2025931 Tested using: 2021-12-08T15:11:45 ok: [master.testrelm.test] => (item=ipa-server) => 2021-12-08T15:11:45 name: ipa-server 2021-12-08T15:11:45 release: 1.el9 2021-12-08T15:11:45 source: rpm 2021-12-08T15:11:45 version: 4.9.8 2021-12-08T15:11:45 ok: [master.testrelm.test] => (item=sssd-ipa) => 2021-12-08T15:11:45 name: sssd-ipa 2021-12-08T15:11:45 release: 1.el9 2021-12-08T15:11:45 source: rpm 2021-12-08T15:11:45 version: 2.6.1 2021-12-08T15:11:45 ok: [master.testrelm.test] => (item=389-ds-base) => 2021-12-08T15:11:45 name: 389-ds-base 2021-12-08T15:11:45 release: 3.el9 2021-12-08T15:11:45 source: rpm 2021-12-08T15:11:45 version: 2.0.11 B) Test-result.txt ============================= test session starts ============================== collecting ... collected 16 items test_integration/test_smb.py::TestSMB::test_samba_uninstallation_without_installation PASSED [ 6%] test_integration/test_smb.py::TestSMB::test_install_samba PASSED [ 12%] test_integration/test_smb.py::TestSMB::test_authentication_with_smb_cifs_principal_alias PASSED [ 18%] test_integration/test_smb.py::TestSMB::test_samba_service_listed PASSED [ 25%] test_integration/test_smb.py::TestSMB::test_smb_access_for_ipa_user_at_ipa_client PASSED [ 31%] test_integration/test_smb.py::TestSMB::test_smb_access_for_ad_user_at_ipa_client PASSED [ 37%] test_integration/test_smb.py::TestSMB::test_smb_mount_and_access_by_different_users PASSED [ 43%] test_integration/test_smb.py::TestSMB::test_smb_service_s4u2self PASSED [ 50%] test_integration/test_smb.py::TestSMB::test_smb_mount_fails_without_kerberos_ticket PASSED [ 56%] test_integration/test_smb.py::TestSMB::test_uninstall_samba PASSED [ 62%] test_integration/test_smb.py::TestSMB::test_repeated_uninstall_samba PASSED [ 68%] test_integration/test_smb.py::TestSMB::test_samba_reinstall PASSED [ 75%] test_integration/test_smb.py::TestSMB::test_cleanup PASSED [ 81%] test_integration/test_smb.py::TestSMB::test_ntlm_authentication_with_auto_domain PASSED [ 87%] test_integration/test_smb.py::TestSMB::test_ntlm_authentication_with_upn_with_lowercase_domain PASSED [ 93%] test_integration/test_smb.py::TestSMB::test_ntlm_authentication_with_upn_with_uppercase_domain PASSED [100%] test_samba_reinstall PASSED based on this marking verified:Tested Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: ipa), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:2387 |
The upstream test `test_smb` fails in `test_integration/test_smb.py::TestSMB::test_samba_reinstall`: ``` self = <ipatests.test_integration.test_smb.TestSMB object at 0x7f2d939a09a0> def test_samba_reinstall(self): """Test samba can be reinstalled. Test installation after uninstallation and do some sanity checks. Test for bug https://pagure.io/freeipa/issue/8021 """ > self.test_install_samba() test_integration/test_smb.py:403: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_integration/test_smb.py:238: in test_install_samba self.smbserver.run_command(['systemctl', 'status', service]) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ipatests.pytest_ipa.integration.host.Host client1.ipa.test (client)> argv = ['systemctl', 'status', 'winbind'], set_env = True, stdin_text = None log_stdout = True, raiseonerr = True, cwd = None, bg = False, encoding = 'utf-8' ok_returncode = 0 def run_command(self, argv, set_env=True, stdin_text=None, log_stdout=True, raiseonerr=True, cwd=None, bg=False, encoding='utf-8', ok_returncode=0): """Wrapper around run_command to log stderr on raiseonerr=True :param ok_returncode: return code considered to be correct, you can pass an integer or sequence of integers """ result = super().run_command( argv, set_env=set_env, stdin_text=stdin_text, log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg, encoding=encoding ) # in FIPS mode SSH may print noise to stderr, remove the string # "FIPS mode initialized" + optional newline. result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes) try: result_ok = result.returncode in ok_returncode except TypeError: result_ok = result.returncode == ok_returncode if not result_ok and raiseonerr: result.log.error('stderr: %s', result.stderr_text) > raise subprocess.CalledProcessError( result.returncode, argv, result.stdout_text, result.stderr_text ) E subprocess.CalledProcessError: Command '['systemctl', 'status', 'winbind']' returned non-zero exit status 3. pytest_ipa/integration/host.py:200: CalledProcessError ``` The journal shows that winbindd coredumped: ``` Jan 30 12:27:36 client1.ipa.test systemd[1]: Starting Samba Winbind Daemon... Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.429596, 0] ../../source3/winbindd/winbindd_cache.c:3204(initialize_winbindd_cache) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: initialize_winbindd_cache: clearing cache and re-creating with version number 2 Jan 30 12:27:36 client1.ipa.test audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=winbind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Jan 30 12:27:36 client1.ipa.test systemd[1]: Started Samba Winbind Daemon. Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.442880, 0] ../../lib/util/become_daemon.c:135(daemon_ready) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: daemon_ready: daemon 'winbindd' finished starting up and ready to serve connections Jan 30 12:27:36 client1.ipa.test systemd[1]: Starting Samba SMB Daemon... Jan 30 12:27:36 client1.ipa.test winbindd[31148]: [2021/01/30 12:27:36.489004, 0] ../../source3/winbindd/winbindd_cm.c:1893(wb_open_internal_pipe) Jan 30 12:27:36 client1.ipa.test winbindd[31148]: open_internal_pipe: Could not connect to dssetup pipe: NT_STATUS_RPC_INTERFACE_NOT_FOUND Jan 30 12:27:36 client1.ipa.test winbindd[31148]: [2021/01/30 12:27:36.491526, 0] ../../librpc/rpc/dcesrv_core.c:3007(dcesrv_call_dispatch_local) Jan 30 12:27:36 client1.ipa.test winbindd[31148]: dcesrv_call_dispatch_local: DCE/RPC fault in call lsarpc:2E - DCERPC_NCA_S_OP_RNG_ERROR Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.676076, 0] ../../lib/util/fault.c:172(smb_panic_log) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: =============================================================== Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.676996, 0] ../../lib/util/fault.c:173(smb_panic_log) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 31146 (4.14.0rc1) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.677666, 0] ../../lib/util/fault.c:177(smb_panic_log) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.678298, 0] ../../lib/util/fault.c:182(smb_panic_log) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: =============================================================== Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.678582, 0] ../../lib/util/fault.c:183(smb_panic_log) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: PANIC (pid 31146): Signal 11: Segmentation fault in 4.14.0rc1 Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.680211, 0] ../../lib/util/fault.c:287(log_stack_trace) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: BACKTRACE: 17 stack frames: Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #0 /lib64/libsamba-util.so.0(log_stack_trace+0x34) [0x7f8bafcc4834] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #1 /lib64/libsamba-util.so.0(smb_panic+0xd) [0x7f8bafcc4a8d] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #2 /lib64/libsamba-util.so.0(+0x11b28) [0x7f8bafcc4b28] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #3 /lib64/libpthread.so.0(+0x13970) [0x7f8baf3a3970] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #4 /lib64/libdcerpc-binding.so.0(dcerpc_binding_handle_call_send+0x76) [0x7f8bb02e59d6] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #5 /usr/lib64/samba/libdcerpc-samba4.so(dcerpc_wbint_AllocateGid_r_send+0xa3) [0x7f8bafefaa73] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #6 /usr/lib64/samba/libdcerpc-samba4.so(dcerpc_wbint_AllocateGid_send+0xba) [0x7f8bafefad4a] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #7 /usr/sbin/winbindd(winbindd_allocate_gid_send+0x76) [0x558e3d243606] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #8 /usr/sbin/winbindd(+0x3450c) [0x558e3d1e950c] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #9 /usr/sbin/winbindd(+0x9b55f) [0x558e3d25055f] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #10 /lib64/libtevent.so.0(tevent_common_invoke_fd_handler+0x7d) [0x7f8baf2a1a4d] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #11 /lib64/libtevent.so.0(+0xd4e7) [0x7f8baf2a54e7] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #12 /lib64/libtevent.so.0(+0x5f57) [0x7f8baf29df57] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #13 /lib64/libtevent.so.0(_tevent_loop_once+0x94) [0x7f8baf2a1414] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #14 /usr/sbin/winbindd(main+0xe50) [0x558e3d1df270] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #15 /lib64/libc.so.6(__libc_start_main+0xd5) [0x7f8baeef7b75] Jan 30 12:27:36 client1.ipa.test audit[31146]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:winbind_t:s0 pid=31146 comm="winbindd" exe="/usr/sbin/winbindd" sig=6 res=1 Jan 30 12:27:36 client1.ipa.test winbindd[31146]: #16 /usr/sbin/winbindd(_start+0x2e) [0x558e3d1dfade] Jan 30 12:27:36 client1.ipa.test winbindd[31146]: [2021/01/30 12:27:36.684856, 0] ../../source3/lib/dumpcore.c:317(dump_core) Jan 30 12:27:36 client1.ipa.test winbindd[31146]: coredump is handled by helper binary specified at /proc/sys/kernel/core_pattern Jan 30 12:27:36 client1.ipa.test winbindd[31146]: Jan 30 12:27:36 client1.ipa.test systemd[1]: Created slice system-systemd\x2dcoredump.slice. ``` Additional info: Upstream ticket: https://pagure.io/freeipa/issue/8687