Samba process crashes randomly, 1-3 times pes day. Client is Windows 11 x86_64 PC. Problem appeared after server upgrading to Fedora 40, and was in both samba-4.20.0-7.fc40.x86_64 and samba-4.20.1-1.fc40.x86_64. smbd log contain typically: [2024/05/21 07:04:16.220395, 0] ../../lib/util/fault.c:178(smb_panic_log) =============================================================== [2024/05/21 07:04:16.220554, 0] ../../lib/util/fault.c:179(smb_panic_log) INTERNAL ERROR: Signal 11: Segmentation fault in smbd (smbd[192.168.18) (client [192.168.18.161]) pid 1332190 (4.20.1) [2024/05/21 07:04:16.220590, 0] ../../lib/util/fault.c:186(smb_panic_log) If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting [2024/05/21 07:04:16.220615, 0] ../../lib/util/fault.c:191(smb_panic_log) =============================================================== [2024/05/21 07:04:16.220636, 0] ../../lib/util/fault.c:192(smb_panic_log) PANIC (pid 1332190): Signal 11: Segmentation fault in 4.20.1 [2024/05/21 07:04:16.221412, 0] ../../lib/util/fault.c:303(log_stack_trace) BACKTRACE: 29 stack frames: #0 /usr/lib64/samba/libgenrand-private-samba.so(log_stack_trace+0x35) [0x7fe12d950e65] #1 /usr/lib64/samba/libgenrand-private-samba.so(smb_panic+0x15) [0x7fe12d951675] #2 /usr/lib64/samba/libgenrand-private-samba.so(+0x372d) [0x7fe12d95172d] #3 /lib64/libc.so.6(+0x40710) [0x7fe12d73c710] #4 /usr/lib64/samba/vfs/recycle.so(+0x3c4d) [0x7fe12ad66c4d] #5 /usr/lib64/samba/vfs/recycle.so(+0x41c6) [0x7fe12ad671c6] #6 /usr/lib64/samba/libsmbd-base-private-samba.so(close_file_smb+0xfed) [0x7fe12dda6edd] #7 /usr/lib64/samba/libsmbd-base-private-samba.so(+0x146b97) [0x7fe12dddeb97] #8 /usr/lib64/samba/libsmbd-base-private-samba.so(smbd_smb2_request_process_close+0x259) [0x7fe12dddf039] #9 /usr/lib64/samba/libsmbd-base-private-samba.so(smbd_smb2_request_dispatch+0x1bc4) [0x7fe12ddd4414] #10 /usr/lib64/samba/libsmbd-base-private-samba.so(+0x13e05a) [0x7fe12ddd605a] #11 /lib64/libtevent.so.0(tevent_common_invoke_fd_handler+0x98) [0x7fe12d8f5748] #12 /lib64/libtevent.so.0(+0x1090e) [0x7fe12d8f990e] #13 /lib64/libtevent.so.0(+0x78b4) [0x7fe12d8f08b4] #14 /lib64/libtevent.so.0(_tevent_loop_once+0x99) [0x7fe12d8f2439] #15 /lib64/libtevent.so.0(tevent_common_loop_wait+0x2b) [0x7fe12d8f256b] #16 /lib64/libtevent.so.0(+0x7934) [0x7fe12d8f0934] #17 /usr/lib64/samba/libsmbd-base-private-samba.so(smbd_process+0x90a) [0x7fe12ddc2d2a] #18 /usr/sbin/smbd(+0xbd26) [0x5580a9e01d26] #19 /lib64/libtevent.so.0(tevent_common_invoke_fd_handler+0x98) [0x7fe12d8f5748] #20 /lib64/libtevent.so.0(+0x1090e) [0x7fe12d8f990e] #21 /lib64/libtevent.so.0(+0x78b4) [0x7fe12d8f08b4] #22 /lib64/libtevent.so.0(_tevent_loop_once+0x99) [0x7fe12d8f2439] #23 /lib64/libtevent.so.0(tevent_common_loop_wait+0x2b) [0x7fe12d8f256b] #24 /lib64/libtevent.so.0(+0x7934) [0x7fe12d8f0934] #25 /usr/sbin/smbd(main+0x14e1) [0x5580a9e049c1] #26 /lib64/libc.so.6(+0x2a088) [0x7fe12d726088] #27 /lib64/libc.so.6(__libc_start_main+0x8b) [0x7fe12d72614b] #28 /usr/sbin/smbd(_start+0x25) [0x5580a9e001c5] [2024/05/21 07:04:16.221839, 0] ../../source3/lib/dumpcore.c:317(dump_core) coredump is handled by helper binary specified at /proc/sys/kernel/core_pattern [2024/05/21 07:04:16.443639, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 1332190 did not clean up record 7E0811A2 [2024/05/21 07:04:16.443667, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 7E0811A2 failed: NT_STATUS_FATAL_APP_EXIT [2024/05/21 07:04:16.444518, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 1332190 did not clean up record 4DAECCCC [2024/05/21 07:04:16.444529, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 4DAECCCC failed: NT_STATUS_FATAL_APP_EXIT [2024/05/21 07:04:26.826848, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 1332190 did not clean up record ED39E62F ... Reproducible: Sometimes Steps to Reproduce: Crash occurs randomly, not possible simulate it, during ordinary (not too intensive) work with files on the server - open PDF file on server with Acrobat Reader, workink with app that has open files on the server etc. SELinux enabled, but in permissive mode, /var/log/messages contain no records about smbd selinux violations.
Looks like vfs_recycle issue. Can you provide a configuration? If you have a core file, that would be great as well...
I have no crash dump core image (actually its creation is disabled). testparm -v -s 2>&1 output is: Load smb config files from /etc/samba/smb.conf Loaded services file OK. Weak crypto is allowed by GnuTLS (e.g. NTLM as a compatibility fallback) WARNING: You have not configured 'require strong key = yes' (the default). Your server is vulnerable to CVE-2022-38023 If required use individual 'require strong key:NETBIOSDOMAIN = no' options Server role: ROLE_STANDALONE # Global parameters [global] abort shutdown script = acl claims evaluation = AD DC only ad dc functional level = 2008_R2 add group script = /usr/sbin/groupadd "%g" additional dns hostnames = add machine script = /usr/sbin/useradd -n -c "Workstation (%u)" -M -d /nohome -s /bin/false "%u" addport command = addprinter command = add share command = add user script = /usr/sbin/useradd "%u" -n -g users add user to group script = afs token lifetime = 604800 afs username map = aio max threads = 100 algorithmic rid base = 1000 allow dcerpc auth level connect = No allow dns updates = secure only allow insecure wide links = No allow nt4 crypto = No allow trusted domains = Yes allow unsafe cluster upgrade = No apply group policies = No async dns timeout = 10 async smb echo handler = No auth event notification = No auto services = binddns dir = /var/lib/samba/bind-dns bind interfaces only = No browse list = Yes cache directory = /var/lib/samba change notify = Yes change share command = check password script = cldap port = 389 client ipc max protocol = default client ipc min protocol = default client ipc signing = default client lanman auth = No client ldap sasl wrapping = seal client max protocol = default client min protocol = SMB2_02 client NTLMv2 auth = Yes client plaintext auth = No client protection = default client schannel = Yes client signing = default client smb encrypt = default client smb3 encryption algorithms = AES-128-GCM, AES-128-CCM, AES-256-GCM, AES-256-CCM client smb3 signing algorithms = AES-128-GMAC, AES-128-CMAC, HMAC-SHA256 client use kerberos = desired client use spnego principal = No client use spnego = Yes cluster addresses = clustering = No config backend = file config file = create krb5 conf = Yes ctdbd socket = ctdb locktime warn threshold = 0 ctdb timeout = 0 cups connection timeout = 30 cups encrypt = No cups server = dcerpc endpoint servers = epmapper, wkssvc, samr, netlogon, lsarpc, drsuapi, dssetup, unixinfo, browser, eventlog6, backupkey, dnsserver deadtime = 10080 debug class = Yes debug encryption = No debug hires timestamp = Yes debug pid = No debug prefix timestamp = No debug syslog format = No winbind debug traceid = No debug uid = No dedicated keytab file = default service = defer sharing violations = Yes delete group script = /usr/sbin/groupdel "%g" deleteprinter command = delete share command = delete user from group script = /usr/sbin/userdel "%u" "%g" delete user script = /usr/sbin/userdel "%u" dgram port = 138 disable netbios = No disable spoolss = No dns forwarder = dns port = 53 dns proxy = Yes dns update command = /usr/sbin/samba_dnsupdate dns zone scavenging = No dns zone transfer clients allow = dns zone transfer clients deny = domain logons = No domain master = Yes dos charset = CP852 dsdb event notification = No dsdb group change notification = No dsdb password event notification = No enable asu support = No enable core files = Yes enable privileges = Yes encrypt passwords = Yes enhanced browsing = Yes enumports command = eventlog list = get quota command = getwd cache = Yes gpo update command = /usr/sbin/samba-gpupdate guest account = pcguest host msdfs = Yes hostname lookups = No idmap backend = tdb idmap cache time = 604800 idmap gid = idmap negative cache time = 120 idmap uid = include system krb5 conf = Yes init logon delay = 100 init logon delayed hosts = interfaces = lo br0 tun0 iprint server = kdc default domain supported enctypes = 0 kdc enable fast = Yes kdc force enable rc4 weak session keys = No kdc supported enctypes = 0 keepalive = 300 kerberos encryption types = all kerberos method = default kernel change notify = Yes kpasswd port = 464 krb5 port = 88 lanman auth = No large readwrite = Yes ldap admin dn = ldap connection timeout = 2 ldap debug level = 0 ldap debug threshold = 10 ldap delete dn = No ldap deref = auto ldap follow referral = Auto ldap group suffix = ldap idmap suffix = ldap machine suffix = ldap max anonymous request size = 256000 ldap max authenticated request size = 16777216 ldap max search request size = 256000 ldap page size = 1000 ldap passwd sync = no ldap replication sleep = 1000 ldap server require strong auth = Yes ldap ssl = start tls ldap suffix = ldap timeout = 15 ldap user suffix = lm announce = Auto lm interval = 60 load printers = No local master = Yes lock directory = /var/lib/samba/lock lock spin time = 200 log file = /var/log/samba/log_%I_%U@%m logging = file log level = 1 log nt token command = logon drive = logon home = \\%N\%U logon path = \\%N\%U\profile logon script = smblogon.bat log writeable files on exit = No lpq cache time = 30 lsa over netlogon = No machine password timeout = 604800 mangle prefix = 1 mangling method = hash2 map to guest = Bad User max disk size = 0 max log size = 999500 max mux = 50 max open files = 31000 max smbd processes = 0 max stat cache size = 512 max ttl = 259200 max wins ttl = 518400 max xmit = 16644 mdns name = netbios message command = min domain uid = 1000 min receivefile size = 0 min wins ttl = 21600 mit kdc command = /usr/sbin/krb5kdc multicast dns register = Yes name cache timeout = 660 name resolve order = lmhosts wins host bcast nbt client socket address = 0.0.0.0 nbt port = 137 ncalrpc dir = /run/samba/ncalrpc netbios aliases = netbios name = S15 netbios scope = neutralize nt4 emulation = No nmbd bind explicit broadcast = Yes nsupdate command = /usr/bin/nsupdate -g nt hash store = always ntlm auth = ntlmv1-permitted nt pipe support = Yes ntp signd socket directory = /var/lib/samba/ntp_signd nt status support = Yes null passwords = No obey pam restrictions = No old password allowed period = 60 oplock break wait time = 0 os2 driver map = os level = 205 pam password change = No panic action = passdb backend = tdbsam passdb expand explicit = No passwd chat = *new*password* %n\n *new*password* %n\n *changed* passwd chat debug = No passwd chat timeout = 2 passwd program = password hash gpg key ids = password hash userPassword schemes = password server = * perfcount module = pid directory = /run preferred master = Yes prefork backoff increment = 10 prefork children = 4 prefork maximum backoff = 120 preload modules = printcap cache time = 750 printcap name = private dir = /var/lib/samba/private raw NTLMv2 auth = No read raw = Yes realm = registry shares = No reject md5 clients = Yes reject md5 servers = Yes remote announce = remote browse sync = rename user script = require strong key = No reset on zero vc = No restrict anonymous = 0 root directory = rpc big endian = No rpc server dynamic port range = 49152-65535 rpc server port = 0 rpc start on demand helpers = Yes samba kcc command = /usr/sbin/samba_kcc security = USER server max protocol = SMB3 server min protocol = SMB2_02 server multi channel support = Yes server role = standalone server server schannel = Yes server schannel require seal = Yes server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate, dns server signing = if_required server smb3 encryption algorithms = AES-128-GCM, AES-128-CCM, AES-256-GCM, AES-256-CCM server smb3 signing algorithms = AES-128-GMAC, AES-128-CMAC, HMAC-SHA256 server string = Samba %v set primary group script = set quota command = show add printer wizard = Yes shutdown script = smb1 unix extensions = Yes smb2 disable lock sequence checking = No smb2 disable oplock break retry = No smb2 leases = Yes smb2 max credits = 8192 smb2 max read = 8388608 smb2 max trans = 8388608 smb2 max write = 8388608 smbd profiling level = off smb passwd file = /var/lib/samba/private/smbpasswd smb ports = 445 139 socket options = TCP_NODELAY spn update command = /usr/sbin/samba_spnupdate stat cache = Yes state directory = /var/lib/samba svcctl list = syslog = 1 syslog only = No template homedir = /home/%D/%U template shell = /bin/false time server = Yes timestamp logs = Yes tls cafile = tls/ca.pem tls certfile = tls/cert.pem tls crlfile = tls dh params file = tls enabled = Yes tls keyfile = tls/key.pem tls priority = NORMAL:-VERS-SSL3.0 tls verify peer = as_strict_as_possible unicode = Yes unix charset = UTF-8 unix password sync = No use mmap = Yes username level = 0 username map = /etc/samba/smbusers username map cache time = 0 username map script = usershare allow guests = No usershare max shares = 0 usershare owner only = Yes usershare path = /var/lib/samba/usershares usershare prefix allow list = usershare prefix deny list = usershare template share = utmp = No utmp directory = winbind cache time = 300 winbindd socket directory = /run/samba/winbindd winbind enum groups = No winbind enum users = No winbind expand groups = 0 winbind max clients = 200 winbind max domain connections = 1 winbind nested groups = Yes winbind normalize names = No winbind nss info = template winbind offline logon = No winbind reconnect delay = 30 winbind refresh tickets = No winbind request timeout = 60 winbind rpc only = No winbind scan trusted domains = No winbind sealed pipes = Yes winbind separator = \ winbind use default domain = No winbind use krb5 enterprise principals = Yes wins hook = wins proxy = No wins server = wins support = Yes workgroup = WORKGROUP write raw = Yes wsp property file = wtmp directory = idmap config * : backend = tdb access based share enum = Yes acl allow execute always = Yes acl check permissions = Yes acl flag inherited canonicalization = Yes acl group control = No acl map full control = Yes administrative share = No admin users = afs share = No aio read size = 1 aio write behind = aio write size = 1 allocation roundup size = 0 available = Yes blocking locks = Yes block size = 1024 browseable = Yes case sensitive = Auto check parent directory delete on close = No comment = copy = create mask = 0744 csc policy = manual cups options = default case = lower default devmode = Yes delete readonly = No delete veto files = No dfree cache time = 0 dfree command = directory mask = 0755 dmapi support = No dont descend = dos filemode = No dos filetime resolution = No dos filetimes = Yes durable handles = Yes ea support = Yes fake directory create times = No fake oplocks = No follow symlinks = Yes smbd force process locks = No force create mode = 0000 force directory mode = 0000 force group = force printername = No force unknown acl user = No force user = fstype = NTFS guest ok = No guest only = No hide dot files = Yes hide files = hide new files timeout = 0 hide special files = No hide unreadable = No hide unwriteable files = No honor change notify privilege = No hosts allow = 192.168.18. 192.168.72. 127. hosts deny = include = inherit acls = No inherit owner = no inherit permissions = No invalid users = kernel oplocks = No kernel share modes = No level2 oplocks = Yes locking = Yes lppause command = lpq command = %p lpresume command = lprm command = magic output = magic script = mangled names = illegal mangling char = ~ map acl inherit = No map archive = No map hidden = No map readonly = no map system = No max connections = 0 max print jobs = 1000 max reported print jobs = 0 min print space = 0 msdfs proxy = msdfs root = No msdfs shuffle referrals = No nt acl support = Yes ntvfs handler = unixuid, default oplocks = Yes path = posix locking = Yes postexec = preexec = preexec close = No preserve case = Yes printable = No print command = printer name = printing = cups printjob username = %U print notify backchannel = No queuepause command = queueresume command = read list = read only = Yes root postexec = root preexec = root preexec close = No server addresses = server smb encrypt = default short preserve case = Yes smb3 unix extensions = No smbd async dosmode = No smbd getinfo ask sharemode = Yes smbd max async dosmode = 0 smbd max xattr size = 65536 smbd search ask sharemode = Yes spotlight = No spotlight backend = noindex store dos attributes = Yes strict allocate = No strict locking = Auto strict rename = No strict sync = Yes sync always = No use client driver = No use sendfile = No valid users = veto files = veto oplock files = vfs objects = volume = volume serial number = -1 wide links = No write list = [IPC$] guest ok = Yes guest only = Yes hosts allow = 192.168.18.0/24 192.168.72.0/24 127.0.0.0/8 hosts deny = 0.0.0.0/0 path = /etc/samba/IPC$ [homes] browseable = No comment = Home Directories read only = No [netlogon] comment = Network Logon Service guest ok = Yes path = /etc/samba/netlogon [tmp] comment = Temporary file space guest ok = Yes path = /mnt/data/Samba/tmp read only = No [public] comment = Public SW force create mode = 0664 force directory mode = 02775 force group = admin guest ok = Yes guest only = Yes path = /mnt/data/Samba/public valid users = @s15 @admin write list = @admin [gulas] comment = soubory create mask = 0660 directory mask = 0770 force create mode = 0664 force directory mode = 02770 force group = s15 path = /mnt/data/Samba/gulas valid users = @admin @s15 vfs objects = recycle write list = @admin @s15 recycle:touch_mtime = no recycle:touch = no recycle:exclude_dir = tmp, cache, profiles recycle:exclude = *.tmp, *.log, *.obj, ~*.*, *.bak recycle:versions = yes recycle:keeptree = yes recycle:repository = .recycle [data] comment = Centralni share S15 force create mode = 0660 force directory mode = 02770 force group = s15 path = /home/Samba/s15 read only = No valid users = @s15 @admin vfs objects = recycle write list = @s15 @admin recycle:touch_mtime = no recycle:touch = no recycle:exclude_dir = tmp, cache, profiles recycle:exclude = *.tmp, *.log, *.obj, ~*.*, *.bak recycle:versions = yes recycle:keeptree = yes recycle:repository = .recycle [s15] comment = Centralni share S15 force create mode = 0660 force directory mode = 02770 force group = s15 path = /mnt/data-ssd-s15/s15 read only = No valid users = @s15 @admin write list = @s15 @admin Another thing I forgot to mention - the server had Fedora 30 installed before. I did a clean install of F40, and then copied the contents of the same directory from F30 into the /var/lib/samba/ directory. I assume that can't be the problem - I've done it a few times as well, and the new version has always taken over and possibly converted the files from the older versions to the new format without any problems. But I should have mentioned it.
Excuse me, I should perhaps incluse terse config (without default values). There is: Load smb config files from /etc/samba/smb.conf Loaded services file OK. Weak crypto is allowed by GnuTLS (e.g. NTLM as a compatibility fallback) WARNING: You have not configured 'require strong key = yes' (the default). Your server is vulnerable to CVE-2022-38023 If required use individual 'require strong key:NETBIOSDOMAIN = no' options Server role: ROLE_STANDALONE # Global parameters [global] add group script = /usr/sbin/groupadd "%g" add machine script = /usr/sbin/useradd -n -c "Workstation (%u)" -M -d /nohome -s /bin/false "%u" add user script = /usr/sbin/useradd "%u" -n -g users debug class = Yes delete group script = /usr/sbin/groupdel "%g" delete user from group script = /usr/sbin/userdel "%u" "%g" delete user script = /usr/sbin/userdel "%u" domain master = Yes dos charset = CP852 guest account = pcguest interfaces = lo br0 tun0 load printers = No log file = /var/log/samba/log_%I_%U@%m logging = file logon script = smblogon.bat map to guest = Bad User max log size = 999500 max open files = 31000 ntlm auth = ntlmv1-permitted os level = 205 preferred master = Yes require strong key = No security = USER server role = standalone server server signing = if_required server string = Samba %v time server = Yes username map = /etc/samba/smbusers wins support = Yes idmap config * : backend = tdb access based share enum = Yes acl allow execute always = Yes hosts allow = 192.168.18. 192.168.72. 127. map archive = No [IPC$] guest ok = Yes guest only = Yes hosts allow = 192.168.18.0/24 192.168.72.0/24 127.0.0.0/8 hosts deny = 0.0.0.0/0 path = /etc/samba/IPC$ [homes] browseable = No comment = Home Directories read only = No [netlogon] comment = Network Logon Service guest ok = Yes path = /etc/samba/netlogon [tmp] comment = Temporary file space guest ok = Yes path = /mnt/data/Samba/tmp read only = No [public] comment = Public SW force create mode = 0664 force directory mode = 02775 force group = admin guest ok = Yes guest only = Yes path = /mnt/data/Samba/public valid users = @s15 @admin write list = @admin [gulas] comment = soubory create mask = 0660 directory mask = 0770 force create mode = 0664 force directory mode = 02770 force group = s15 path = /mnt/data/Samba/gulas valid users = @admin @s15 vfs objects = recycle write list = @admin @s15 recycle:touch_mtime = no recycle:touch = no recycle:exclude_dir = tmp, cache, profiles recycle:exclude = *.tmp, *.log, *.obj, ~*.*, *.bak recycle:versions = yes recycle:keeptree = yes recycle:repository = .recycle [data] comment = Centralni share S15 force create mode = 0660 force directory mode = 02770 force group = s15 path = /home/Samba/s15 read only = No valid users = @s15 @admin vfs objects = recycle write list = @s15 @admin recycle:touch_mtime = no recycle:touch = no recycle:exclude_dir = tmp, cache, profiles recycle:exclude = *.tmp, *.log, *.obj, ~*.*, *.bak recycle:versions = yes recycle:keeptree = yes recycle:repository = .recycle [s15] comment = Centralni share S15 force create mode = 0660 force directory mode = 02770 force group = s15 path = /mnt/data-ssd-s15/s15 read only = No valid users = @s15 @admin write list = @s15 @admin
(In reply to Alexander Bokovoy from comment #1) > Looks like vfs_recycle issue. Can you provide a configuration? If you have a > core file, that would be great as well... About vfs_recycle - smb log has, possibly related, error messages: ... [2024/05/20 09:13:17.558051, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 951647 did not clean up record 77F72CBA [2024/05/20 09:13:17.558074, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 77F72CBA failed: NT_STATUS_FATAL_APP_EXIT [2024/05/20 09:13:17.743417, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 950801 did not clean up record 58B4C488 [2024/05/20 09:13:17.743471, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 58B4C488 failed: NT_STATUS_FATAL_APP_EXIT [2024/05/20 09:13:17.768500, 1] ../../source3/smbd/smbXsrv_open.c:294(smbXsrv_open_global_verify_record) smbXsrv_open_global_verify_record: smbd 951647 did not clean up record 4B08C516 [2024/05/20 09:13:17.768552, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 4B08C516 failed: NT_STATUS_FATAL_APP_EXIT [2024/05/20 09:38:47.404315, 1] ../../source3/smbd/close.c:872(close_normal_file) Failed to disconnect durable handle for file Allplan_S15/Allplan 2023/Net/n0000017.lck: NT_STATUS_NOT_SUPPORTED - proceeding with normal close [2024/05/20 09:40:34.250038, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 6CA8F17F failed: NT_STATUS_NOT_FOUND [2024/05/20 09:42:46.919868, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 3F7508F6 failed: NT_STATUS_NOT_FOUND [2024/05/20 09:42:50.217792, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 4736DAE5 failed: NT_STATUS_NOT_FOUND [2024/05/20 09:42:50.218546, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 82A26FFD failed: NT_STATUS_NOT_FOUND [2024/05/20 09:42:50.219207, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for D9A6A5CA failed: NT_STATUS_NOT_FOUND [2024/05/20 09:43:59.642328, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 6E404E97 failed: NT_STATUS_NOT_FOUND [2024/05/20 09:44:06.970515, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 672FB076 failed: NT_STATUS_NOT_FOUND [2024/05/20 09:44:07.023257, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 149FF152 failed: NT_STATUS_NOT_FOUND [2024/05/20 09:44:07.034922, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for D8CD7450 failed: NT_STATUS_NOT_FOUND [2024/05/20 10:01:11.794795, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 0C11B4F7 failed: NT_STATUS_NOT_FOUND [2024/05/20 10:04:02.285684, 1, class=recycle] ../../source3/modules/vfs_recycle.c:275(recycle_create_dir) recycle_create_dir: recycle: mkdirat failed for .recycle/Projekty_S15 with error: Permission denied [2024/05/20 10:04:02.290102, 1, class=recycle] ../../source3/modules/vfs_recycle.c:275(recycle_create_dir) recycle_create_dir: recycle: mkdirat failed for .recycle/Projekty_S15 with error: Permission denied [2024/05/20 11:58:47.368373, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 4F5DB749 failed: NT_STATUS_NOT_FOUND [2024/05/20 11:58:59.005269, 1, class=recycle] ../../source3/modules/vfs_recycle.c:275(recycle_create_dir) recycle_create_dir: recycle: mkdirat failed for .recycle/Projekty_S15 with error: Permission denied [2024/05/20 11:59:16.516625, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 51603F6F failed: NT_STATUS_NOT_FOUND [2024/05/20 11:59:17.518253, 1] ../../source3/smbd/smbXsrv_open.c:1199(smb2srv_open_recreate_fn) smb2srv_open_recreate_fn: smbXsrv_open_global_verify_record for 4B09E0C9 failed: NT_STATUS_NOT_FOUND [2024/05/20 12:48:12.003485, 1, class=recycle] ../../source3/modules/vfs_recycle.c:275(recycle_create_dir) recycle_create_dir: recycle: mkdirat failed for .recycle/Projekty_S15 with error: Permission denied
Without getting a proper backtrace it is hard to say what condition is causing this issue in vfs_recycle, unfortunately. Judging by the code, it could be anything within recycle_unlink_internal()...
Alexander, thanks. I just installed abrt package on this machine, and I hope that after the crash the backtrace will be visible somewhere. And then I'll put it in here.
I will ask for patience - now crashdump was not created because bad package signing: May 24 10:18:12 s15 systemd[1]: systemd-coredump: Deactivated successfully. May 24 10:18:12 s15 abrt-server[581566]: Package 'samba' isn't signed with proper key May 24 10:18:12 s15 abrt-server[581566]: 'post-create' on '/var/spool/abrt/ccpp-2024-05-24-10:18:12.513093-532924' exited with 1 May 24 10:18:12 s15 abrt-server[581566]: Deleting problem directory '/var/spool/abrt/ccpp-2024-05-24-10:18:12.513093-532924'
Created attachment 2035854 [details] smbd crash backtrace 2024.05.31 Hi Alexander, I just added crash backtrace, as saved by abrtd - hope it is right one.
Thank you. Do you see the same crash in coredumpctl? Does 'coredumpctl dump /usr/sbin/smbd' produce a backtrace? Our goal here is to be able to see individual source lines in the output. If you have actual coredump file, I'd like to get access to it.
Created attachment 2035886 [details] coredumpctl dump /usr/sbin/smbd stdout xz archive Compressed stdout output of: 'coredumpctl dump /usr/sbin/smbd 2>/coredumpctl_dump_.usr.sbin.smbd.err|tee /coredumpctl_dump_.usr.sbin.smbd.dat|xz >/coredumpctl_dump_.usr.sbin.smbd.dat.xz' (raw size is ~ 50 MB) same command stderr output will be next attachment. And there is also (abrt created?) file (2.3 MB size): # ll -h /var/spool/abrt/ccpp-2024-05-31-07:40:14.775578-3374369/coredump.zst -rw-r-----. 1 root abrt 2.3M May 31 07:40 /var/spool/abrt/ccpp-2024-05-31-07:40:14.775578-3374369/coredump.zst which I may also upload, if help it...
Created attachment 2035888 [details] stderr output from 'coredumpctl dump /usr/sbin/smbd' cmd
Two more notes: 1) It seems as smbd crash occurs only with Win11 (x86_64) client, not at Win10 client. 2) On share with recycle vfs are created (from smbd, not by win client action) directories with weird, non-UTF8 names - e.g.: # cd /$SharePath # ll drwx--S---. 3 s15-3 s15 4096 May 31 13:55 ''$'\264\201' drwx--S---. 3 s15-3 s15 4096 May 31 15:18 '&' drwx--S---. 3 s15-3 s15 4096 May 31 14:55 '0W'$'\366''i'$'\240''U' drwx--S---. 3 s15-3 s15 4096 May 31 13:51 5 drwxrws--x. 21 root s15 4096 Apr 24 17:03 Instal_S15 drwx--S---. 3 s15-3 s15 4096 May 31 13:58 ''$'\340\251\366''i'$'\240''U' drwx--S---. 3 svajcr s15 4096 May 31 15:45 ''$'\340'';'$'\376''i'$'\240''U' drwx--S---. 3 s15-10 s15 4096 May 31 15:58 ''$'\300''J'$'\371''i'$'\240''U' drwxrws--x. 8 root s15 4096 Dec 28 14:40 Podklady_S15 drwx--S---. 3 s15-1 s15 4096 May 22 09:59 q drwx--S---. 3 s15-1 s15 4096 May 31 10:41 z drwx--S---. 3 svajcr s15 4096 May 31 16:38 'Н'$'\372''i'$'\240''U' Under these weird directories are (now with correct path names) files, which were perhaps deleted and shoul be under .recycle directory.
(gdb) print dir_exclude_list[0] $3 = 0x10040424d53fe <error: Cannot access memory at address 0x10040424d53fe> (gdb) up #11 recycle_unlink_internal (handle=0x55a069f92820, dirfsp=0x55a06a055830, smb_fname=0x55a069fb6a00, flags=0) at ../../source3/modules/vfs_recycle.c:564 564 if (matchdirparam(config->exclude_dir, path_name)) { (gdb) print *config $4 = {repository = 0x55a069f6a870 "\026", keeptree = true, versions = true, touch = false, touch_mtime = false, exclude = 0x55a069f8b160, exclude_dir = 0x55a069f8be60, noversions = 0x0, directory_mode = 448, subdir_mode = 448, minsize = 0, maxsize = 0} (gdb) print *config->exclude_dir $5 = 0x10040424d53fe <error: Cannot access memory at address 0x10040424d53fe> Something wrong with the configuration for exclude_dir. I'll look at this tomorrow morning.
My current working theory is that vfs_recycle's config is overwritten by some other VFS plugin. This is can be judged by the fact that config->exclude is pointing to a string list that contains IP addresses: (gdb) p config->exclude[0] $19 = 0x55a069f93e30 "192.168.18.0/24" (gdb) p config->exclude[1] $16 = 0x55a069f4d590 "192.168.72.0/24" (gdb) p config->exclude[2] $17 = 0x55a06a046980 "127.0.0.0/8" (gdb) p config->exclude[3] $18 = 0x0 This does correspond to the hosts allow = 192.168.18. 192.168.72. 127. from the globals section. Something is fishy in the way custom vfs handles are done. I don't remember all changes here since I wrote that code ~20 years ago but it looks like a wrong handle data is used by some other module....
Hi Alexander, now I don't know if something is expected from me? Can I help with something?
No, nothing expected from you at this moment. I have no time to look at this for next week or so. I'll check if Andreas or Pavel could look into it.
I think this is fixed with: https://bugzilla.samba.org/show_bug.cgi?id=15659 https://gitlab.com/samba-team/samba/-/merge_requests/3673
Samba fixes were merged today to git master, 4.20 updates are currently going under autobuild process and will eventually land in v-4-20 too. So we'll be able to provide an update here soon.