Description of problem: This core dumped at startup, but it happens anytime find is run on my system: [root@kelesis ~]# find / -type d -name "*remmina*" -ls 6309 0 drwxr-x--- 2 scott.a.clark scott.a.clark 6 Jan 9 07:30 /home/scott.a.clark/.cache/remmina 269578515 4 drwx------ 2 scott.a.clark scott.a.clark 4096 May 11 2017 /home/scott.a.clark/.local/share/remmina 537325403 0 drwx------ 2 scott.a.clark scott.a.clark 26 Mar 9 08:00 /home/scott.a.clark/.config/remmina Aborted (core dumped) After I rebooted and ran fsck, it seems to work. Possible missing journal info after a rebuild? Version-Release number of selected component: 1:findutils-4.6.0-16.fc27 Additional info: reporter: libreport-2.9.3 backtrace_rating: 4 cmdline: find . -type f -name *.txt crash_function: leave_dir executable: /usr/bin/find journald_cursor: s=78425573b8a941b59247848bb5f8b4c7;i=878e;b=b4f269965a0b40879001b6430a009dad;m=36b887bcc;t=567c7a0723a5a;x=bc730be344ff63c8 kernel: 4.15.9-300.fc27.x86_64 rootdir: / runlevel: N 5 type: CCpp uid: 1000 Truncated backtrace: Thread no. 1 (5 frames) #2 leave_dir at ../../../gl/lib/fts-cycle.c:136 #3 fts_build at ../../../gl/lib/fts.c:1380 #4 fts_read at ../../../gl/lib/fts.c:959 #5 find at ../../find/ftsfind.c:576 #6 process_all_startpoints at ../../find/ftsfind.c:638
Created attachment 1410162 [details] File: backtrace
Created attachment 1410163 [details] File: cgroup
Created attachment 1410164 [details] File: core_backtrace
Created attachment 1410165 [details] File: cpuinfo
Created attachment 1410166 [details] File: dso_list
Created attachment 1410167 [details] File: environ
Created attachment 1410168 [details] File: limits
Created attachment 1410169 [details] File: maps
Created attachment 1410170 [details] File: mountinfo
Created attachment 1410171 [details] File: open_fds
Created attachment 1410172 [details] File: proc_pid_status
This is the same backtrace as in bug #1544429, which was confirmed to be fixed in findutils-4.6.0-16.fc27. This could be caused by some inconsistency in the file system data as you suggest. According to attachment #1410170 [details] /home/scott.a.clark/Documents is a cifs mount. Could it be the cause of the crash? From the backtrace it is not clear which path exactly was being processed while the assertion failure was triggered. Could you please capture strace's output if the crash reoccurs?
I unmounted the CIFS and still got the issue. strace shows that the error is in the home partition. I am attaching the strace output here.
Created attachment 1410476 [details] strace on findutils
Do I understand it correctly that the crash is still reproducible? Would you be able to run a modified build of findutils and attach its debugging output?
Yes, still reproducible. This is a workstation laptop, so I can definitely install a debugging build of findutils. I'm beginning to think, though, that I may have filesystem problems on this installation. Maybe a debug would bear that out?
That is difficult to tell at this point. You said that you ran fsck. Did it find any errors? Was it able to fix them? Please try findutils-4.6.0-16.1.fc27 from the following copr: https://copr.fedorainfracloud.org/coprs/kdudka/findutils-debug The following commands will do it for you: # dnf copr enable kdudka/findutils-debug # dnf update findutils Then please re-run the 'find' command with -D fts: $ find -D fts ...
OK, made some progress. I do believe the CIFS mount has something to do with this. I got a core dump when it was mounted, but unmounting it I got no errors, even scanning the entire filesystem. Excerpt: On CIFS: === pre-pop ========== 7 2: 5 2-leaving: /home/scott.a.clark/Public find: ===== check ===== cwd: 6 fts_read: p=/home/scott.a.clark/Public entering: /home/scott.a.clark/Documents fts_read: p=/home/scott.a.clark/Documents 4-leaving: /home/scott.a.clark/Documents find: ===== check ===== cwd: 7 === post-push ========== 5 2: 5 fts_read: p=/home/scott.a.clark/Documents/UnifiedGlobalServerNaming.xls fts_read: p=/home/scott.a.clark/Documents/Pictures 4-leaving: /home/scott.a.clark/Documents/Pictures Aborted CIFS Unmounted: === post-push ========== 5 3: 5 === pre-pop ========== 7 3: 5 2-leaving: /var/yp find: ===== check ===== cwd: 6 fts_read: p=/var/yp fts_read: p=/var/.updated 3-leaving: /var find: ===== check ===== cwd: 5 fts_read: p=/var === post-push ========== 6 3: 6 3-leaving: / find: ===== check ===== cwd: -1 fts_read: p=/
This looks suspicious. The debugging output should contain the line: entering: /home/scott.a.clark/Documents/Pictures ... before the line: 4-leaving: /home/scott.a.clark/Documents/Pictures Could you please retry it with 'find -noleaf ...' ? According to attachment #1410170 [details] /home/scott.a.clark/Documents is a cifs mount. However, in attachment #1410476 [details] fstatfs() returns f_type=XFS_SB_MAGIC for that directory. Is the file system auto-mounted?
Yes, the CIFS is auto-mounted via fstab. I can consistently reproduce core dump when using find on the Documents mount; if I unmount, it works. [root@kelesis ~]# find / -type f -name mime find: ‘/run/user/1000/doc’: Permission denied find: ‘/run/user/1000/gvfs’: Permission denied /var/lib/docker/overlay2/66c58816cdaee70ce6f6adea90d092116b3febf2a184c47cb906dc9d2e5354da/diff/usr/share/terminfo/m/mime /var/lib/docker/overlay2/c9dfb741a113d5fbe51aab168691e30059bac9d015c5d33ae9ba66b667c7e67e/diff/usr/share/terminfo/m/mime [root@kelesis ~]# mount /home/scott.a.clark/Documents/ [root@kelesis ~]# find / -type f -name mime Aborted (core dumped)
(In reply to Scott Clark from comment #20) > Yes, the CIFS is auto-mounted via fstab. By auto-mounted, I meant "lazily" mounted on access by the automount service. Could you please retry it with 'find -noleaf ...' ?
No, it's not using the automount service since it requires either being on my corporate network or VPN. It's just an fstab mount that either mounts at boot or I mount manually after connecting. Here's the noleaf run: [root@kelesis ~]# find / -noleaf -type f -name mime find: ‘/run/user/1000/doc’: Permission denied find: ‘/run/user/1000/gvfs’: Permission denied /var/lib/docker/overlay2/66c58816cdaee70ce6f6adea90d092116b3febf2a184c47cb906dc9d2e5354da/diff/usr/share/terminfo/m/mime /var/lib/docker/overlay2/c9dfb741a113d5fbe51aab168691e30059bac9d015c5d33ae9ba66b667c7e67e/diff/usr/share/terminfo/m/mime This succeeded with the CIFS mount active.
Could you please try findutils-4.6.0-18.1.fc27 from my findutils-debug copr? # dnf copr enable kdudka/findutils-debug # dnf update findutils Does it work now without the -noleaf option, too?
Seeing this same bug, although bizarrely, it seems like it's happening where a CIFS mount /used/ to be, not where it is now?
(In reply to Geoffrey Wiseman from comment #24) > Seeing this same bug, although bizarrely, it seems like it's happening where > a CIFS mount /used/ to be, not where it is now? Geoffrey, which version of findutils are you referring to? The testing build mentioned in comment #23 or a stable release?
Fedora 27 before and after dnf upgrade, but not including the testing build. Would you like me to try the testing build as well?
(In reply to Geoffrey Wiseman from comment #26) > Would you like me to try the testing build as well? That would be much appreciated!
Kamil, I'm still getting a core dump when CIFS is mounted and -noleaf is not specified. [root@kelesis ~]# find / -type f -name mime Aborted (core dumped) [root@kelesis ~]# find / -noleaf -type f -name mime find: ‘/run/user/1000/gvfs’: Permission denied /var/lib/docker/overlay2/66c58816cdaee70ce6f6adea90d092116b3febf2a184c47cb906dc9d2e5354da/diff/usr/share/terminfo/m/mime /var/lib/docker/overlay2/c9dfb741a113d5fbe51aab168691e30059bac9d015c5d33ae9ba66b667c7e67e/diff/usr/share/terminfo/m/mime /var/lib/docker/overlay2/8f945510efb85334fcf5d3a6c1274ed1b5f6d6805237476ca4431ae06e418e48/diff/usr/share/terminfo/m/mime /var/lib/docker/overlay2/bcd450a5c349b6e51cd3c07af97337135376425850fb29f361b7b7452b4748aa/diff/usr/share/terminfo/m/mime [root@kelesis ~]# dnf list findutils Last metadata expiration check: 2:43:50 ago on Mon 09 Apr 2018 09:42:15 AM CDT. Installed Packages findutils.x86_64 1:4.6.0-18.1.fc27 @kdudka-findutils-debug Available Packages findutils.src 1:4.6.0-18.1.fc27 kdudka-findutils-debug
Thanks for reply! Could you please attach a pair of strace outputs with/without the -noleaf option enabled?
Created attachment 1419687 [details] straces for find with and without noleaf in a .tgz
Same results as Scott -- with `-noleaf` I don't get a core dump without it, I do. And, yes, looks like it's in the folder with the CIFS mount that I get the core dump -- not the folder where it used to be. I couldn't tell from the output, but I narrowed down the folder structure used in the tests until it broke and it's definitely in `/mnt`.
Created attachment 1419866 [details] sclark stacktrace 4-10
Thank you for providing the strace outputs! I will forward it upstream and, in case they do not see an obvious bug, I will try to create a local reproducer and debug it myself.
Paul Eggert from upstream has identified some bugs in the fts gnulib module and created a patch to fix them. Could you pleas retry it with findutils-4.6.0-18.2 from my copr: # dnf copr enable kdudka/findutils-debug # dnf update findutils Does it work now without the -noleaf option, too?
Yes, this seems to make the core dump vanish for me at least.
(In reply to Geoffrey Wiseman from comment #35) > Yes, this seems to make the core dump vanish for me at least. Thanks for confirmation! upstream commits: http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=2e53df54 http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=81b8c0d3
Sorry for the delay, I can also confirm that the latest update works without the -noleaf option.
Thanks for the confirmation! Official updates coming soon...
findutils-4.6.0-19.fc27 coreutils-8.27-21.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-72c7737ac6
coreutils-8.29-6.fc28 findutils-4.6.0-19.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-8998b98f9c
coreutils-8.27-21.fc27, findutils-4.6.0-19.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-72c7737ac6
coreutils-8.29-6.fc28, findutils-4.6.0-19.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-8998b98f9c
Maybe not available yet? ``` $ sudo dnf install coreutils findutils --enablerepo=updates-testing Last metadata expiration check: 0:01:30 ago on Sat 21 Apr 2018 04:16:46 PM EDT. Package coreutils-8.27-20.fc27.x86_64 is already installed, skipping. Package findutils-1:4.6.0-18.2.fc27.x86_64 is already installed, skipping. Dependencies resolved. Nothing to do. Complete! ```
(In reply to Geoffrey Wiseman from comment #43) > Maybe not available yet? It takes some time till the mirrors are synchronized. You can always install the packages directly from Koji: $ sudo dnf install https://kojipkgs.fedoraproject.org//packages/coreutils/8.27/21.fc27/x86_64/coreutils-8.27-21.fc27.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/coreutils/8.27/21.fc27/x86_64/coreutils-common-8.27-21.fc27.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/findutils/4.6.0/19.fc27/x86_64/findutils-4.6.0-19.fc27.x86_64.rpm
How long is 'some time'? :) Still seems not to be available for me. It's not super-important, even the original core dump wasn't causing that much problem, and the test build has fixed that, so I'm not blocked by this, I'm just curious how long it takes, 'cause it seems to be significantly longer than I expected.
Maybe a stupid question but did you try 'dnf update' instead of 'dnf install'?
coreutils-8.27-21.fc27, findutils-4.6.0-19.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
coreutils-8.29-6.fc28, findutils-4.6.0-19.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.