Description of problem: I tried to debug something (probably unrelated) and I wanted to do some changes in systemd scriptlets. But i was not able to rebuilds srpm in copr Version-Release number of selected component (if applicable): systemd-245~rc1-2.fc33.src.rpm How reproducible: Deterministic Steps to Reproduce: 1. dnf install -y fedpkg 2. fedpkg clone --anonymous systemd 3. cd systemd/ 4. git log -1 5. # create project in copr copr.fedorainfracloud.org 6. # rebuild systemd-245~rc1-2.fc33.src.rpm created in the current directory Actual results: sh-5.0$ fedpkg clone --anonymous systemd Cloning into 'systemd'... remote: Counting objects: 9867, done. remote: Compressing objects: 100% (7026/7026), done. remote: Total 9867 (delta 4766), reused 7225 (delta 2824) Receiving objects: 100% (9867/9867), 7.04 MiB | 823.00 KiB/s, done. Resolving deltas: 100% (4766/4766), done. sh-5.0$ cd systemd/ sh-5.0$ git log -1 commit 3666983037e56865f622c2bc1aa30399b75bd725 (HEAD -> master, origin/master, origin/f32, origin/HEAD) Author: Zbigniew Jędrzejewski-Szmek <zbyszek.pl> Date: Tue Feb 11 14:16:34 2020 +0100 Run tests with a timeout multiplier Tests fail to pass on s390x, and this seems to be just a timeout. sh-5.0$ fedpkg srpm Downloading systemd-245-rc1.tar.gz ######################################################################## 100.0% setting SOURCE_DATE_EPOCH=1581033600 Wrote: /tmp/data/systemd/systemd-245~rc1-2.fc33.src.rpm ### check build log sh-5.0$ curl https://download.copr.fedorainfracloud.org/results/lslebodn/test_rawhide/fedora-rawhide-x86_64/01239669-systemd/build.log.gz | zgrep FAIL % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 109k 100 109k 0 0 1079k 0 --:--:-- --:--:-- --:--:-- 1090k Installing man/SD_BUS_ERROR_AUTH_FAILED.3 to /builddir/build/BUILDROOT/systemd-245~rc1-2.fc32.x86_64/usr/share/man/man3 Installing man/SD_BUS_ERROR_FAILED.3 to /builddir/build/BUILDROOT/systemd-245~rc1-2.fc32.x86_64/usr/share/man/man3 285/490 test-mountpoint-util FAIL 0.47 s (killed by signal 6 SIGABRT) Expected results: No failure or SIGABRT in build.log Additional info: sample build https://copr.fedorainfracloud.org/coprs/lslebodn/test_rawhide/build/1239669/ https://download.copr.fedorainfracloud.org/results/lslebodn/test_rawhide/fedora-rawhide-x86_64/01239669-systemd/build.log.gz You can from copr https://download.copr.fedorainfracloud.org/results/lslebodn/test_rawhide/fedora-rawhide-x86_64/01239669-systemd/systemd-245~rc1-2.fc32.src.rpm
Actually it is simpler to rebuild srpm from koji. https://kojipkgs.fedoraproject.org//packages/systemd/245~rc1/2.fc32/src/systemd-245~rc1-2.fc32.src.rpm e.g. https://copr.fedorainfracloud.org/coprs/lslebodn/test_rawhide/build/1239672/ But it does not contain commit 3666983037e56865f622c2bc1aa30399b75bd725
Yeah, it fails this way on one of my machines too. I think it's a kernel bug that has long been fixed (I haven't seen it since F30, but it reproduced reliably with linux-5.2.11-100.fc29.x86_64. Unfortunately that build log gives 404 now. Do you know what kernel version was/is running in copr?
https://github.com/systemd/systemd/pull/14970
Sorry, wrong bug.
(In reply to Zbigniew Jędrzejewski-Szmek from comment #2) > Yeah, it fails this way on one of my machines too. I think it's a kernel bug > that > has long been fixed (I haven't seen it since F30, but it reproduced reliably > with > linux-5.2.11-100.fc29.x86_64. > > Unfortunately that build log gives 404 now. Do you know what kernel version > was/is > running in copr? Here is a fresh rebuild https://download.copr.fedorainfracloud.org/results/lslebodn/test_rawhide/fedora-rawhide-x86_64/01259489-systemd/ I briefly checked and I was not able to find anything about version of kernel in logs. copr devs might know more details.
(In reply to Zbigniew Jędrzejewski-Szmek from comment #2) > Yeah, it fails this way on one of my machines too. I think it's a kernel bug > that > has long been fixed (I haven't seen it since F30, but it reproduced reliably > with > linux-5.2.11-100.fc29.x86_64. > BTW, feel free to close it if it is a kernel bug. I hit this issue as part of debugging BZ1803240
There's kernel 5.0.16, we did not have to update the cloud images for builders for some time already.
The issue appears to occur when name_to_handle_at is not available. It is possible that our fallback code is buggy. Let's leave this open for now.
I was able to reproduce this on F32 with the latest kernel few days back.
$ fedpkg srpm # fedora master branch Downloading systemd-245.4.tar.gz ######################################################################## 100.0% setting SOURCE_DATE_EPOCH=1585699200 Wrote: /tmp/systemd/systemd-245.4-1.fc33.src.rpm $ mock -r fedora-rawhide-x86_64 /tmp/systemd/systemd-245.4-1.fc33.src.rpm ... 283/490 test-libmount OK 0.016834259033203125 s 284/490 test-mount-util OK 0.014326333999633789 s 285/490 test-mountpoint-util FAIL 0.637660026550293 s (killed by signal 6 SIGABRT) 286/490 test-exec-util OK 1.1383476257324219 s 287/490 test-hexdecoct OK 0.014191627502441406 s ... $ cat ~/.config/mock.cfg config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['keep_mounted'] = True config_opts['nosync'] = True $ cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.6.3-300.fc32.x86_64 ...
Apparently I'm already CC'ed, but wanted to report that this happens also with the RHEL 8.2 systemd, as well as the latest rawhide (fc33) systemd. Rawhide output: https://copr.fedorainfracloud.org/coprs/cyberpear/systemd/build/1367589/ 285/491 test-mountpoint-util FAIL 0.4175901412963867 s (killed by signal 6 SIGABRT) Output for EPEL 8, RHEL 8.2 systemd: https://copr.fedorainfracloud.org/coprs/cyberpear/systemd/build/1367207/ 133/314 test-mount-util FAIL 0.42 s (killed by signal 6 SIGABRT) --- command --- SYSTEMD_KBD_MODEL_MAP='/builddir/build/BUILD/systemd-239/src/locale/kbd-model-map' PATH='/builddir/build/BUILD/systemd-239/x86_64-redhat-linux-gnu:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin' SYSTEMD_LANGUAGE_FALLBACK_MAP='/builddir/build/BUILD/systemd-239/src/locale/language-fallback-map' /builddir/build/BUILD/systemd-239/x86_64-redhat-linux-gnu/test-mount-util --- stderr --- mnt id of /proc/keys is 423 mnt id of /dev/loop9 is 518 mnt id of /proc/sys/kernel/random/boot_id is 433 mnt id of /proc/acpi is 426 mnt id of /proc is 524 mnt id of /dev/loop10 is 509 mnt id of /proc/kcore is 422 mnt id of /run/systemd/nspawn/incoming is 520 mnt id of /proc/timer_list is 425 mnt id of /proc/fs is 428 mnt id of /run is 519 mnt id of /dev is 503 mnt id of /sys/fs/cgroup is 432 mnt id of /dev/pts is 505 mnt id of /dev/loop6 is 515 mnt id of /dev/loop2 is 511 mnt id of /var/cache/yum is 498 mnt id of /dev/loop5 is 514 mnt id of /sys is 501 mnt id of /proc/sys is 420 mnt id of /dev/loop1 is 508 mnt id of /etc/resolv.conf is 521 mnt id of /dev/loop-control is 506 mnt id of /dev/loop7 is 516 mnt id of /var/cache/dnf is 499 mnt id of /proc/sysrq-trigger is 424 mnt id of /dev/loop8 is 517 mnt id of /proc/filesystems is 524 Assertion 'path_equal_ptr(hashmap_get(h, INT_TO_PTR(mnt_id2)), p)' failed at ../src/test/test-mount-util.c:77, function test_mnt_id(). Aborting. -------
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle. Changing version to 33.
I managed to find a reliable reproducer on one of the fedora test machines. AFAICT, it's a kernel bug. (gdb) !uname -r 5.6.19-200.fc31.ppc64le (gdb) !cat /proc/self/mountinfo 697 664 253:0 /var/lib/mock/fedora-31-ppc64le/root / rw,relatime shared:298 master:1 - xfs /dev/mapper/fedora_rh--power--vm14-root rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota 698 697 253:0 /var/cache/mock/fedora-31-ppc64le/yum_cache /var/cache/yum rw,relatime shared:299 master:1 - xfs /dev/mapper/fedora_rh--power--vm14-root rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota 699 697 253:0 /var/cache/mock/fedora-31-ppc64le/dnf_cache /var/cache/dnf rw,relatime shared:300 master:1 - xfs /dev/mapper/fedora_rh--power--vm14-root rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota 700 697 0:32 /mock-selinux-plugin.7me9bfpi /proc/filesystems rw,nosuid,nodev shared:301 master:18 - tmpfs tmpfs rw,seclabel <========================================================== 701 697 0:41 / /sys ro,nosuid,nodev,noexec,relatime shared:302 - sysfs sysfs ro,seclabel 702 701 0:21 / /sys/fs/selinux ro,nosuid,nodev,noexec,relatime shared:306 master:8 - selinuxfs selinuxfs rw 703 697 0:42 / /dev rw,nosuid shared:303 - tmpfs tmpfs rw,seclabel,mode=755 704 703 0:43 / /dev/shm rw,nosuid,nodev shared:304 - tmpfs tmpfs rw,seclabel 705 703 0:45 / /dev/pts rw,nosuid,noexec,relatime shared:307 - devpts devpts rw,seclabel,gid=5,mode=620,ptmxmode=666 706 703 0:6 /btrfs-control /dev/btrfs-control rw,nosuid shared:308 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 707 703 0:6 /loop-control /dev/loop-control rw,nosuid shared:309 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 708 703 0:6 /loop0 /dev/loop0 rw,nosuid shared:310 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 709 703 0:6 /loop1 /dev/loop1 rw,nosuid shared:311 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 710 703 0:6 /loop10 /dev/loop10 rw,nosuid shared:312 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 711 703 0:6 /loop11 /dev/loop11 rw,nosuid shared:313 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 712 703 0:6 /loop2 /dev/loop2 rw,nosuid shared:314 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 713 703 0:6 /loop3 /dev/loop3 rw,nosuid shared:315 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 714 703 0:6 /loop4 /dev/loop4 rw,nosuid shared:316 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 715 703 0:6 /loop5 /dev/loop5 rw,nosuid shared:317 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 716 703 0:6 /loop6 /dev/loop6 rw,nosuid shared:318 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 717 703 0:6 /loop7 /dev/loop7 rw,nosuid shared:319 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 718 703 0:6 /loop8 /dev/loop8 rw,nosuid shared:320 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 719 703 0:6 /loop9 /dev/loop9 rw,nosuid shared:321 master:9 - devtmpfs devtmpfs rw,seclabel,size=4107840k,nr_inodes=64185,mode=755 720 697 0:44 / /run rw,nosuid,nodev shared:305 - tmpfs tmpfs rw,seclabel,mode=755 721 720 0:25 /systemd/nspawn/propagate/9cc8a155d0244558b273f773d2b92142 /run/systemd/nspawn/incoming ro master:12 - tmpfs tmpfs rw,seclabel,mode=755 722 697 0:32 /mock-resolv.dvml91hp /etc/resolv.conf rw,nosuid,nodev shared:322 master:18 - tmpfs tmpfs rw,seclabel 725 697 0:47 / /proc rw,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 603 725 0:47 /sys /proc/sys ro,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 604 725 0:44 /systemd/inaccessible/reg /proc/kallsyms ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 605 725 0:44 /systemd/inaccessible/reg /proc/kcore ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 606 725 0:44 /systemd/inaccessible/reg /proc/keys ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 607 725 0:44 /systemd/inaccessible/reg /proc/sysrq-trigger ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 608 725 0:44 /systemd/inaccessible/reg /proc/timer_list ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 609 725 0:47 /bus /proc/bus ro,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 610 725 0:47 /fs /proc/fs ro,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 611 725 0:47 /irq /proc/irq ro,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 612 725 0:47 /scsi /proc/scsi ro,nosuid,nodev,noexec,relatime shared:323 - proc proc rw 613 703 0:46 / /dev/mqueue rw,nosuid,nodev,noexec,relatime shared:324 - mqueue mqueue rw,seclabel 614 701 0:26 / /sys/fs/cgroup rw,nosuid,nodev,noexec,relatime shared:325 - cgroup2 cgroup rw,seclabel,nsdelegate 615 603 0:44 /.#proc-sys-kernel-random-boot-id4fbdce67af46d1c2//deleted /proc/sys/kernel/random/boot_id ro,nosuid,nodev,noexec shared:305 - tmpfs tmpfs rw,seclabel,mode=755 616 725 0:44 /.#proc-sys-kernel-random-boot-id4fbdce67af46d1c2//deleted /proc/sys/kernel/random/boot_id rw,nosuid,nodev shared:305 - tmpfs tmpfs rw,seclabel,mode=755 617 725 0:44 /.#proc-kmsg5b7a8bcfe6717139//deleted /proc/kmsg rw,nosuid,nodev shared:305 - tmpfs tmpfs rw,seclabel,mode=755 The test process does name_to_handle_at("/proc/filesystems") which returns -EOPNOTSUPP, and then openat(AT_FDCWD, "/proc/filesystems") which returns 4, and then read(4, ...) which gives "pos:\t0\nflags:\t012100000\nmnt_id:\t725\n" which is clearly inconsistent with "700" in /proc/self/mountinfo. We could either drop the fallback path (and fail name_to_handle_at() is not avaiable) or ignore the error in the test. Not sure what is better.
https://github.com/systemd/systemd/pull/17050
FEDORA-2020-0d29e88946 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-0d29e88946
FEDORA-2020-dc4f0fb907 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-dc4f0fb907
FEDORA-2020-0d29e88946 has been pushed to the Fedora 32 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-0d29e88946` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-0d29e88946 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2020-dc4f0fb907 has been pushed to the Fedora 31 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-dc4f0fb907` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-dc4f0fb907 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2020-0d29e88946 has been pushed to the Fedora 32 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2020-dc4f0fb907 has been pushed to the Fedora 31 stable repository. If problem still persists, please make note of it in this bug report.