Since kernel-6.11.0-0.rc3.20240815git1fb918967b56.33.fc42 landed in Fedora Rawhide (we untagged it soon after), podman's tests are failing in several places where it expects to see output to the journal. Ed Santiago investigated the failure, and here are his conclusions: "It's an overflow of some sort. In case it helps, here's a podmanless reproducer: Window 1: # journalctl -f Window 2: # for i in $(seq 1 1111);do echo $i | systemd-cat;done The for loop will actually work fine the first time. Repeat (!!), and journalctl will stop partway through. No more logs after that. And this is fun: # systemctl restart systemd-journald.service ...and the bug no longer triggers. Repeat the for loop as long as you want, run podman tests, do whatever you want, all works fine. Reboot, and the bug reproduces again." Peter Robinson got the list of commits between the previous 32.fc42 build and the first failing build: 1c0e5881691a787a9399a99bff4d56ead6e75e91 KVM: SEV: uapi: fix typo in SEV_RET_INVALID_CONFIG 66155de93bcf4f2967e602a4b3bf7ebe58f34b11 KVM: x86: Disallow read-only memslots for SEV-ES and SEV-SNP (and TDX) 100bff23818eb61751ed05d64a7df36ce9728a4d perf/bpf: Don't call bpf_overflow_handler() for tracing events c9b35a6f4edea698a5bb4dd8029e7104ee0a3726 KVM: eventfd: Use synchronize_srcu_expedited() on shutdown 238d3d63d1e27c8d9733b48f7b682fc6aba86672 KVM: selftests: Add a testcase to verify x2APIC is fully readonly 4b7c3f6d04bd53f2e5b228b6821fb8f5d1ba3071 KVM: x86: Make x2APIC ID 100% readonly 15e1c3d65975524c5c792fcd59f7d89f00402261 KVM: x86: Use this_cpu_ptr() instead of per_cpu_ptr(smp_processor_id()) 6252690f7e1b173b86a4c27dfc046b351ab423e7 btrfs: fix invalid mapping of extent xarray state b098495e69491c2225681f43228312d32477217b KVM: x86: hyper-v: Remove unused inline function kvm_hv_free_pa_page() 810ee43d9cd245d138a2733d87a24858a23f577d Squashfs: sanity check symbolic link size e3786b29c54cdae3490b07180a54e2461f42144c 9p: Fix DIO read through netfs 2a0629834cd82f05d424bbc193374f9a43d1f87d vfs: Don't evict inode under the inode lru traversing context 46a6e10a1ab16cc71d4a3cab73e79aabadd6b8ea btrfs: send: allow cloning non-aligned extent if it ends at i_size ae1e766f623f7a2a889a0b09eb076dd9a60efbe9 btrfs: only run the extent map shrinker from kswapd tasks 31723c9542dba1681cc3720571fdf12ffe0eddd9 btrfs: tree-checker: reject BTRFS_FT_UNKNOWN dir type 42fac187b5c746227c92d024f1caf33bc1d337e4 btrfs: check delayed refs when we're checking if a ref exists cd2d00606553e631e9b5d11cca7da38fc95433e6 KVM: SVM: Fix an error code in sev_gmem_post_populate() 92b6c2f0076c50aaa919d16b595f34f3e9967bea KVM: SVM: Fix uninitialized variable bug 662c3e2db00f92e50c26e9dc4fe47c52223d9982 selftests/bpf: Add a test to verify previous stacksafe() fix bed2eb964c70b780fb55925892a74f26cb590b25 bpf: Fix a kernel verifier crash in stacksafe() fdad456cbcca739bae1849549c7a999857c56f88 bpf: Fix updating attached freplace prog in prog_array map 7b589a9b45ae32aa9d7bece597490e141198d7a6 netfs: Fix handling of USE_PGPRIV2 and WRITE_TO_CACHE flags 8e5ced7804cb9184c4a23f8054551240562a8eda netfs, ceph: Revert "netfs: Remove deprecated use of PG_private_2 as a second writeback flag" 86509e38a80da34d7800985fa2be183475242c8c file: fix typo in take_fd() comment 3b5bbe798b2451820e74243b738268f51901e7d0 pidfd: prevent creation of pidfds for kthreads 889ced4c9388785952d78d20d338bda2df209bb5 netfs: clean up after renaming FSCACHE_DEBUG config 64a7ce76fb901bf9f9c36cf5d681328fc0fd4b5a libfs: fix infinite directory reads for offset dir 42b0f8da3acc87953161baeb24f756936eb4d4b2 nsfs: fix ioctl declaration f71aa06398aabc2e3eaac25acdf3d62e0094ba70 fs/netfs/fscache_cookie: add missing "n_accesses" check 3f65f3c099bcb27949e712f39ba836f21785924a filelock: fix name of file_lease slab cache 98055bc3595500bcf2126b93b1595354bdb86a66 netfs: Fault in smaller chunks for non-large folio mappings 05a3d6e9307250a5911d75308e4363466794ab21 selinux: revert our use of vma_is_initial_heap() 9eb18136af9fe4dd688724070f2bfba271bd1542 KVM: arm64: vgic: Hold config_lock while tearing down a CPU interface 6dd1e4c045afa6a4ba5d46f044c83bd357c593c2 selinux: add the processing of the failure of avc_add_xperms_decision() ad518452fd263766946346324810f14bd8bb8b34 KVM: selftests: arm64: Correct feature test for S1PIE in get-reg-list 7e814a20f6da2bd2044b1a4682dd92a6f0df5a92 KVM: arm64: Tidying up PAuth code in KVM 01ab08cafeced7ae1d6c01a08218742c8182f8da KVM: arm64: vgic-debug: Exit the iterator properly w/o LPI 10f2ad032defe906240d0c3b62dcbceace96b230 KVM: arm64: Enforce dependency on an ARMv8.4-aware toolchain cff59d8631e1409ffdd22d9d717e15810181b32c s390/uv: Panic for set and remove shared access UVC errors 379d9af3f3da2da1bbfa67baf1820c72a080d1f1 selinux: fix potential counting error in avc_add_xperms_decision() 5a44bb061d04b0306f2aa8add761d86d152b9377 KVM: s390: fix validity interception issue when gisa is switched off 7fef1eb0b013eaa42019a95a08f71368e5a22dba docs: KVM: Fix register ID of SPSR_FIQ 19d837bc881b2f9f72f9eb506b46c2e2d983896d KVM: arm64: vgic: fix unexpected unlock sparse warnings 0aa34b37a78d063da58838b84b20a68a94d919fd KVM: arm64: fix kdoc warnings in W=1 builds 963a08e586bd45fd55f4c1752e98029ce83fc091 KVM: arm64: fix override-init warnings in W=1 builds 32b9a52f88a5713bf8a02dae66f2ad69705de69f KVM: arm64: free kvm->arch.nested_mmus with kvfree() Since we have a shortish list of suspect commits and an easy reproducer, this should be trivial to bisect, for someone who's set up to do kernel bisections. (I am not). Anyhow, since at least four folks are interested in it now, filing a bug for convenient tracking. Note though that the affected kernel builds are being blocked in gating, so this is not affecting actual Rawhide or F41 users ATM. Reproducible: Always
This sounds like it may be the same as / related to bug 2305270 - the commit I bisected it to is in the range of suspected commits above and it's also about systemd-journald.
(In reply to Ondrej Mosnáček from comment #1) > This sounds like it may be the same as / related to bug 2305270 - the commit > I bisected it to is in the range of suspected commits above and it's also > about systemd-journald. Yep, looks the same to me. *** This bug has been marked as a duplicate of bug 2305270 ***