Description of problem: When opening lots of videos in Firefox simultaneously in different tabs, Firefox hangs for 10 seconds or so and the journal is full of the error messages: Oct 11 02:38:14 ++++ pipewire-pulse[1603]: default: stream 0x5617a8da3860: can't make node: Too many open files Oct 11 02:38:15 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files [++++ redacted machine name] The videos don't have to autoplay - just the fact that many tabs contain a video seems enough. The number of tabs to trigger this is about the 20 mark. Version-Release number of selected component (if applicable): cairomm-1.14.2-14.fc34.x86_64 Additional info: This started with one minute of upgrading to cairomm-1.14.2-14. I ran for four days with this version, before frustration at the freezes. Investigating, the problem began within one minute (yes, really one minute) of the upgrade of cairomm to cairomm-1.14.2-14. I downgraded to the previous version of cairomm-1.14.2-8 and have had no problem since, so I'm confident that this is the cause.
This is a very interesting report. I can verify that while there were a number of changes to documentation packaging and spec file syntax between cairomm-1.14.2-8 and cairomm-1.14.2-14, there were absolutely no changes to meson options or compiler flags, and the source tarball is exactly the same. Unfortunately, that doesn’t really lead me to anything I could revert. If indeed the difference lies in the cairomm package, it seems the only place something could have changed that might cause this is in a dependency’s header files or in the toolchain (compiler/linker). I rebuilt cairomm-1.14.2-8 exactly, but using the current toolchain and dependencies in Fedora 34: https://koji.fedoraproject.org/koji/taskinfo?taskID=77465388. It would be interesting to know if you encounter the same problem after manually installing those RPMs. Also, does it matter whether you reboot after the cairomm upgrade/downgrade? I am eager to help, but I unfortunately have very little to go on here at the moment.
Ben Thanks for looking into this. The koji link is to cairomm-1.14.2-8, and as you can see from the dnf below, I did download that from koji manually to downgrade. Looking at my Firefox history, I had many video tabs open *before* I installed cairomm-1.14.2-14 This must be why I got a lock-up in Firefox immediately I installed the new version. dnf history list cairomm ID | Command line | Date and time | Action(s) | Altered --------------------------------------------------------------------------------------------------------------------- 1903 | downgrade /home/jonathan/Desktop/cairomm-1.14.2-8.fc34.x86_64. | 2021-10-14 16:20 | Downgrade | 1 < 1898 | upgrade --refresh | 2021-10-10 21:42 | Upgrade | 3 >< You can see the exact same date/time in the journal. Oct 10 21:42:53 ++++ pipewire-pulse[1603]: default: stream 0x5617af12db20: can't make node: Too many open files Oct 10 21:42:58 ++++ pipewire-pulse[1603]: default: stream 0x5617af2e13e0: can't make node: Too many open files Oct 10 21:43:23 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:43:23 ++++ pipewire-pulse[1603]: default: stream 0x5617af135630: can't make node: Too many open files Oct 10 21:43:28 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:43:28 ++++ pipewire-pulse[1603]: default: stream 0x5617b1ef5900: can't make node: Too many open files Oct 10 21:43:53 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:43:53 ++++ pipewire-pulse[1603]: default: stream 0x5617aed30ce0: can't make node: Too many open files Oct 10 21:43:58 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:43:58 ++++ pipewire-pulse[1603]: default: stream 0x5617b0b04630: can't make node: Too many open files Oct 10 21:44:23 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:44:23 ++++ pipewire-pulse[1603]: default: stream 0x5617ab20f900: can't make node: Too many open files Oct 10 21:44:28 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:44:28 ++++ pipewire-pulse[1603]: default: stream 0x5617b06d8e20: can't make node: Too many open files Oct 10 21:44:53 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:44:53 ++++ pipewire-pulse[1603]: default: stream 0x5617af132810: can't make node: Too many open files Oct 10 21:45:23 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 10 21:45:23 ++++ pipewire-pulse[1603]: default: stream 0x5617af2e87c0: can't make node: Too many open files Oct 11 01:46:26 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 11 01:46:26 ++++ pipewire-pulse[1603]: default: stream 0x5617ace0c910: can't make node: Too many open files Oct 11 01:46:56 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 11 01:46:56 ++++ pipewire-pulse[1603]: default: stream 0x5617a89855b0: can't make node: Too many open files Oct 11 01:52:16 ++++ pipewire-pulse[1603]: mod.adapter: can't load spa node: Too many open files Oct 11 01:52:16 ++++ pipewire-pulse[1603]: mod.adapter: usage: node.name=<string> Oct 11 01:52:16 ++++ pipewire-pulse[1603]: default: stream 0x5617ad763ea0: can't make node: Invalid argument Oct 11 01:52:34 ++++ pipewire-pulse[1603]: default: mempool 0x56177f201b00: Failed to create memfd: Too many open files Oct 11 01:52:34 ++++ pipewire-pulse[1603]: default: stream 0x56177f336c70: can't make node: Too many open files Oct 11 02:12:45 ++++ pipewire-pulse[1603]: default: stream 0x5617a261fe40: can't make node: Too many open files Oct 11 02:13:28 ++++ pipewire-pulse[1603]: default: stream 0x5617a2620ed0: can't make node: Too many open files Oct 11 02:13:58 ++++ pipewire-pulse[1603]: default: stream 0x5617a6bce7b0: can't make node: Too many open files Oct 11 02:14:34 ++++ pipewire-pulse[1603]: default: stream 0x5617a6bcf840: can't make node: Too many open files Oct 11 02:15:47 ++++ pipewire-pulse[1603]: default: stream 0x5617ace0a1a0: can't make node: Too many open files Oct 11 02:16:17 ++++ pipewire-pulse[1603]: default: stream 0x5617ace0b230: can't make node: Too many open files Oct 11 02:16:37 ++++ pipewire-pulse[1603]: default: stream 0x5617a8da27d0: can't make node: Too many open files Oct 11 02:16:48 ++++ pipewire-pulse[1603]: default: stream 0x5617a8da3860: can't make node: Too many open files I attach also a verbose journal of the above - perhaps the source-code references there can give a clue as to what is going on. Perhaps I should mention that my DE is cinnamon - though that shouldn't be relevant. If I can help more in any way, let me know.
Created attachment 1834434 [details] journalctl --since="2021-10-10 21:42" --until="2021-10-11 02:17" -p3 -o verbose
Ben Should have said that rebooting made no difference. Errors/freezes immediately after installation and the same errors/freezes after reboot.
> The koji link is to cairomm-1.14.2-8, and as you can see from the dnf below, I did download that from koji manually to downgrade. Please note that the cairomm-1.14-2-8 build I linked is > $ sha256sum -b cairomm-1.14.2-8.fc34.x86_64.rpm > 83a520a2d44c70c32f891a775f18d0d97c91217ef7837bdd5679da2427f395f3 *cairomm-1.14.2-8.fc34.x86_64.rpm and it is *not* the same as the one you might have found in https://koji.fedoraproject.org/koji/packageinfo?packageID=1437, but a fresh build done today from the same dist-git commit (https://src.fedoraproject.org/rpms/cairomm/c/57b849212d845c4915541582feb0cc3dfbcc0459?branch=rawhide) as the original cairomm-1.14.2-8 build. So, if something changed external to the cairomm package to cause the problem you are seeing, in theory this 1.14.2-8 build ought to exhibit the same problem. I know that’s a little confusing—did I explain well enough? ----- I tried opening and staring thirty music videos on YouTube on my desktop machine (Fedora 34, fully updated, GNOME/Wayland) and produced an interesting jumble of sounds but so far couldn’t manage to reproduce this myself.
Sorry Ben Rereading your comment 1, I realized that I interpreted: 'I rebuilt cairomm-1.14.2-8 exactly' as '[In order to build cairomm-1.14.2-14,] I rebuilt cairomm-1.14.2-8 exactly' Anyway, I installed your rebuild with: dnf reinstall https://kojipkgs.fedoraproject.org//work/tasks/5606/77465606/cairomm-1.14.2-8.fc34.x86_64.rpm and tried to replicate the problem, opening nearly 40 video tabs. However, even with about twice as many tabs as triggered the problem before (under cairomm-1.14.2-14), everything worked fine - no freezes and no error messages in the journal. Note that cinnamon runs only under X - it can't run under Wayland.
Since the rebuilt 1.14.2-8 doesn’t seem to trigger the problem for you, I’ll focus on potential differences between it and 1.14.2-14. As I mentioned before, the diff between the spec files for those two package versions doesn’t include any changes to the source tarball or compiler flags. Only the documentation (in the -doc subpackage) is supposed to be different. So I’ll be grasping at straws, and looking for anything that might indicate what’s going on. I compared the libraries (/usr/lib64/libcairomm-1.0.so.1.4.0) between the *rebuilt* 1.14.2-8 and the 1.14.2-14 build using “objdump -x” to compare the ELF headers. The results were identical. I compared again with diffoscope: > $ diffoscope 8new/usr/lib64/libcairomm-1.0.so.1.4.0 14/usr/lib64/libcairomm-1.0.so.1.4.0 > |# | N/A% None ETA: --:--:-- --- 8new/usr/lib64/libcairomm-1.0.so.1.4.0 > +++ 14/usr/lib64/libcairomm-1.0.so.1.4.0 > ├── readelf --wide --notes {} > │ @@ -1,15 +1,15 @@ > │ > │ Displaying notes found in: .note.gnu.property > │ Owner Data size Description > │ GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0 Properties: x86 feature: IBT, SHSTK > │ > │ Displaying notes found in: .note.gnu.build-id > │ Owner Data size Description > │ - GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) Build ID: 11e990e4531eeb7f3d951f638afaccbd2747a405 > │ + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) Build ID: 7e92e1994c7912a7fe848c4672a30e6192fb250e > │ > │ Displaying notes found in: .gnu.build.attributes > │ Owner Data size Description > │ GA$<version>3a1 0x00000010 OPEN Applies to region from 0x14000 to 0x21369 > │ GA$<version>3g979 0x00000010 OPEN Applies to region from 0x17630 to 0x2135a > │ GA*<stack prot>strong 0x00000000 OPEN Applies to region from 0x17630 to 0x2135a > │ GA$<tool>annobin gcc 11.2.1 20210728 0x00000000 OPEN Applies to region from 0x17630 to 0x2135a > ├── strings --all --bytes=8 {} > │ @@ -1019,16 +1019,17 @@ > │ GA!stack_realign > │ GA*FORTIFY > │ GA+GLIBCXX_ASSERTIONS > │ GA*cf_protection > │ GA+omit_frame_pointer > │ GA+stack_clash > │ GA!stack_realign > │ -libcairomm-1.0.so.1.4.0-1.14.2-8.fc34.x86_64.debug > │ -T28.LMGF > │ +libcairomm-1.0.so.1.4.0-1.14.2-14.fc34.x86_64.debug > │ +utvRj<(W > │ +E!_m 69e > │ .shstrtab > │ .note.gnu.property > │ .note.gnu.build-id > │ .gnu.hash > │ .gnu.version > │ .gnu.version_r > │ .rela.dyn > ├── readelf --wide --decompress --hex-dump=.gnu_debuglink {} > │ @@ -1,7 +1,7 @@ > │ > │ Hex dump of section '.gnu_debuglink': > │ 0x00000000 6c696263 6169726f 6d6d2d31 2e302e73 libcairomm-1.0.s > │ - 0x00000010 6f2e312e 342e302d 312e3134 2e322d38 o.1.4.0-1.14.2-8 > │ - 0x00000020 2e666333 342e7838 365f3634 2e646562 .fc34.x86_64.deb > │ - 0x00000030 75670000 9b9283fd ug...... > │ + 0x00000010 6f2e312e 342e302d 312e3134 2e322d31 o.1.4.0-1.14.2-1 > │ + 0x00000020 342e6663 33342e78 38365f36 342e6465 4.fc34.x86_64.de > │ + 0x00000030 62756700 88b3c04c bug....L > ├── readelf --wide --decompress --hex-dump=.gnu_debugdata {} > │ @@ -1,176 +1,176 @@ > │ > │ Hex dump of section '.gnu_debugdata': > │ 0x00000000 fd377a58 5a000004 e6d6b446 02002101 .7zXZ......F..!. > │ - 0x00000010 16000000 742fe5a3 e0333f0a 8e5d003f ....t/...3?..].? > │ + 0x00000010 16000000 742fe5a3 e0333f0a 8d5d003f ....t/...3?..].? > │ 0x00000020 91458468 3d89a6da 8ae18332 4ed90496 .E.h=......2N... > │ 0x00000030 036ff3d8 3bfb1f3a 44cfb028 4eaeb4c4 .o..;..:D..(N... > │ 0x00000040 6295b14e 9f51cbf1 aa78cfc5 eaa5eae4 b..N.Q...x...... > │ 0x00000050 8525040e 292bd642 d4ac4716 c233b610 .%..)+.B..G..3.. > │ 0x00000060 b67c3fc9 05d79247 c5823a87 41d49164 .|?....G..:.A..d > │ 0x00000070 f7ce78da f3a6a328 02cbb04d 6b54f87b ..x....(...MkT.{ > │ 0x00000080 7d1ab393 e22f7b65 1f398c77 8fa99ceb }..../{e.9.w.... > │ 0x00000090 b50fd53e 465b41a9 3327edf9 78b3c8cd ...>F[A.3'..x... > │ 0x000000a0 4c435c71 8dc0593d 173161e3 61a98d17 LC\q..Y=.1a.a... > │ 0x000000b0 01184847 b3ea93e4 51fef7fd 54ff9293 ..HG....Q...T... > │ - 0x000000c0 3dfdc4f9 f399c50a 5a6e59bf 3ceaf708 =.......ZnY.<... > │ - 0x000000d0 c9fcec61 8874cedb 9acef910 6a497fd0 ...a.t......jI.. > [ … snip … ] > │ - 0x00000ab0 1a8adbae 882646d2 0001aa15 c0660000 .....&F......f.. > │ - 0x00000ac0 33a7c5e9 b1c467fb 02000000 0004595a 3.....g.......YZ > │ + 0x000000c0 3dfdc4f9 f399c50a 5a6e59bf 3cebf51e =.......ZnY.<... > │ + 0x000000d0 ea35ee3d 58691325 b9203990 afef4ea9 .5.=Xi.%. 9...N. > [ … snip … ] > │ + 0x00000ab0 69e3c094 4d20fc01 0001a915 c0660000 i...M .......f.. > │ + 0x00000ac0 9dd5516f b1c467fb 02000000 0004595a ..Qo..g.......YZ I’m just not seeing anything that could account for a difference. Only the build ID and debug data are different. ----- Since a fresh build of 1.14.2-8 worked for you, and I can’t find any differences, what happens if you try a fresh build of 1.14.2-14? I’ve bumped the release number and changed nothing else to make a 1.14.2-15 (https://koji.fedoraproject.org/koji/taskinfo?taskID=77572484). If that works for you, I’m willing to issue it as an update for F34, although the lack of a plausible explanation for what you’re seeing does bother me.
Hi Ben Thanks for the new build. Note that I've deliberately made no other updates (such as kernel, systemd, etc) in order to ensure that the cairomm builds can be compared properly. This is definitely *much* better. I had about 25 tabs open and installed your build - no problem this time, whereas 1.14.2-14 would have complained immediately at this point. I continued to open more tabs to see if there would be a problem. All went well, and I was about to reply here that the problem was solved ... ... when I got the error again. :-( However, at that stage, I had well over 40 tabs open and Firefox was struggling anyway ... I wouldn't normally have so many tabs open; this was purely to test the new build, which wouldn't error until I got to this stage. In other words, I can't swear that previous versions might not have errored with so many tabs open. Not sure whether it's relevant, but the first error this time was for mod.adapter. (The previous time - with cairomm-1.14.2-14, the stream and mempool errors came first.) Oct 20 22:53:15 ++++ pipewire-pulse[1604]: mod.adapter: can't load spa node: Too many open files Oct 20 22:53:15 ++++ pipewire-pulse[1604]: mod.adapter: usage: node.name=<string> Oct 20 22:53:15 ++++ pipewire-pulse[1604]: default: stream 0x55a613ea50d0: can't make node: Invalid argument Oct 20 22:53:45 ++++ pipewire-pulse[1604]: default: mempool 0x55a60b0a9b00: Failed to create memfd: Too many open files Oct 20 22:53:45 ++++ pipewire-pulse[1604]: default: stream 0x55a63237f0f0: can't make node: Too many open files Oct 20 22:53:46 ++++ pipewire-pulse[1604]: default: mempool 0x55a60b0a9b00: Failed to create memfd: Too many open files Oct 20 22:53:46 ++++ pipewire-pulse[1604]: default: stream 0x55a620a24140: can't make node: Too many open files Oct 20 22:54:18 ++++ pipewire-pulse[1604]: default: mempool 0x55a60b0a9b00: Failed to create memfd: Too many open files Oct 20 22:54:18 ++++ pipewire-pulse[1604]: default: stream 0x55a620a4bc90: can't make node: Too many open files Oct 20 22:54:21 ++++ pipewire-pulse[1604]: default: mempool 0x55a60b0a9b00: Failed to create memfd: Too many open files Oct 20 22:54:21 ++++ pipewire-pulse[1604]: default: stream 0x55a61e50a2e0: can't make node: Too many open files Oct 20 22:54:51 ++++ pipewire-pulse[1604]: mod.adapter: can't load spa node: Too many open files Oct 20 22:54:51 ++++ pipewire-pulse[1604]: mod.adapter: usage: node.name=<string> Oct 20 22:54:51 ++++ pipewire-pulse[1604]: default: stream 0x55a60c551880: can't make node: Invalid argument Oct 20 22:55:20 ++++ pipewire-pulse[1604]: default: stream 0x55a62c164f80: can't make node: Too many open files Oct 20 22:55:52 ++++ pipewire-pulse[1604]: default: stream 0x55a60dc4ca20: can't make node: Too many open files Oct 20 22:55:55 ++++ pipewire-pulse[1604]: default: stream 0x55a60e97a7d0: can't make node: Too many open files Oct 20 22:56:31 ++++ pipewire-pulse[1604]: default: stream 0x55a62146bf90: can't make node: Too many open files Oct 20 22:57:06 ++++ pipewire-pulse[1604]: default: stream 0x55a610de5dd0: can't make node: Too many open files Oct 20 22:57:27 ++++ pipewire-pulse[1604]: default: stream 0x55a63600ff00: can't make node: Too many open files Given that I had to open an unrealistic number of video tabs to trigger an error on this new build, I think we can consider the problem solved. Thanks for looking into this.
My instinct tells me that this all feels like an exceptionally-elaborate coincidence—despite your careful and methodical testing—but I am happy that your results are better with the rebuild, whatever the means. I’ve rebuilt again, this time as an official build for F34, and you should see an update notification in this issue shortly. Please do try that update out if you have a chance. I’ll also rebuild the F35 package as a precaution.
FEDORA-2021-c08f780711 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-c08f780711
FEDORA-2021-7bfca4f076 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-7bfca4f076
FEDORA-2021-7bfca4f076 has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-7bfca4f076` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-7bfca4f076 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-c08f780711 has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-c08f780711` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-c08f780711 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Ben I've tried the new build and it seems perfect. I've stressed it as much as, probably more than, the scratch build from comment 7. I can't get it to throw any errors. I've given karma on bodhi. Thanks for your efforts.
Thanks for responding with details and trying out various builds. I wish I knew what had changed, but I’m really glad that the result is working for you.
FEDORA-2021-c08f780711 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2021-7bfca4f076 has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.