I ran into weird issue per description, which suggests some kind of race condition might be involved. Situation is like this: # pstree -ap -- $(pgrep -f dnf) > dnf,17499 -OO /usr/bin/dnf update -x synergy > └─sh,32627 /var/tmp/rpm-tmp.1UWM2w 1 > └─systemctl,32644 try-restart snmpd.service snmptrapd.service Output of dnf stopped progressing in this situation: > Cleanup : rpm-devel-4.12.0.1-11.fc22.x86_64 259/341 > Cleanup : libreoffice-impress-1:4.4.4.2-3.fc22.x86_64 260/341 > Cleanup : libreoffice-math-1:4.4.4.2-3.fc22.x86_64 261/341 > Cleanup : rpm-build-4.12.0.1-11.fc22.x86_64 262/341 > Cleanup : net-snmp-1:5.7.2-24.fc22.x86_64 263/341 For posterity, net-snmp is being updated from - to: - net-snmp-1:5.7.2-24.fc22.x86_64 + net-snmp-1:5.7.3-5.fc22.x86_64 Now the more interesting part: # rpm -q systemd > systemd-219-18.fc22.x86_64 (gdb bt output slightly reformatted) # gdb systemctl 32644 > [...] (gdb) t a a bt f > > Thread 1 (Thread 0x7f8dcbae9880 (LWP 32644)): > #0 __GI_ppoll > (fds=fds@entry=0x7fffdaf15080, nfds=1, timeout=<optimized out>, > sigmask=sigmask@entry=0x0) > at ../sysdeps/unix/sysv/linux/ppoll.c:51 > resultvar = 18446744073709551102 > tval = {tv_sec = 140736866636208, tv_nsec = 140246984712280} > result = <optimized out> > > #1 __ppoll_chk > at ppoll_chk.c:28 > No locals. > > #2 __ppoll_alias > at /usr/include/bits/poll2.h:71 > No locals. > > #3 bus_poll > at ../src/libsystemd/sd-bus/sd-bus.c:2791 > p = {{fd = 3, events = 1, revents = 0}, {fd = 0, events = 0, > revents = 0}} > r = <optimized out> > e = <optimized out> > n = <optimized out> > ts = {tv_sec = -1, tv_nsec = 140247012215312} > m = <optimized out> > __PRETTY_FUNCTION__ = "bus_poll" > > #4 sd_bus_wait > No locals. > > #5 bus_process_wait > (bus=0x7f8dcd580030) > at ../src/libsystemd/sd-bus/bus-util.c:1687 > r = <optimized out> > > #6 bus_wait_for_jobs > (quiet=<optimized out>, d=0x7f8dcd580a10) > at ../src/libsystemd/sd-bus/bus-util.c:1751 > r = 0 > > #7 start_unit > w = <optimized out> > method = <optimized out> > mode = <optimized out> > one_name = <optimized out> > suffix = <optimized out> > names = 0x7f8dcd580990 > name = <optimized out> > r = <optimized out> > __PRETTY_FUNCTION__ = "start_unit" > __func__ = "start_unit" > > #8 systemctl_main > (bus_error=0, argv=0x7fffdaf15458, argc=<optimized out>, > bus=0x7f8dcd580030) > at ../src/systemctl/systemctl.c:7216 > left = 3 > verbs = {{verb = 0x7f8dcbb92b1d "list-units", argc_cmp = MORE, > argc = 0, dispatch = 0x7f8dcbb555e0 > <list_units.lto_priv.347>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b28 "list-unit-files", argc_cmp = MORE, argc > = 1, dispatch = 0x7f8dcbb561f0 > <list_unit_files.lto_priv.348>, bus = NOBUS}, { > verb = 0x7f8dcbb92b38 "list-sockets", argc_cmp = MORE, argc = > 1, dispatch = 0x7f8dcbb54ac0 <list_sockets.lto_priv.349>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b45 "list-timers", argc_cmp = MORE, argc = > 1, dispatch = 0x7f8dcbb54390 <list_timers.lto_priv.350>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b51 "list-jobs", argc_cmp = MORE, argc = 1, > dispatch = 0x7f8dcbb2dcb0 <list_jobs.lto_priv.351>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb92b5b "list-machines", argc_cmp = MORE, argc = > 1, dispatch = 0x7f8dcbb2ecf0 <list_machines.lto_priv.352>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb91193 "clear-jobs", argc_cmp = EQUAL, argc = > 1, dispatch = 0x7f8dcbb39ae0 <daemon_reload.lto_priv.353>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb9119e "cancel", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb2d9b0 <cancel_job.lto_priv.354>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb90531 "start", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb90718 "stop", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb9071d "condstop", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb90738 "restart", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb9074c "try-restart", argc_cmp = MORE, argc = > 2, dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb90773 "reload-or-restart", argc_cmp = MORE, > argc = 2, dispatch = 0x7f8dcbb306c0 > <start_unit.lto_priv.355>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb90799 "reload-or-try-restart", argc_cmp = > MORE, argc = 2, dispatch = 0x7f8dcbb306c0 > <start_unit.lto_priv.355>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb907d1 "force-reload", argc_cmp = MORE, argc = > 2, dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb907c6 "condreload", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb90767 "condrestart", argc_cmp = MORE, argc = > 2, dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, > bus = (unknown: 0)}, { > verb = 0x7f8dcbb90549 "isolate", argc_cmp = EQUAL, argc = 2, > dispatch = 0x7f8dcbb306c0 <start_unit.lto_priv.355>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb92b69 "kill", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb32220 <kill_unit.lto_priv.356>, bus = > (unknown: 0)}, { > verb = 0x7f8dcbb92b6e "is-active", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb32750 > <check_unit_active.lto_priv.357>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b78 "check", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb32750 > <check_unit_active.lto_priv.357>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b7e "is-failed", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb32730 > <check_unit_failed.lto_priv.358>, bus = (unknown: 0)}, { > verb = 0x7f8dcbb92b88 "show", argc_cmp = MORE, argc = 1, > dispatch = 0x7f8dcbb68a80 <show.lto_priv.359>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb92b8d "cat", > argc_cmp = MORE, argc = 2, dispatch = 0x7f8dcbb3d630 > <cat.lto_priv.360>, bus = NOBUS}, {verb = 0x7f8dcbb92b91 > "status", argc_cmp = MORE, argc = 1, > dispatch = 0x7f8dcbb68a80 <show.lto_priv.359>, bus = (unknown: > 0)}, {verb = 0x7f8dcbb929d6 "help", argc_cmp = MORE, argc > = 2, > dispatch = 0x7f8dcbb68a80 <show.lto_priv.359>, bus = (unknown: > 0)}, {verb = 0x7f8dcbb92b98 "snapshot", argc_cmp = LESS, > argc = 2, > dispatch = 0x7f8dcbb3aea0 <snapshot.lto_priv.361>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb92ba1 "delete", argc_cmp = > MORE, argc = 2, > dispatch = 0x7f8dcbb3abc0 <delete_snapshot.lto_priv.362>, bus > = (unknown: 0)}, {verb = 0x7f8dcbb92ba8 "daemon-reload", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb39ae0 <daemon_reload.lto_priv.353>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb911a5 "daemon-reexec", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb39ae0 <daemon_reload.lto_priv.353>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb92bb6 "show-environment", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb3a710 <show_environment.lto_priv.363>, bus > = (unknown: 0)}, {verb = 0x7f8dcbb9112e "set-environment", > argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb398c0 <set_environment.lto_priv.364>, bus > = (unknown: 0)}, {verb = 0x7f8dcbb92bc7 "unset- > environment", argc_cmp = MORE, argc = 2, > dispatch = 0x7f8dcbb398c0 <set_environment.lto_priv.364>, bus > = (unknown: 0)}, {verb = 0x7f8dcbb92bd9 "import- > environment", argc_cmp = MORE, argc = 1, > dispatch = 0x7f8dcbb39f00 <import_environment.lto_priv.365>, > bus = (unknown: 0)}, {verb = 0x7f8dcbb90cb2 "halt", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > FORCE}, {verb = 0x7f8dcbb90cdc "poweroff", argc_cmp = > EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > FORCE}, {verb = 0x7f8dcbb90cf3 "reboot", argc_cmp = MORE, > argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > FORCE}, {verb = 0x7f8dcbb90d07 "kexec", argc_cmp = EQUAL, > argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90d2d "suspend", argc_cmp > = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90d46 "hibernate", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90d64 "hybrid-sleep", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90de5 "default", argc_cmp > = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90dc3 "rescue", argc_cmp = > EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90ddb "emergency", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb90d19 "exit", argc_cmp = > EQUAL, argc = 1, > dispatch = 0x7f8dcbb35600 <start_special.lto_priv.366>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb911b3 "reset-failed", > argc_cmp = MORE, argc = 1, > dispatch = 0x7f8dcbb3a8c0 <reset_failed.lto_priv.367>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb91062 "enable", argc_cmp = > MORE, argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb91085 "disable", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb912d2 "is-enabled", argc_cmp = > MORE, argc = 2, > dispatch = 0x7f8dcbb72160 <unit_is_enabled.lto_priv.369>, bus > = NOBUS}, {verb = 0x7f8dcbb913f1 "reenable", argc_cmp = > MORE, argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb913ff "preset", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb92bec "preset-all", argc_cmp = > EQUAL, argc = 1, > dispatch = 0x7f8dcbb725e0 <preset_all.lto_priv.370>, bus = > NOBUS}, {verb = 0x7f8dcbb91423 "mask", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb91432 "unmask", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb913fa "link", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb3be70 <enable_unit.lto_priv.368>, bus = > NOBUS}, {verb = 0x7f8dcbb92bf7 "switch-root", argc_cmp = > MORE, argc = 2, > dispatch = 0x7f8dcbb3a2e0 <switch_root.lto_priv.371>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb92c03 "list-dependencies", > argc_cmp = LESS, argc = 2, > dispatch = 0x7f8dcbb30310 <list_dependencies.lto_priv.372>, > bus = (unknown: 0)}, {verb = 0x7f8dcbb92c15 "set-default", > argc_cmp = EQUAL, argc = 2, > dispatch = 0x7f8dcbb31eb0 <set_default.lto_priv.373>, bus = > NOBUS}, {verb = 0x7f8dcbb92c21 "get-default", argc_cmp = > EQUAL, argc = 1, > dispatch = 0x7f8dcbb2e580 <get_default.lto_priv.374>, bus = > NOBUS}, {verb = 0x7f8dcbb92c2d "set-property", argc_cmp = > MORE, argc = 3, > dispatch = 0x7f8dcbb3b1f0 <set_property.lto_priv.375>, bus = > (unknown: 0)}, {verb = 0x7f8dcbb92c3a "is-system-running", > argc_cmp = EQUAL, argc = 1, > dispatch = 0x7f8dcbb72070 <is_system_running.lto_priv.376>, > bus = (unknown: 0)}, {verb = 0x7f8dcbb92c4c "add-wants", > argc_cmp = MORE, argc = 3, > dispatch = 0x7f8dcbb72e70 <add_dependency.lto_priv.377>, bus = > NOBUS}, {verb = 0x7f8dcbb92c56 "add-requires", argc_cmp = > MORE, argc = 3, > dispatch = 0x7f8dcbb72e70 <add_dependency.lto_priv.377>, bus = > NOBUS}, {verb = 0x7f8dcbb92c63 "edit", argc_cmp = MORE, > argc = 2, > dispatch = 0x7f8dcbb71740 <edit.lto_priv.378>, bus = NOBUS}, > {verb = 0x0, argc_cmp = MORE, argc = 0, dispatch = 0x0, > bus = (unknown: 0)}} > verb = 0x7f8dcbbb4b60 <verbs+416> > > #9 main > (argc=<optimized out>, argv=0x7fffdaf15458) > at ../src/systemctl/systemctl.c:7474 > bus = 0x7f8dcd580030 > r = 0 > __func__ = "main" > __PRETTY_FUNCTION__ = "main" # strace -fp 32644 > Process 32644 attached > ppoll([{fd=3, events=POLLIN}], 1, NULL, NULL, 8^CProcess 32644 detached > <detached ...> back in debugger, at frame > #6 bus_wait_for_jobs (quiet=<optimized out>, d=0x7f8dcd580a10) > at ../src/libsystemd/sd-bus/bus-util.c:1751 (gdb) p *d->jobs > $1 = { > b = { > hash_ops = 0x7f8dcbbb5ee0 <string_hash_ops>, > { > indirect = { > storage = 0x7f8dcd582700 "/org/freedesktop/systemd1/job/248358", > hash_key = "p&X\177\000\000\000\000\000\000\000\000\000", > n_entries = 0, > n_buckets = 0, > idx_lowest_entry = 4294902016, > _pad = "\000\000" > }, > direct = { > storage = "\000'X\177\000\000p&X\177", '\000' <repeats 19 times>, "\001\377\377\000\000" > } > }, > type = HASHMAP_TYPE_SET, > has_indirect = false, > n_direct_entries = 2, > from_pool = true > } > } # busctl introspect org.freedesktop.systemd1 /org/freedesktop/systemd1/job/248358 > NAME TYPE SIGNATURE RESULT/VALUE FLAGS > org.freedesktop.DBus.Introspectable interface - - - > .Introspect method - s - > org.freedesktop.DBus.Peer interface - - - > .GetMachineId method - s - > .Ping method - - - > org.freedesktop.DBus.Properties interface - - - > .Get method ss v - > .GetAll method s a{sv} - > .Set method ssv - - > .PropertiesChanged signal sa{sv}as - - > org.freedesktop.systemd1.Job interface - - - > .Cancel method - - - > .Id property u 248358 const > .JobType property s "nop" const > .State property s "waiting" emits-change > .Unit property (so) "snmpd.service" "/org/freedesktop/sys... const So it seems that this very systemctl process is in the idle loop waiting for snmpd.service to finish (as it seems that it would have errorred early on if snmpd.service could not have been [re]started) but in fact no such process is running: # pgrep -f snmp > 32644 # ps aux | grep $(pgrep -f snmp) > root 32644 0.0 0.0 130536 244 pts/28 S+ Jul13 0:00 systemctl try-restart snmpd.service snmptrapd.service # systemctl status snmpd.service snmptrapd.service > ● snmpd.service - Simple Network Management Protocol (SNMP) Daemon. > Loaded: loaded (/usr/lib/systemd/system/snmpd.service; disabled; vendor preset: disabled) > Active: inactive (dead) > > ● snmptrapd.service - Simple Network Management Protocol (SNMP) Trap Daemon. > Loaded: loaded (/usr/lib/systemd/system/snmptrapd.service; disabled; vendor preset: disabled) > Active: inactive (dead) # journalctl --unit=snmpd.service --unit=snmptrapd.service > -- Logs begin at Wed 2014-10-22 15:45:17 CEST, end at Fri 2015-07-17 22:25:18 CEST. -- (nope, I haven't ever conciously run anything related to snmp) Please let me known how to provide better diagnostics.
Can you reproduce this on your machine? I can't on mine. I repeatedly tried to update/downgrade net-snmp and its dependencies using the same versions as in the report and there were no lockups.
Provided the expected character of the bug -- race condition -- I doubt the bug is reasonably reproducible. That's why I am asking for help in after-the-fact diagnostics. My sihcere hope is that systemd has its means for that, be it statically observable in various logs or dynamically including use of a debugger, and I am just not familiar enough. Regarding correlation on the timeline, the sequence of vaguely interesting point is as follows: for some reason, systemd suddenly started emitting following lines with a high cadence (since the very first occurrence): > Jul 13 00:00:02 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. > Jul 13 00:00:03 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. > Jul 13 00:00:04 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. > Jul 13 00:00:05 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. > Jul 13 00:00:06 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. - two notes: - these lines hasn't stopped till now as I haven't rebooted the machine yet to allow for diagnosing this very case - there is some likelihood that this was provoked either by low-memory circumstance or by execution on user's cronjob that happened just a second prior to this manual run of dnf: # ps --no-header -p "$(pgrep -f dnf)" -o lstart > Mon Jul 13 16:24:04 2015 which likely triggered systemd reload: > Jul 13 16:28:27 juicyfruit audit[1]: <audit-1107> pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc: received policyload notice (seqno=4) > exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?' > Jul 13 16:28:27 juicyfruit systemd[1]: Reloading. > Jul 13 16:28:27 juicyfruit systemd-sysv-generator[32606]: Overwriting existing symlink /run/systemd/generator.late/netconsole.service with real service > Jul 13 16:28:27 juicyfruit systemd-sysv-generator[32606]: Overwriting existing symlink /run/systemd/generator.late/network.service with real service > Jul 13 16:28:27 juicyfruit systemd[1]: Configuration file /usr/lib/systemd/system/auditd.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway. > Jul 13 16:28:27 juicyfruit systemd[1]: Looping too fast. Throttling execution a little. > Jul 13 16:28:28 juicyfruit systemd[1]: Reloading. > Jul 13 16:28:29 juicyfruit systemd-sysv-generator[32632]: Overwriting existing symlink /run/systemd/generator.late/network.service with real service > Jul 13 16:28:29 juicyfruit systemd-sysv-generator[32632]: Overwriting existing symlink /run/systemd/generator.late/netconsole.service with real service > Jul 13 16:28:29 juicyfruit systemd[1]: Configuration file /usr/lib/systemd/system/auditd.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway. Accidentally, at 16:28:28 also that systemctl command with PID 32644 was run: # ps --no-header -p 32644 -o lstart > Mon Jul 13 16:28:28 2015 Which might confirm a race-condition hypothesis, couldn't it?
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days