Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1666680

Summary: tuned fails due to "RuntimeError: Set changed size during iteration"
Product: Red Hat Enterprise Linux 8 Reporter: Ondřej Lysoněk <olysonek>
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED DUPLICATE QA Contact: qe-baseos-daemons
Severity: medium Docs Contact:
Priority: medium    
Version: 8.0CC: jeder, jskarvad, lcapitulino, olysonek
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1592743 Environment:
Last Closed: 2019-01-24 21:12:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1592743    
Bug Blocks: 1640832, 1665995    

Comment 1 Luiz Capitulino 2019-01-24 18:07:21 UTC
We seem to be triggering this issue with tuned-2.10.0-14.el8
when applying the realtime-virtual-host profile:

2019-01-14 09:01:12,156 ERROR    tuned.units.manager: BUG: Unhandled exception in start_tuning: dictionary changed size during iteration

It seems that the realtime-virtual-host profile is not fully
applied because of this bug.

Do you think it's the same issue? If yes, do we have a plan to
get it fixed?

Comment 2 Ondřej Lysoněk 2019-01-24 19:17:35 UTC
Oh wow, I misread the error message. I thought it's the same as in this bug, sorry! Could you try to reproduce the issue with the following patch applied to /usr/lib/python3.6/site-packages/tuned/units/manager.py? Alternatively, you can pass the '-D' option to tuned in /usr/lib/systemd/system/tuned.service. Please provide a new log after doing that. I'll provide a test build, if this is any trouble.

diff --git a/tuned/units/manager.py b/tuned/units/manager.py
index 62ed0a8..2e9928e 100644
--- a/tuned/units/manager.py
+++ b/tuned/units/manager.py
@@ -77,7 +77,7 @@ class Manager(object):
 			trace = traceback.format_exc()
 			log.error("BUG: Unhandled exception in %s: %s"
 					% (caller, str(e)))
-			log.debug(trace)
+			log.error(trace)
 			return exc_ret
 
 	def destroy_all(self):
--

Anyway, I've been working on the 'Set changed size' problem. Here's a WIP branch. I haven't tested the patch yet, so I'm sorry if it's completely broken. I'm really afraid it won't fix the 'dictionary changed size' problem though..
https://github.com/olysonek/tuned/tree/hotplug-races

Comment 3 Luiz Capitulino 2019-01-24 20:04:04 UTC
Here you go. It took a bit to reproduce, and for some reason I'm now
only reproducing it when stopping the profile (although I did get it
when starting the profile in comment 1).

2019-01-24 14:58:26,409 ERROR    tuned.units.manager: BUG: Unhandled exception in stop_tuning: dictionary changed size during iteration
2019-01-24 14:58:26,409 ERROR    tuned.units.manager: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tuned/units/manager.py", line 75, in _try_call
    return f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tuned/plugins/instance/instance.py", line 82, in unapply_tuning
    self._plugin.instance_unapply_tuning(self, full_rollback)
  File "/usr/lib/python3.6/site-packages/tuned/plugins/base.py", line 291, in instance_unapply_tuning
    self._instance_unapply_static(instance, full_rollback)
  File "/usr/lib/python3.6/site-packages/tuned/plugins/plugin_scheduler.py", line 418, in _instance_unapply_static
    self._restore_ps_affinity()
  File "/usr/lib/python3.6/site-packages/tuned/plugins/plugin_scheduler.py", line 395, in _restore_ps_affinity
    ps = self.get_processes()
  File "/usr/lib/python3.6/site-packages/tuned/plugins/plugin_scheduler.py", line 145, in get_processes
    ps.reload_threads()
  File "/usr/lib/python3.6/site-packages/procfs/procfs.py", line 462, in reload_threads
    for pid in self.processes.keys():
RuntimeError: dictionary changed size during iteration

Comment 4 Ondřej Lysoněk 2019-01-24 21:12:12 UTC
Thanks, Luiz. It's indeed a different problem. It turned out to be a bug in python-linux-procfs, reported as bug#1669294. There's a patch, which you may want to try out.

*** This bug has been marked as a duplicate of bug 1669294 ***