Bug 2227536

Hi Frank,

Thanks for reporting the issue. However I think it is expected due to upstream patch: 9a1fd29a82c97 ("activate_mapping: report error reason") and eee7917ef52726 ("activate_mapping: only blacklist irq if error is considered permanent"). In short words, if a IRQ number failed to set affinity due to !(ENOSPC && EAGAIN && EBUSY && EINVAL && ERANGE && ENOMEM) error, it will be kicked out of schedule list in case of re-set affinity next time, and at the same time throw out a error message to report the failing details.

Previous version of irqbalance will just fail with no error report, so users don't know if a irq number is affinity set successfully or not, this bug is addressed in bz2184735, and the patches I mentioned previously is meant to fix this bz.

Thanks,
Tao Liu

(In reply to ltao from comment #1)
> Hi Frank,
> 
> Thanks for reporting the issue. However I think it is expected due to
> upstream patch: 9a1fd29a82c97 ("activate_mapping: report error reason") and
> eee7917ef52726 ("activate_mapping: only blacklist irq if error is considered
> permanent"). In short words, if a IRQ number failed to set affinity due to
> !(ENOSPC && EAGAIN && EBUSY && EINVAL && ERANGE && ENOMEM) error, it will be
> kicked out of schedule list in case of re-set affinity next time, and at the
> same time throw out a error message to report the failing details.
> 
> Previous version of irqbalance will just fail with no error report, so users
> don't know if a irq number is affinity set successfully or not, this bug is
> addressed in bz2184735, and the patches I mentioned previously is meant to
> fix this bz.
> 
> Thanks,
> Tao Liu

Thanks, ltao

I think it is good to throw out fail messages accordingly.
Now just feel that there are too many lines(128 similar lines in attachment) in journal logs. Is it possible to suppress them or save details in other place?

Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: Cannot change IRQ 99 affinity: Input/output error
Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: IRQ 99 affinity is now unmanaged

(In reply to Frank Liang from comment #2)
> (In reply to ltao from comment #1)
> > Hi Frank,
> > 
> > Thanks for reporting the issue. However I think it is expected due to
> > upstream patch: 9a1fd29a82c97 ("activate_mapping: report error reason") and
> > eee7917ef52726 ("activate_mapping: only blacklist irq if error is considered
> > permanent"). In short words, if a IRQ number failed to set affinity due to
> > !(ENOSPC && EAGAIN && EBUSY && EINVAL && ERANGE && ENOMEM) error, it will be
> > kicked out of schedule list in case of re-set affinity next time, and at the
> > same time throw out a error message to report the failing details.
> > 
> > Previous version of irqbalance will just fail with no error report, so users
> > don't know if a irq number is affinity set successfully or not, this bug is
> > addressed in bz2184735, and the patches I mentioned previously is meant to
> > fix this bz.
> > 
> > Thanks,
> > Tao Liu
> 
> Thanks, ltao
> 
> I think it is good to throw out fail messages accordingly.
> Now just feel that there are too many lines(128 similar lines in attachment)
> in journal logs. Is it possible to suppress them or save details in other
> place?
> 
> Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal
> /usr/sbin/irqbalance[1041]: Cannot change IRQ 99 affinity: Input/output error
> Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal
> /usr/sbin/irqbalance[1041]: IRQ 99 affinity is now unmanaged

Hi Frank,

I see from your log, it seems to me you are doing some testing. The test script recongnized the error log throwed out by irqbalance. And it contains 4 parts:

1)
Jul 30 09:27:04 ip-10-22-1-117.us-west-2.compute.internal kdumpctl[1446]: kdump: kexec: loaded kdump kernel
Jul 30 09:27:04 ip-10-22-1-117.us-west-2.compute.internal kdumpctl[1446]: kdump: Starting kdump: [OK]
Jul 30 09:27:04 ip-10-22-1-117.us-west-2.compute.internal systemd[1]: Finished Crash recovery kernel arming.
Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal systemd[1]: Startup finished in 1.409s (kernel) + 2.546s (initrd) + 12.764s (
userspace) = 16.720s.
Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: Cannot change IRQ 108 affinity: Input/output error
Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: IRQ 108 affinity is now unmanaged
...snip...

This part is the real log outputted by irqbalance, 64 items in total.

2)
2023-07-30 09:36:09,607:INFO:error found!
2023-07-30 09:36:09,608:INFO:Checking:Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: Cannot change IRQ 76 affinity: Input/output error
...snip...

This part is the test script finishes testing, and it detected the error log so it output all the error info it found.

3)
2023-07-30 09:36:15,239:INFO:Below items are unknow!
['Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: Cannot change IRQ 76 affinity: Input/output error', 'Jul 30 09:27:05 ip-10-22-1-117.us-west-2.compute.internal /usr/sbin/irqbalance[1041]: ...snip...

This part is the same error log as 2, but report in array format.

4)
AssertionError: New error in journalctl -b 0|grep -Ev "os_tests.tests.test_general_check.TestGeneralCheck.test_check_journalctl_error"|
grep -Ev test_check|grep -Ev "UpdateGSErrors" log

This part is also the same error log as 2.

I suppose the system you are testing is a large system which contains hundreds of IRQs, among which, 64 IRQs failed to set affinity, it is acceptable to me. And also irqbalance only reported them once, it is also expected. I think suppressing the error logs is better to be handled in the test script side, it doesn't need to repeat outputting the error logs 3 times.

Thanks,
Tao Liu

Ok, pls feel free to close it as not a bug. Thanks

OK, I will close the bz as not a bug.