The Linux kernel does not correctly mitigate SMT attacks, as discovered
through a strange pattern in the kernel API using STIBP as a mitigation[1
<https://docs.kernel.org/userspace-api/spec_ctrl.html>], leaving the
process exposed for a short period of time after a syscall. The kernel also
does not issue an IBPB immediately during the syscall.
The ib_prctl_set [2
updates the Thread Information Flags (TIFs) for the task and updates the
SPEC_CTRL MSR on the function __speculation_ctrl_update [3
but the IBPB is only issued on the next schedule, when the TIF bits are
checked. This leaves the victim vulnerable to values already injected on
the BTB, prior to the prctl syscall.
The behavior is only corrected after a reschedule of the task happens.
Furthermore, the kernel entrance (due to the syscall itself), does not
issue an IBPB in the default scenarios (i.e., when the kernel protects
itself via retpoline or eIBRS).