From vendor-sec:
Christian Borntraeger found a security relevant problem with the Linux system calls argument passing for at least s390, powerpc, sparc64 and mips:

The ABI for some architectures defines that the caller of a function has to sign extend each parameter to full register width. This is a problem in Linux system call handling.

For example with this system call on 64 bit:

asmlinkage long sys_example(unsigned int index)
{
        if (index > 5)
                return -EINVAL;
        return example_array[index];
}

It would mean that the caller has to sign extend index to 64 bit. In this case the caller is userspace. So we cannot rely on a correct sign extension.

But actually we just pass userspace delivered parameters unmodified to the system call function.

The above example could break like this:

The compiler can create code that will only test the lower 32 bits of the register that contain index to make sure its value is <= 5. However to access the memory location to get the value out of the array it can use the same unmodified (64 bit) register as index register, since the caller had to make sure that the upper 32 bits of the register contain zeroes. But since the value came from user space this isn't guaranteed and can lead to an addressing exception. Or userspace reads or even writes from/to memory locations it is not allowed to have access to.

Please note that this has nothing to do with compat system calls. This is an issue with 64 bit kernel and 64 bit userspace but 32 bit arguments.

Unfortunately the issue must be considered to be already public:

http://marc.info/?l=linux-kernel&m=123155111608910&w=2