Bug 748645

-- Problem Description --
After the install of MRG 2.1 Preview on an HS21xm, we see 
the following messages in /var/log/boot.log:

ROOT:elm9m94:log]> grep chrt boot.log-20111023 
Setting kernel thread priorities: chrt: failed to set pid 7's policy: Invalid argument
chrt: failed to set pid 9's policy: Invalid argument
chrt: failed to set pid 14's policy: Invalid argument
chrt: failed to set pid 18's policy: Invalid argument
chrt: failed to set pid 22's policy: Invalid argument
chrt: failed to set pid 26's policy: Invalid argument
chrt: failed to set pid 30's policy: Invalid argument
chrt: failed to set pid 34's policy: Invalid argument

ROOT:elm9m94:~]> uname -a
Linux elm9m94 3.0.6-rt18.35.el6rt.x86_64 #1 SMP PREEMPT RT Thu Oct 13 12:55:23 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

ROOT:elm9m94:~]> cat /etc/*release
Red Hat Enterprise Linux Server release 6.1 (Santiago)
Red Hat Enterprise Linux Server release 6.1 (Santiago)

No changes had been made to /etc/init.d/rtctl or /etc/sysconfig/rtctl  or any other file (pristine install). 

Note that pid 7, 9, 14, ... are all the various migration threads 
(per-cpu) and are running at SCHED_FIFO, prio 99  from ps
output:

[migration/0]                   7     99 FF
[migration/1]                   9     99 FF
[migration/2]                  14     99 FF
[migration/3]                  18     99 FF
[migration/4]                  22     99 FF
[migration/5]                  26     99 FF
[migration/6]                  30     99 FF
[migration/7]                  34     99 FF


So it might just be that the error msg is due to the fact that
the migration threads are already at prio 99 and SCHED_FIFO
to start with. 

Setting the RT scheduler priority or policy to be the same as the 
current might have the effect of taking the running task off the 
cpu and putting it to the back of the runqueue for that priority (IIRC??).
It's possible chrt just returns an error in that case? Not checked. 

This might just be chrt handling of that situation, but worth looking
into. 

This is new in MRG 2.1.  Was not seen in MRG 2.0 kernels or MRG
1.3 and earlier. 

Nothing relevant in other logs.

Might be confusing for customers. This was discussed with Clark
Williams (Red Hat) in today's IBM/RH  RT Interlock, and Clark
has also reproduced. We're filing bug for tracking purposes.