Description of problem:
when file is being written, especially appended to, then file ping-pongs between cold and hot tier

Version-Release number of selected component (if applicable):


How reproducible:
always


Steps to Reproduce:
1. create file BIGDEMO-1 on cold tier at mount point: /mnt/twoXtwo
2. attach hot tier bricks
3. start appending to file using the following shell command:

while true; do
    dd if=/dev/urandom of=/mnt/twoXtwo/BIGDEMO-1 ibs=1024 obs=2014 \
       conv=notrunc oflag=append count=100;
    ls -l /mnt/twoXtwo;
    sleep 1;
done