I am currently having to firewall any robot that gets into /webx as
they all seem to be getting stuck and then the app servers go into
load averages of 15-20. I need some help or suggestions here
updated robots.txt to keep the robots going into infinite loops.