Bug 1090958

Description of problem:
This issue was discovered by automation which tests functions from CLI client sample file deploy-to-and-restart-JBAS.js.

Following part taken from deploy-to-and-restart-JBAS.js shows the problem:

var schedule = OperationManager.scheduleGroupOperation(group.id, resourceIds, false, operationName, conf, 0, 0, 0, 0, null);

    // now we need to wait for all the individual operations on resources to
    // complete
    var crit = new GroupOperationHistoryCriteria;
    crit.addFilterJobId(new JobId(schedule.jobName, schedule.jobGroup));

    while (true) {
        // XXX should this be configurable?
        // we're waiting for an AS to start/shutdown which takes time..
        // Let's check in 5s intervals.
        // We're waiting at the start of this loop to also ensure that the server
        // had time to issue the quartz job and persist the history entry.
        java.lang.Thread.currentThread().sleep(5000);

        var results = OperationManager.findGroupOperationHistoriesByCriteria(crit);

        if (results.empty) {
            throw "Could not find operation history for schedule " + schedule + ". This should not happen.";
        }

        var history = results.get(0);

        if (history.status != OperationRequestStatus.INPROGRESS) {
            return history;
        }
    } 


Operation was scheduled but sometimes an error "Could not find operation history for schedule ..... This should not happen." was thrown. This means that no GroupOperationHistories were returned even though the script waits 5 seconds after the operation was scheduled. I added another test which showed that it takes up to 24 seconds to get GroupOperationHistories after a operation was scheduled. 

Version-Release number of selected component (if applicable):
Version :	
4.11.0-SNAPSHOT
Build Number :	
251d60a

How reproducible:
3/4 by our automation which runs on HA environment. I'm not able to reproduce it on a simple single node environment.


Additional info:
One way of fixing this (just making the test pass) would be a fix in deploy-to-and-restart-JBAS.js sample script (e.g. add a waiting for GroupOperationHistories to be available with some timeout). But more important question is if it is expected that it can take so long.