Bug 696080

A couple of comments on this issue:

1) QMF does not have a "submit" feature.  If QMF method-call throughput is in question, then it should be benchmarked using asynchronous method calls.

2) The measurements made here are based on synchronous method calls (caller is blocked until the method completes and returns) where the remote processing takes some time.  It is unlikely that you will see very high submit throughput unless you use asynchronous method calls or use many separate submitter processes.

1) This bug was submitted for Grid QMF object com.redhat.grid.scheduler which has method SubmitJob.

2) I would call this method synchronously as I do synchronously submitting via command line.

Sorry for confusing, but my question was if that difference between command line submitting and QMF submitting is OK or not?
Do I anything wrong? Is those condor settings ok for best Condor<->QMF performance?

Summary - take with a grain of salt -

CLI @ 21/s
QMF @ 8/s
Aviary @ 43/s

--

CLI -

T510, 12 runs - avg 4.73s ~= 21/s (single, time ./sub.sh 100)

$ cat sub.sh 
#!/bin/sh

TMP=$(mktemp XXXXXX.sub)
cat > $TMP <<EOF
executable=/bin/sleep
iwd=/tmp
universe=vanilla
arguments=10000
queue
EOF

TIMES=$1
for i in $(seq $TIMES); do
   condor_submit $TMP
done

rm $TMP

--

QMF -

T510, 12 runs - avg 11.44s ~= 8/s (single, ./sub.py)

NOTE: There is a 3-4 second startup delay for each run, thus the script calculates its own runtime.

$ cat sub.py 
#!/usr/bin/python

from qmf.console import Session
import time
ad = {"cmd":"/bin/sleep",
      "args":"10000",
      "requirements":"TRUE",
      "iwd":"/tmp",
      "owner":"matt",
      "universe":"vanilla",
      "!!descriptors":  {"requirements":"com.redhat.grid.Expression"}
}

session = Session();
broker = session.addBroker("cumin/cumin@localhost:5672", 60, 'PLAIN')
scheduler = session.getObjects(_class="scheduler",
_package="com.redhat.grid")[0]

start = time.time()
print "START", start
for i in range(100):
  time1 = time.time()
  result = scheduler.SubmitJob(ad)
  time2 = time.time()
  print str(time2-time1)
stop = time.time()
print "STOP", stop
print "RUN", stop-start

session.delBroker(broker)
session.close()

--

Aviary -

T510, 12 runs - avg 2.29s ~= 43/s (single, ./submit.py)

NOTE: The overhead (diff w/ time) is ~0.1s, but the script still calculates its own runtime to correspond to QMF submission.

$ cat submit.py 
#!/usr/bin/python

from suds import *
from suds.client import Client
from sys import exit, argv
import time, pwd, os
import logging
import argparse

# change these for other default locations and ports
wsdl = 'file:/var/lib/condor/aviary/services/job/aviary-job.wsdl'
url = 'http://localhost:9090/services/job/submitJob'
client = Client(wsdl)
client.set_options(location=url)
start = time.time()
for i in range(100):
   time1 = time.time()
   result = client.service.submitJob(
               '/bin/sleep',
	       '120',
	       "matt",
	       '/tmp',
	       'perf sub')
   if result.status.code != "OK":
      print result.status.code,"; ", result.status.text
      exit(1)
   time2 = time.time()
   print time2-time1
stop = time.time()
print "RUN", stop-start

Martin, thank you for bringing attention to this. If you wish to produce performance regression test for submission methods please focus on the CLI and Aviary methods. 40Hz CLI and 80Hz Aviary make a reasonable baseline -- from grid0.

The QMF submission performance is disappointing, but does not warrant attention at this time. It will be replaced with Aviary submission, which performs better than CLI and QMF, in the simple tests found in comment 3.