Bug 705365 - [RFE] Update aviary packaging to include examples.
Summary: [RFE] Update aviary packaging to include examples.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor-aviary
Version: Development
Hardware: All
OS: Linux
low
low
Target Milestone: 2.1
: ---
Assignee: Pete MacKinnon
QA Contact: Martin Kudlej
URL:
Whiteboard:
Depends On:
Blocks: 743350
TreeView+ depends on / blocked
 
Reported: 2011-05-17 13:48 UTC by Timothy St. Clair
Modified: 2012-03-20 09:29 UTC (History)
5 users (show)

Fixed In Version: condor-7.6.5-0.4
Doc Type: Bug Fix
Doc Text:
This update adds several example scripts to the condor-aviary package for job submission, control, attribute set, queries for status, summary, and details. A directed acyclic graph (DAG) submission example has also been added. Users are now provided with a practical set of client examples for the Aviary web service interface written in Python and using the Suds client.
Clone Of:
Environment:
Last Closed: 2012-01-23 17:26:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2012:0045 0 normal SHIPPED_LIVE Red Hat Enterprise MRG Grid 2.1 bug fix and enhancement update 2012-01-23 22:22:58 UTC

Description Timothy St. Clair 2011-05-17 13:48:57 UTC
B/C it is an api, it would be useful if the package would contain the examples + a possible doc to be placed under /usr/share/condor/aviary.  

+ Update examples to include more elaborate submissions
 - DAGS
 - VM 
 - etc.  

For debugging purposes it may be beneficial to supply a -l in the example which dumps the JobAd.

Comment 1 Pete MacKinnon 2011-08-25 13:13:20 UTC
Not sure what you mean by a "-l" option. The getJobDetails method is intended to be a complete JobAd dump.

Also, which package? Runtime or devel?

Comment 4 Martin Kudlej 2011-10-19 15:32:42 UTC
These examples are in package condor-aviary. I've tested them with condor-aviary-7.6.4-0.8 in RHEL 5.8 and 6.1 x x86_64/i386.

rpm -ql condor-aviary | grep "share/condor.*py$"
/usr/share/condor/aviary/dag/dag-submit.py
/usr/share/condor/aviary/jobcontrol.py
/usr/share/condor/aviary/jobquery.py
/usr/share/condor/aviary/module/aviary/__init__.py
/usr/share/condor/aviary/module/aviary/https.py
/usr/share/condor/aviary/module/aviary/util.py
/usr/share/condor/aviary/setattr.py
/usr/share/condor/aviary/submissions.py
/usr/share/condor/aviary/submit.py


$ ./submit.py 
  File "./submit.py", line 25, in ?
    import argparse
ImportError: No module named argparse

$ ./jobcontrol.py --cmd=holdJob --cproc=2.0
  File "./jobcontrol.py", line 79, in ?
    print cmd, 'succeeded'
NameError: name 'cmd' is not defined

same with removeJob and releaseJob.

jobquery.py - OK

submissions.py - OK

$ ./setattr.py 
One of these commands must be supplied
Traceback (most recent call last):
  File "./setattr.py", line 42, in ?
    print 'One of these commands must be supplied', cmds
NameError: name 'cmds' is not defined

$ ./dag-submit.py

<snip/>
  --r=ROOT, --root=ROOT
                        server SSL certificate file
  --s, --verify         enable server certificate verification
  --d=DAG, --dag=DAG    full path to dag file
Traceback (most recent call last):
  File "./dag-submit.py", line 39, in ?
    exit(1)
TypeError: 'str' object is not callable


$ ./dag-submit.py --d=diamond.dag
./dag-submit.py --d=diamond.dag 
invocation failed at:  http://localhost:9090/services/job/submitJob
substring not found

and aviary scheduler plugin is loaded.

I've tried these examples without ssl, because it doesn't work in this version (bug 746251).

Comment 5 Pete MacKinnon 2011-10-20 20:51:31 UTC
script errors fixed upstream

Comment 6 Pete MacKinnon 2011-10-24 14:01:37 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: Desire to expose python client examples of Aviary API.
Consequence: Users did not have practical Aviary API examples. 
Change: Several Aviary example scripts were added to condor-aviary package for job submission, control, attribute set, and queries for status, summary, and details. A dag submission example was also added.
Result: Users now have a practical set of client examples for Aviary written in python using Suds.

Comment 7 Martin Kudlej 2011-10-27 12:35:23 UTC
Tested examples on RHEL 5.7/6.1 x i386/x86_64 with condor-aviary-7.6.5-0.2 and all works(including ssl) except:
$ ./dag-submit.py

<snip/>
  --r=ROOT, --root=ROOT
                        server SSL certificate file
  --s, --verify         enable server certificate verification
  --d=DAG, --dag=DAG    full path to dag file
Traceback (most recent call last):
  File "./dag-submit.py", line 39, in ?
    exit(1)
TypeError: 'str' object is not callable

and there is no error or warning when client is not authenticated to server. Clients(I've tried submissions and submit for this) just stuck.

Configuration: 
SCHEDD.AVIARY_SSL = True
QUERY_SERVER.AVIARY_SSL = True

$ ./submit.py --s #also without --s; we have here wrong certificates or no certificates; I should kill it

^CTraceback (most recent call last):
  File "./submit.py", line 93, in <module>
    extras
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 542, in __call__
    return client.invoke(args, kwargs)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 602, in invoke
    result = self.send(soapenv)
  File "/usr/lib/python2.6/site-packages/suds/client.py", line 643, in send
    reply = transport.send(request)
  File "/usr/lib/python2.6/site-packages/suds/transport/https.py", line 64, in send
    return  HttpTransport.send(self, request)
  File "/usr/lib/python2.6/site-packages/suds/transport/http.py", line 77, in send
    fp = self.u2open(u2request)
  File "/usr/lib/python2.6/site-packages/suds/transport/http.py", line 118, in u2open
    return url.open(u2request, timeout=tm)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1190, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1163, in do_open
    r = h.getresponse()
  File "/usr/lib/python2.6/httplib.py", line 990, in getresponse
    response.begin()
  File "/usr/lib/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.6/socket.py", line 433, in readline
    data = recv(1)
KeyboardInterrupt


Still 2 known bugs. -->ASSIGNED

Comment 8 Pete MacKinnon 2011-10-27 17:47:49 UTC
c7 issue 1 - sys.exit(1) was missing for that path which is a problem in py2.4
c7 issue 2 - introduced a timeout variable for URL connection with a default of 15 seconds, see --help; note cumin hasn't matched my results of this test yet

Comment 11 Martin Kudlej 2011-11-04 16:42:50 UTC
Teste on RHEL 5.7/6.1 x x86_64/i386 with condor-aviary-7.6.5-0.5 and issues from comment 8 are solved. -->VERIFIED

Comment 12 Tomas Capek 2011-11-17 13:08:38 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,4 +1 @@
-Cause: Desire to expose python client examples of Aviary API.
+This update adds several example scripts to the condor-aviary package for job submission, control, attribute set, queries for status, summary, and details. A directed acyclic graph (DAG) submission example has also been added. Users are now provided with a practical set of client examples for the Aviary web service interface written in Python and using the Suds client.-Consequence: Users did not have practical Aviary API examples. 
-Change: Several Aviary example scripts were added to condor-aviary package for job submission, control, attribute set, and queries for status, summary, and details. A dag submission example was also added.
-Result: Users now have a practical set of client examples for Aviary written in python using Suds.

Comment 13 errata-xmlrpc 2012-01-23 17:26:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-0045.html


Note You need to log in before you can comment on or make changes to this bug.