Description of problem: With the new hierarchical accounting group support (#1393), users may find it useful to have condor_userprio report stats such that stats for a group include stats for its children (subtree). For example, if the usage for groups "a","a.b","a.c" are 1, 2, 3 respectively, the usage reported for "a" would be 6. (Currently, condor_userprio reports statistics for accounting groups independently of any children). Proposal is to provide condor_userprio with an option to report stats rolled up by accounting group hierarchy. Expected results: ability to roll up stats hierarchically with condor_userprio Additional info: upstream https://condor-wiki.cs.wisc.edu/index.cgi/tktview?tn=1926
Available on tracking branch: UPSTREAM-7.7.3-BZ707771-userprio-rollup
REPRO/TEST This is a new feature, and so there is no repro condition. Consider the following accounting group configuration, with a simple hierarchy: NEGOTIATOR_DEBUG = D_FULLDEBUG NEGOTIATOR_INTERVAL = 30 SCHEDD_INTERVAL = 15 NEGOTIATOR_USE_SLOT_WEIGHTS = FALSE GROUP_QUOTA_MAX_ALLOCATION_ROUNDS = 1 NUM_CPUS = 10 GROUP_NAMES = a, a.b, a.c GROUP_QUOTA_a = 10 GROUP_QUOTA_a.b = 5 GROUP_QUOTA_a.c = 5 GROUP_ACCEPT_SURPLUS = TRUE Submit the following to populate the acct groups with some jobs: universe = vanilla cmd = /bin/sleep args = 6000 should_transfer_files = if_needed when_to_transfer_output = on_exit +AccountingGroup="a.c.u3" queue 1 +AccountingGroup="a.b.u1" queue 1 +AccountingGroup="a.b.u2" queue 1 Wait for the jobs to spin up, and test output of condor_userprio. The updated usage statement shows two new options "-grouporder" and "-grouprollup" $ condor_userprio -h usage: condor_userprio [ -pool hostname ] [ -all | -usage | { -setprio | -setfactor | -setaccum | -setbegin | -setlast } user value | -resetusage user | -resetall | -getreslist user | -delete user ] [-allusers | -activefrom month day year] [-l] [-grouprollup] [-grouporder] Legacy default behavior is preserved (sorting by prio, no rollup, etc): $ condor_userprio -all -allusers Last Priority Update: 9/22 11:36 Effective Real Priority Res Total Usage Usage Last User Name Priority Priority Factor Used (wghted-hrs) Start Time Usage Time ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- <none> 0.50 0.50 1.00 0 0.00 12/31/1969 17:00 ??? a 0.50 0.50 1.00 0 0.00 12/31/1969 17:00 ??? a.c 0.50 0.50 1.00 1 0.02 9/22/2011 11:35 9/22/2011 11:36 a.c.u3@localdomain 0.50 0.50 1.00 1 0.02 9/22/2011 11:35 9/22/2011 11:36 a.b.u2@localdomain 0.50 0.50 1.00 1 0.02 9/22/2011 11:35 9/22/2011 11:36 a.b.u1@localdomain 0.50 0.50 1.00 1 0.02 9/22/2011 11:35 9/22/2011 11:36 a.b 0.50 0.50 1.00 2 0.03 9/22/2011 11:35 9/22/2011 11:36 ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- Number of users: 3 3 0.05 9/22/2011 11:35 ??? Using the new "-grouporder" will place acct groups at top of output, in breadthfirst order (no rollup). Individual submitters are still sorted by priority as usual: $ condor_userprio -all -allusers -grouporder Last Priority Update: 9/22 11:37 Effective Real Priority Res Total Usage Usage Last User Name Priority Priority Factor Used (wghted-hrs) Start Time Usage Time ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- <none> 0.50 0.50 1.00 0 0.00 12/31/1969 17:00 ??? a 0.50 0.50 1.00 0 0.00 12/31/1969 17:00 ??? a.b 0.50 0.50 1.00 2 0.09 9/22/2011 11:35 9/22/2011 11:37 a.c 0.50 0.50 1.00 1 0.04 9/22/2011 11:35 9/22/2011 11:37 a.c.u3@localdomain 0.50 0.50 1.00 1 0.04 9/22/2011 11:35 9/22/2011 11:37 a.b.u2@localdomain 0.50 0.50 1.00 1 0.04 9/22/2011 11:35 9/22/2011 11:37 a.b.u1@localdomain 0.50 0.50 1.00 1 0.04 9/22/2011 11:35 9/22/2011 11:37 ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- Number of users: 3 3 0.13 9/22/2011 11:35 ??? With the "-grouprollup" option, acct group stats are rolled up the hierarchy (note that priority values are zeroed out for groups as rolled up priorities are meaningless). Non-group submitters are ordered as usual. $ condor_userprio -all -allusers -grouporder -grouprollup Last Priority Update: 9/22 11:41 Effective Real Priority Res Total Usage Usage Last User Name Priority Priority Factor Used (wghted-hrs) Start Time Usage Time ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- <none> 0.00 0.00 0.00 3 0.30 12/31/1969 17:00 9/22/2011 11:41 a 0.00 0.00 0.00 3 0.30 12/31/1969 17:00 9/22/2011 11:41 a.b 0.00 0.00 0.00 2 0.20 9/22/2011 11:35 9/22/2011 11:41 a.c 0.00 0.00 0.00 1 0.10 9/22/2011 11:35 9/22/2011 11:41 a.c.u3@localdomain 0.50 0.50 1.00 1 0.10 9/22/2011 11:35 9/22/2011 11:41 a.b.u2@localdomain 0.50 0.50 1.00 1 0.10 9/22/2011 11:35 9/22/2011 11:41 a.b.u1@localdomain 0.50 0.50 1.00 1 0.10 9/22/2011 11:35 9/22/2011 11:41 ------------------------------ --------- -------- ------------ ---- ----------- ---------------- ---------------- Number of users: 3 3 0.30 9/22/2011 11:35 ???
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Cause: Customers wanted to see accounting statistics for accounting groups rolled up by group hierarchy Consequence: Some confusion when accountant reported the unrolled statistics used internally by the negotiator and accountant. Change: Two new options -grouporder and -grouprollup were added to condor_userprio to allow accounting group hierarchies to be displayed and group statistics to be rolled up by hierarchy, respectively. Result: Customers will have the option to view accounting groups and group statistics rolled up by group hierarchy.
Tested on: $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el5 $ $CondorPlatform: I686-RedHat_5.7 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el5 $ $CondorPlatform: X86_64-RedHat_5.7 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el6 $ $CondorPlatform: I686-RedHat_6.1 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el6 $ $CondorPlatform: X86_64-RedHat_6.1 $ If I run 'condor_userprio (-all) -grouprollup' I get completely the same result as if I run 'condor_userprio (-all) -grouprollup -allusers -grouporder'. Is it OK?
> > If I run 'condor_userprio (-all) -grouprollup' I get completely the same result > as if I run 'condor_userprio (-all) -grouprollup -allusers -grouporder'. Is it > OK? Yes, that behavior is consistent with the semantics for -grouprollup and -grouporder
Tested on: $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el5 $ $CondorPlatform: I686-RedHat_5.7 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el5 $ $CondorPlatform: X86_64-RedHat_5.7 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el6 $ $CondorPlatform: I686-RedHat_6.1 $ $CondorVersion: 7.6.4 Oct 14 2011 BuildID: RH-7.6.4-0.8.el6 $ $CondorPlatform: X86_64-RedHat_6.1 $ New options work correctly. >>> VERIFIED
Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,11 +1 @@ -Cause: +With this update, two new options, "-grouporder" and "-grouprollup", have been added to the condor_userprio utility to allow accounting group hierarchies to be displayed and group statistics to be rolled up by hierarchy, respectively.-Customers wanted to see accounting statistics for accounting groups rolled up by group hierarchy - -Consequence: -Some confusion when accountant reported the unrolled statistics used internally by the negotiator and accountant. - -Change: -Two new options -grouporder and -grouprollup were added to condor_userprio to allow accounting group hierarchies to be displayed and group statistics to be rolled up by hierarchy, respectively. - -Result: -Customers will have the option to view accounting groups and group statistics rolled up by group hierarchy.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2012-0045.html