Bug 1270176

Summary: Python PMAPI pmSetOptionHostList no workie
Product: [Fedora] Fedora Reporter: Marko Myllynen <myllynen>
Component: pcpAssignee: Nathan Scott <nathans>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: brolley, fche, lberk, mgoodwin, nathans, pcp, scox
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pcp-3.10.9-1.fc23 pcp-3.10.9-1.fc22 pcp-3.11.1-1.el5 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-06 14:55:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
pmsetopt-list-test.py test script none

Description Marko Myllynen 2015-10-09 08:23:44 UTC
Description of problem:
With something like:

    def connect(self):
        """ Establish a PMAPI context to archive, host or local, via args """
        self.opts.pmSetOptionHostList("localhost")
        self.context = pmapi.pmContext.fromOptions(self.opts, sys.argv)

I get from a Python client (without -h in use):

pcp: too many hosts requested: localhost

Using a list instead of a string won't help and in fact in pmapi.py we see:

    def pmSetOptionHostList(self, hosts):       # str
        return c_api.pmSetOptionHostList(hosts)

The same happens with pmSetOptionArchiveList.

Comment 1 Nathan Scott 2015-10-12 01:20:59 UTC
There's a bit of a conceptual disconnect here, I think Marko...

|        self.context = pmapi.pmContext.fromOptions(self.opts, sys.argv)

The pmContext fromOptions helper interface is intended for simpler tools (the vast majority of PCP tools fall into this catgeory though), where there is only one source of metrics - a live host, or a single archive.

We'll need a higher level python API (i.e. pcp.pmcc rather than pcp.pmapi) to be able to construct multiple pmContext objects from a list of hosts.

So, the pmOptions class supports this notion of multiple sources (from the command line and via the pmSetOptionHost/ArchiveList pmOption interfaces), but there's nothing so far to turn that into a class that managed multiple contexts - pcp.pmContext is not that class.

Comment 2 Marko Myllynen 2015-10-12 05:41:31 UTC
(In reply to Nathan Scott from comment #1)
> There's a bit of a conceptual disconnect here, I think Marko...

Sorry for being horribly unclear here.

pcp.pmapi makes it clear that multiple contexts are not supported and I wasn't after multiple contexts support but rather a way to specify the source for a context in the same way one can specify e.g. interval or samples for the context (now the only way to specify a host to be used for a context seems to be using via the command line option). Ok, there's a difference of changing parameters like interval compared to changing the source for a context but given that pmSetOptionHostList() exists, I thought using it with one host would be possible prior context creation as above (otherwise I wonder why it exists at all in Python PMAPI?).

A practical use case would be to allow supporting "source" configuration file directive in pmrep configuration file.

Thanks.

Comment 3 Nathan Scott 2015-10-12 07:41:05 UTC
(In reply to Marko Myllynen from comment #2)
> (In reply to Nathan Scott from comment #1)
> > There's a bit of a conceptual disconnect here, I think Marko...
> 
> Sorry for being horribly unclear here.
> 

Ah, I see where you're coming from now.

> pcp.pmapi makes it clear that multiple contexts are not supported and I
> wasn't after multiple contexts support but rather a way to specify the

Well, there is some support - even the fromOptions context builder interface allows for multiple contexts, via iterative calling (there are several non-default parameters there, one of which is "index").  But anyway, not what you're after here.

> source for a context but given that pmSetOptionHostList() exists, I thought
> using it with one host would be possible prior context creation as above
> (otherwise I wonder why it exists at all in Python PMAPI?).
> 
> A practical use case would be to allow supporting "source" configuration
> file directive in pmrep configuration file.

Yep, I see where you're coming from now - not sure off the top of my head why your hostlist approach didn't work.  Could you make a little minimal test case script & send it through (will save me some time) - I'll take a look tomorrow.

cheers.

Comment 4 Marko Myllynen 2015-10-12 09:10:50 UTC
Created attachment 1081923 [details]
pmsetopt-list-test.py test script

A quick and simple test script attached. Setting archive folio works but setting host/archive fails. Interval setting example also included to show optimal outcome, default set in the script can be overridden from the command line.

Thanks.

Comment 5 Nathan Scott 2015-10-13 03:18:21 UTC
(In reply to Marko Myllynen from comment #4)
> 
> A quick and simple test script attached. Setting archive folio works but
> setting host/archive fails.

Thanks Marko.  The reason it fails with --host-list and --archive-list with that error (e.g. "too many hosts requested") is that these options require the script to be flagged as allowing multiple contexts via PM_OPTFLAG_MULTI.

So, the error is actually coming out of the libpcp getopts code, which is perhaps a little too strict in the case of a host/archive list with only one element as you built here.  I'll fix that up to make it behave more intuitively.

However, I think what you really want here is an interface for setting just one source (then you wouldn't have got thrown a red herring by the list interfaces) - these single-source interfaces exist in libpcp, but are missing in the python API.  With this, setting a host from your config file will be just like setting sample interval.

I've converted your little script into a regression test for these new python interfaces, and written the new API code - will commit shortly.

Comment 6 Marko Myllynen 2015-10-13 06:24:43 UTC
(In reply to Nathan Scott from comment #5)
> 
> However, I think what you really want here is an interface for setting just
> one source (then you wouldn't have got thrown a red herring by the list
> interfaces) - these single-source interfaces exist in libpcp, but are
> missing in the python API.  With this, setting a host from your config file
> will be just like setting sample interval.

Yes, this sounds perfect, thanks!

Comment 7 Fedora Update System 2015-10-29 21:51:15 UTC
pcp-3.10.8-1.fc21 has been submitted as an update to Fedora 21. https://bodhi.fedoraproject.org/updates/FEDORA-2015-3fc0cf98a1

Comment 8 Fedora Update System 2015-10-29 21:51:18 UTC
pcp-3.10.8-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-b1cbfc4842

Comment 9 Fedora Update System 2015-10-29 21:51:33 UTC
pcp-3.10.8-1.fc23.1 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-8f41fd0048

Comment 10 Fedora Update System 2015-10-29 21:51:35 UTC
pcp-3.10.8-1.el5 has been submitted as an update to Fedora EPEL 5. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-60b17897b4

Comment 11 Fedora Update System 2015-11-01 06:57:12 UTC
pcp-3.10.8-1.fc23.1 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-8f41fd0048

Comment 12 Fedora Update System 2015-11-01 16:48:03 UTC
pcp-3.10.8-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'yum --enablerepo=epel-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-60b17897b4

Comment 13 Fedora Update System 2015-11-02 00:25:51 UTC
pcp-3.10.8-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-b1cbfc4842

Comment 14 Fedora Update System 2015-11-02 02:54:07 UTC
pcp-3.10.8-1.fc21 has been pushed to the Fedora 21 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-3fc0cf98a1

Comment 15 Fedora Update System 2015-12-17 01:55:13 UTC
pcp-3.10.9-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d08245c076

Comment 16 Fedora Update System 2015-12-17 01:55:51 UTC
pcp-3.10.9-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-2b40815137

Comment 17 Fedora Update System 2015-12-17 02:18:37 UTC
pcp-3.10.9-1.el5 has been submitted as an update to Fedora EPEL 5. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-2ac90519bc

Comment 18 Fedora Update System 2015-12-17 10:27:01 UTC
pcp-3.10.9-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-2b40815137

Comment 19 Fedora Update System 2015-12-17 10:27:20 UTC
pcp-3.10.9-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d08245c076

Comment 20 Fedora Update System 2015-12-17 12:49:26 UTC
pcp-3.10.9-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'yum --enablerepo=epel-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-2ac90519bc

Comment 21 Fedora Update System 2016-01-05 21:58:16 UTC
pcp-3.10.9-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 22 Fedora Update System 2016-01-05 22:54:53 UTC
pcp-3.10.9-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Comment 23 Fedora Update System 2016-02-03 03:57:17 UTC
pcp-3.11.0-1.el5 has been submitted as an update to Fedora EPEL 5. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-57b7efb2d7

Comment 24 Fedora Update System 2016-02-03 22:17:51 UTC
pcp-3.11.0-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-57b7efb2d7

Comment 25 Fedora Update System 2016-03-22 00:18:08 UTC
pcp-3.11.1-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-5b519318e0

Comment 26 Fedora Update System 2016-04-06 14:54:22 UTC
pcp-3.11.1-1.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.