have a generic plugin that can execute any CLI executable.

You can configure what it means that a  CLI is available (if its exit code matches a value, or its output matches a value).
You can configure what a CLI's version is (have regex matching of output to pick out a version string) - same with description.
You can have generic operations by defining the arguments to pass and the results capture the exit code and output.
You can define metric definitions by defining the arguments to pass and the regex to pick out the values.