Bug 1293988

+++ This bug was initially created as a clone of Bug #1253726 +++ But is designed to be an abstraction "so that the same concepts used by EAP can be used by developers for other application types / frameworks 

USECASE: That is not defined: 
  As a "Developer" I would like to write an application that is "scalable", and has the ability to use "libraries/tools" to form a "cluster" so that data between my pods is replicated or shared (in case one pod is lost). How do I "discover" or get "notifications" about members of my cluster (should match the number of "pods" defined in cluster settings). 

Document URL: 
    NONE

Section Number and Name: 
    No section. I'm suggesting a new one.

Describe the issue: 

The issue here is we do not describe how a "developer" from with in a pod can / should collect information from the Kubernetties or OpenShift API's (events) to be notified or see what "peers" (pods) it has. 

Suggestions for improvement: 

Create a documenation section that "documents" how EAP / JGroups uses the KUBE_PING or DNS_PING protocols to query the OpenShift and Kubernetties API's to see members of the cluster. 

EAP gets the list of pods from kubernetes, in the specified namespace, and filtered by application label.

    For example: https://HOST:443/api/v1/namespaces/NAMESPACE/pods?labelSelector=application%3APPLICATION-NAME

JGroups then parses the JSON it gets back from the API call to form the cluster. 

source code is here: https://github.com/jboss-openshift/openshift-ping/tree/1.0.0.Beta8/kube/src/main/java/org/openshift/ping/kube
    (Look at KubePing.java and Client.java , mostly)

Additional information:

https://docs.openshift.com/enterprise/3.1/rest_api/kubernetes_v1.html#list-or-watch-objects-of-kind-endpoints

Can likely be used to watched a stream of add, update, and remove notifications. However we need to explain to customer how a "service account" or SCC needs to be created / enabled to access these endpoints. 

If I am not mistaken EAP enables Authorization (to the API), which must be granted to the service account the pod is running under to be allowed to access Kubernetes' REST api. This is done on the command line. These are needed to get the KUBE_PING to work or to allow a pod to access the kube service. So this too must be explained / elaborated on so developers can be successful. 

Example 1. Policy commands
http://ce-docs.usersys.redhat.com/openshift/eap/6.4/clustering.html

Using the default service account in the myproject namespace:

    # oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

Using the eap-service-account in the myproject namespace:

    # oc policy add-role-to-user view system:serviceaccount:$(oc project -q):eap-service-account -n $(oc project -q)