Bug 1797806

Summary:	UPI installs reporting as IPI in telemetry
Product:	OpenShift Container Platform	Reporter:	Alex Crawford <crawford>
Component:	Installer	Assignee:	Alex Crawford <crawford>
Installer sub component:	openshift-installer	QA Contact:	Johnny Liu <jialiu>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	unspecified
Priority:	unspecified	CC:	yanyang
Version:	4.4
Target Milestone:	---
Target Release:	4.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-05-04 11:33:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1802331

Description Alex Crawford 2020-02-03 22:36:53 UTC

When doing a UPI install, the cluster will report as having gone through the IPI installation flow if the OPENSHIFT_INSTALL_INVOKER environment variable is specified. This was the intended behavior from https://bugzilla.redhat.com/show_bug.cgi?id=1791437, but we've since decided that we would like to distinguish between UPI and IPI more accurately. We need to follow up and change the way that the metrics count an IPI vs a UPI installation.

Comment 1 Johnny Liu 2020-02-06 14:13:39 UTC

Reproduce this bug with 4.4.0-0.nightly-2020-02-04-171905.

Trigger UPI install with OPENSHIFT_INSTALL_INVOKER ENV var setting.
After installation, checking:
# oc get cm -n openshift-config openshift-install -o yaml
apiVersion: v1
data:
  invoker: jialiu
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-06T08:57:36Z"
  name: openshift-install
  namespace: openshift-config
  resourceVersion: "963"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install
  uid: 8d32bd2f-ba75-45fd-bb79-642aa011228b


# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`

# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   410  100   410    0     0   3487      0 --:--:-- --:--:-- --:--:--  3504
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.65.75:9099",
          "invoker": "jialiu",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-674c54969c-hkwz2",
          "service": "cluster-version-operator",
          "type": "openshift-install",     -------> this line indicate it is a IPI install (if I am wrong, correct me)
          "version": "v4.4.0"
        },
        "value": [
          1580998128.629,
          "1"
        ]
      }
    ]
  }
}

Comment 4 Johnny Liu 2020-02-10 07:17:08 UTC

Verified this bug with 4.4.0-0.nightly-2020-02-09-220310, and PASS.

Whatever OPENSHIFT_INSTALL_INVOKER set or not, now openshift-install-manifests.yaml file always created when running `openshift-install create manifests` command

# openshift-install create  manifests --dir demo1
# cat demo1/openshift/openshift-install-manifests.yaml 
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: openshift-install-manifests
  namespace: openshift-config

OPENSHIFT_INSTALL_INVOKER value setting will also affect the `invoker` value in this yaml file.
The difference between UPI and IPI is this configmap's name, in UPI it is "openshift-install-manifests", in IPI, this configmap's name is overwritten as "openshift-install".
The configmap name will be used to determinate it is a upi or ipi in telemetry.

Trigger a common IPI install without OPENSHIFT_INSTALL_INVOKER ENV.
# oc get cm -n openshift-config openshift-install -o yaml
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T02:20:27Z"
  name: openshift-install
  namespace: openshift-config
  resourceVersion: "1488"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install
  uid: 23d61d25-e41e-4dc7-8c88-c3490f871f5b

# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   409  100   409    0     0   2543      0 --:--:-- --:--:-- --:--:--  2540
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.128.86:9099",
          "invoker": "user",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-5hnpz",
          "service": "cluster-version-operator",
          "type": "openshift-install",   ===========> indicate this is ipi install
          "version": "v4.4.0"
        },
        "value": [
          1581305053.812,
          "1"
        ]
      }
    ]
  }
}

Trigger a common IPI install without OPENSHIFT_INSTALL_INVOKER ENV.
#  oc get cm openshift-install-manifests -n openshift-config -o yaml
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T03:29:22Z"
  name: openshift-install-manifests
  namespace: openshift-config
  resourceVersion: "1469"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install-manifests
  uid: 488f42ae-a2b9-4dbe-9041-5f0956dcc357
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   344  100   344    0     0   2717      0 --:--:-- --:--:-- --:--:--  2730
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.50.57:9099",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-kl5lh",
          "service": "cluster-version-operator"
        },
        "value": [
          1581307523.39,
          "1"
        ]
      }
    ]
  }
}

Trigger a common IPI install with OPENSHIFT_INSTALL_INVOKER ENV.
# oc get cm -n openshift-config
NAME                               DATA   AGE
admin-kubeconfig-client-ca         1      178m
etcd-ca-bundle                     1      178m
etcd-metric-serving-ca             1      178m
etcd-serving-ca                    1      178m
initial-etcd-ca                    1      178m
initial-kube-apiserver-server-ca   1      178m
openshift-install-manifests        2      178m
#  oc get cm openshift-install-manifests -n openshift-config -o yaml
apiVersion: v1
data:
  invoker: jialiu
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T04:16:31Z"
  name: openshift-install-manifests
  namespace: openshift-config
  resourceVersion: "1401"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install-manifests
  uid: 864d06f4-eb6e-47ef-9a02-babfedcf9f12
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   345  100   345    0     0   2436      0 --:--:-- --:--:-- --:--:--  2446
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.55.97:9099",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-2pzbf",
          "service": "cluster-version-operator"
        },
        "value": [
          1581318899.596,
          "1"
        ]
      }
    ]
  }
}

Comment 6 errata-xmlrpc 2020-05-04 11:33:06 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581