Bug 1797806 - UPI installs reporting as IPI in telemetry
Summary: UPI installs reporting as IPI in telemetry
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.4.0
Assignee: Alex Crawford
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks: 1802331
TreeView+ depends on / blocked
 
Reported: 2020-02-03 22:36 UTC by Alex Crawford
Modified: 2020-05-04 11:33 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-04 11:33:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 3065 0 None closed Bug 1797806: asset/manifests: add new installation config map 2021-01-17 08:15:19 UTC
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:33:36 UTC

Description Alex Crawford 2020-02-03 22:36:53 UTC
When doing a UPI install, the cluster will report as having gone through the IPI installation flow if the OPENSHIFT_INSTALL_INVOKER environment variable is specified. This was the intended behavior from https://bugzilla.redhat.com/show_bug.cgi?id=1791437, but we've since decided that we would like to distinguish between UPI and IPI more accurately. We need to follow up and change the way that the metrics count an IPI vs a UPI installation.

Comment 1 Johnny Liu 2020-02-06 14:13:39 UTC
Reproduce this bug with 4.4.0-0.nightly-2020-02-04-171905.

Trigger UPI install with OPENSHIFT_INSTALL_INVOKER ENV var setting.
After installation, checking:
# oc get cm -n openshift-config openshift-install -o yaml
apiVersion: v1
data:
  invoker: jialiu
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-06T08:57:36Z"
  name: openshift-install
  namespace: openshift-config
  resourceVersion: "963"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install
  uid: 8d32bd2f-ba75-45fd-bb79-642aa011228b


# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`

# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   410  100   410    0     0   3487      0 --:--:-- --:--:-- --:--:--  3504
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.65.75:9099",
          "invoker": "jialiu",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-674c54969c-hkwz2",
          "service": "cluster-version-operator",
          "type": "openshift-install",     -------> this line indicate it is a IPI install (if I am wrong, correct me)
          "version": "v4.4.0"
        },
        "value": [
          1580998128.629,
          "1"
        ]
      }
    ]
  }
}

Comment 4 Johnny Liu 2020-02-10 07:17:08 UTC
Verified this bug with 4.4.0-0.nightly-2020-02-09-220310, and PASS.

Whatever OPENSHIFT_INSTALL_INVOKER set or not, now openshift-install-manifests.yaml file always created when running `openshift-install create manifests` command

# openshift-install create  manifests --dir demo1
# cat demo1/openshift/openshift-install-manifests.yaml 
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: openshift-install-manifests
  namespace: openshift-config

OPENSHIFT_INSTALL_INVOKER value setting will also affect the `invoker` value in this yaml file.
The difference between UPI and IPI is this configmap's name, in UPI it is "openshift-install-manifests", in IPI, this configmap's name is overwritten as "openshift-install".
The configmap name will be used to determinate it is a upi or ipi in telemetry.

Trigger a common IPI install without OPENSHIFT_INSTALL_INVOKER ENV.
# oc get cm -n openshift-config openshift-install -o yaml
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T02:20:27Z"
  name: openshift-install
  namespace: openshift-config
  resourceVersion: "1488"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install
  uid: 23d61d25-e41e-4dc7-8c88-c3490f871f5b

# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   409  100   409    0     0   2543      0 --:--:-- --:--:-- --:--:--  2540
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.128.86:9099",
          "invoker": "user",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-5hnpz",
          "service": "cluster-version-operator",
          "type": "openshift-install",   ===========> indicate this is ipi install
          "version": "v4.4.0"
        },
        "value": [
          1581305053.812,
          "1"
        ]
      }
    ]
  }
}

Trigger a common IPI install without OPENSHIFT_INSTALL_INVOKER ENV.
#  oc get cm openshift-install-manifests -n openshift-config -o yaml
apiVersion: v1
data:
  invoker: user
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T03:29:22Z"
  name: openshift-install-manifests
  namespace: openshift-config
  resourceVersion: "1469"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install-manifests
  uid: 488f42ae-a2b9-4dbe-9041-5f0956dcc357
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   344  100   344    0     0   2717      0 --:--:-- --:--:-- --:--:--  2730
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.50.57:9099",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-kl5lh",
          "service": "cluster-version-operator"
        },
        "value": [
          1581307523.39,
          "1"
        ]
      }
    ]
  }
}

Trigger a common IPI install with OPENSHIFT_INSTALL_INVOKER ENV.
# oc get cm -n openshift-config
NAME                               DATA   AGE
admin-kubeconfig-client-ca         1      178m
etcd-ca-bundle                     1      178m
etcd-metric-serving-ca             1      178m
etcd-serving-ca                    1      178m
initial-etcd-ca                    1      178m
initial-kube-apiserver-server-ca   1      178m
openshift-install-manifests        2      178m
#  oc get cm openshift-install-manifests -n openshift-config -o yaml
apiVersion: v1
data:
  invoker: jialiu
  version: v4.4.0
kind: ConfigMap
metadata:
  creationTimestamp: "2020-02-10T04:16:31Z"
  name: openshift-install-manifests
  namespace: openshift-config
  resourceVersion: "1401"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/openshift-install-manifests
  uid: 864d06f4-eb6e-47ef-9a02-babfedcf9f12
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=cluster_installer' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   345  100   345    0     0   2436      0 --:--:-- --:--:-- --:--:--  2446
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "cluster_installer",
          "endpoint": "metrics",
          "instance": "10.0.55.97:9099",
          "job": "cluster-version-operator",
          "namespace": "openshift-cluster-version",
          "pod": "cluster-version-operator-54fd5685f7-2pzbf",
          "service": "cluster-version-operator"
        },
        "value": [
          1581318899.596,
          "1"
        ]
      }
    ]
  }
}

Comment 6 errata-xmlrpc 2020-05-04 11:33:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.