Bug 1771522

Summary: [operator-registry] Version and skiprange fields not populated during insert
Product: OpenShift Container Platform Reporter: lgallett
Component: OLMAssignee: lgallett
OLM sub component: OLM QA Contact: Jian Zhang <jiazha>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: bandrade, chuo, jfan, jiazha, nhale, scolange
Version: 4.3.0   
Target Milestone: ---   
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-23 11:12:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description lgallett 2019-11-12 14:28:09 UTC
Description of problem:

Version and skiprange fields in the operatorbundle table of the operator-registry database are not populated during insert.


Version-Release number of selected component (if applicable):


How reproducible:

Always

Steps to Reproduce:
1. Use any binary to insert operator bundles or manifests into the operator-registry db
2. Open the db using any sqlite db interface
3. Notice the version and skiprange fields that are set in the CSV are not populated in the operatorbundle table

Actual results:

Version and skiprange fields are null

Expected results:

Version and skiprange should contain the corresponding information from the CSV

Additional info:

Comment 2 Jian Zhang 2019-11-14 08:51:27 UTC
Hi, Lance

Could you help provide more details? Actually, I couldn't reproduce this issue. Below are the steps:

1, Build these binaries without the fixed PR, see below:
mac:bin jianzhang$ pwd
/Users/jianzhang/goproject/src/github.com/operator-framework/operator-registry/bin
mac:bin jianzhang$ git branch
* master

mac:bin jianzhang$ git log
commit 237e0d77dd18fee42be455f29ed6b9191f9a6160 (HEAD -> master, origin/master, origin/HEAD)
Merge: 5d342b2 ab7034f
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Nov 8 19:39:32 2019 +0100

    Merge pull request #121 from kevinrizza/fix-docker-image-unpack
    
    Make imagereader work for all layer paths

mac:bin jianzhang$ ls -l
total 441976
-rwxr-xr-x  1 jianzhang  staff  51692152 Nov 13 13:31 appregistry-server
-rw-r--r--  1 jianzhang  staff    118784 Nov 14 16:07 bundles.db
-rwxr-xr-x  1 jianzhang  staff  50706856 Nov 13 13:31 configmap-server
-rwxr-xr-x  1 jianzhang  staff  33671736 Nov 13 13:32 initializer
-rwxr-xr-x  1 jianzhang  staff  51425072 Nov 13 13:32 opm
-rwxr-xr-x  1 jianzhang  staff  34589936 Nov 13 13:32 registry-server

2, Add the "skipRange" in the CSV file, like https://github.com/jianzhangbjz/learn-operator/blob/multi-versions/manifests/learn/0.0.3/learn-operator.v0.0.3.clusterserviceversion.yaml#L5

3, Insert these bundles into SQlite by using the `initializer` binary.
mac:bin jianzhang$ ./initializer --manifests /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 
INFO[0000] loading Bundles                               dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=bundles
INFO[0000] found csv, loading bundle                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundles
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.crd.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.package.yaml load=bundle
INFO[0000] loading Packages and Entries                  dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=package

4, Check the operatorbundle data in this bundles.db.
mac:bin jianzhang$ sqlite3 bundles.db 
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> .tables
api                channel            package          
api_provider       channel_entry      related_image    
api_requirer       operatorbundle     schema_migrations

As you can see the `skipRange` and `version` fields are there:
sqlite> select * from operatorbundle;
learn-operator.v0.0.3|{"apiVersion":"operators.coreos.com/v1alpha1","kind":"ClusterServiceVersion","metadata":{"annotations":{"alm-examples":"[\n  {\n    \"apiVersion\": \"app.learn.com/v1\",\n    \"kind\": \"Learn\",\n    \"metadata\": {\n      \"name\": \"example-learn\"\n    },\n    \"spec\": {\n      \"size\": 2\n    }\n  }\n]","capabilities":"Basic Install","olm.skipRange":"\u003e 0.0.1"},"name":"learn-operator.v0.0.3","namespace":"learn"},"spec":{"apiservicedefinitions":{},"customresourcedefinitions":{"owned":[{"description":"An example to show how to make it managed by OLM","displayName":"Jian's Learn Operator","kind":"Learn","name":"learns.app.learn.com","version":"v1"}]},"description":"Placeholder description","displayName":"Learn Operator","icon":null,"install":{"spec":{"deployments":[{"name":"learn-operator","spec":{"replicas":1,"selector":{"matchLabels":{"name":"learn-operator"}},"strategy":{},"template":{"metadata":{"labels":{"name":"learn-operator"}},"spec":{"containers":[{"command":["learn-operator"],"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.annotations['olm.targetNamespaces']"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"OPERATOR_NAME","value":"learn-operator"}],"image":"quay.io/jiazha/learn-operator-image","imagePullPolicy":"Always","name":"learn-operator","resources":{}}],"serviceAccountName":"learn-operator"}}}}],"permissions":[{"rules":[{"apiGroups":[""],"resources":["pods","services","services/finalizers","endpoints","persistentvolumeclaims","events","configmaps","secrets"],"verbs":["*"]},{"apiGroups":["apps"],"resources":["deployments","daemonsets","replicasets","statefulsets"],"verbs":["*"]},{"apiGroups":["monitoring.coreos.com"],"resources":["servicemonitors"],"verbs":["get","create"]},{"apiGroups":["apps"],"resourceNames":["learn-operator"],"resources":["deployments/finalizers"],"verbs":["update"]},{"apiGroups":[""],"resources":["pods"],"verbs":["get"]},{"apiGroups":["apps"],"resources":["replicasets","deployments"],"verbs":["get"]},{"apiGroups":["app.learn.com"],"resources":["*"],"verbs":["*"]}],"serviceAccountName":"learn-operator"}]},"strategy":"deployment"},"installModes":[{"supported":true,"type":"OwnNamespace"},{"supported":true,"type":"SingleNamespace"},{"supported":false,"type":"MultiNamespace"},{"supported":false,"type":"AllNamespaces"}],"maturity":"alpha","provider":{},"version":"0.0.3"}}
|{"apiVersion":"operators.coreos.com/v1alpha1","kind":"ClusterServiceVersion","metadata":{"annotations":{"alm-examples":"[\n  {\n    \"apiVersion\": \"app.learn.com/v1\",\n    \"kind\": \"Learn\",\n    \"metadata\": {\n      \"name\": \"example-learn\"\n    },\n    \"spec\": {\n      \"size\": 2\n    }\n  }\n]","capabilities":"Basic Install","olm.skipRange":"\u003e 0.0.1"},"name":"learn-operator.v0.0.3","namespace":"learn"},"spec":{"apiservicedefinitions":{},"customresourcedefinitions":{"owned":[{"description":"An example to show how to make it managed by OLM","displayName":"Jian's Learn Operator","kind":"Learn","name":"learns.app.learn.com","version":"v1"}]},"description":"Placeholder description","displayName":"Learn Operator","icon":null,"install":{"spec":{"deployments":[{"name":"learn-operator","spec":{"replicas":1,"selector":{"matchLabels":{"name":"learn-operator"}},"strategy":{},"template":{"metadata":{"labels":{"name":"learn-operator"}},"spec":{"containers":[{"command":["learn-operator"],"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.annotations['olm.targetNamespaces']"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"OPERATOR_NAME","value":"learn-operator"}],"image":"quay.io/jiazha/learn-operator-image","imagePullPolicy":"Always","name":"learn-operator","resources":{}}],"serviceAccountName":"learn-operator"}}}}],"permissions":[{"rules":[{"apiGroups":[""],"resources":["pods","services","services/finalizers","endpoints","persistentvolumeclaims","events","configmaps","secrets"],"verbs":["*"]},{"apiGroups":["apps"],"resources":["deployments","daemonsets","replicasets","statefulsets"],"verbs":["*"]},{"apiGroups":["monitoring.coreos.com"],"resources":["servicemonitors"],"verbs":["get","create"]},{"apiGroups":["apps"],"resourceNames":["learn-operator"],"resources":["deployments/finalizers"],"verbs":["update"]},{"apiGroups":[""],"resources":["pods"],"verbs":["get"]},{"apiGroups":["apps"],"resources":["replicasets","deployments"],"verbs":["get"]},{"apiGroups":["app.learn.com"],"resources":["*"],"verbs":["*"]}],"serviceAccountName":"learn-operator"}]},"strategy":"deployment"},"installModes":[{"supported":true,"type":"OwnNamespace"},{"supported":true,"type":"SingleNamespace"},{"supported":false,"type":"MultiNamespace"},{"supported":false,"type":"AllNamespaces"}],"maturity":"alpha","provider":{},"version":"0.0.3"}}
{"apiVersion":"apiextensions.k8s.io/v1beta1","kind":"CustomResourceDefinition","metadata":{"name":"learns.app.learn.com"},"spec":{"group":"app.learn.com","names":{"kind":"Learn","listKind":"LearnList","plural":"learns","singular":"learn"},"scope":"Namespaced","subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"Learn is the Schema for the learns API","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds","type":"string"},"metadata":{"type":"object"},"spec":{"description":"LearnSpec defines the desired state of Learn","type":"object"},"status":{"description":"LearnStatus defines the observed state of Learn","type":"object"}},"type":"object"}},"version":"v1","versions":[{"name":"v1","served":true,"storage":true}]}}

Comment 3 Jian Zhang 2019-11-18 08:07:54 UTC
Per comment 2, change the status to ASSIGNED first. Please feel free to change it back.

Comment 4 lgallett 2019-11-18 12:45:11 UTC
From what you posted above, I can only see the "olm.skipRange" field that is part of the CSV. What I meant, is the table attributes "skipRange" and "version" of the operatorbundle table are not populated using that same insert flow as above.

Comment 5 lgallett 2019-11-19 18:54:14 UTC
can you run `select skiprange, version from operatorbundle;`?

Comment 6 lgallett 2019-12-02 18:26:32 UTC
Hi Jian,

I don't have enough information in the output you sent above to validate whether the fields are missing. From what I have tested (that is running "select skiprange, version from operatorbundle;" with the same flow you described above), these fields are there - so I will move this back to ON_QA for you to take a look at. Please let me know if I can assist with testing on your end.

Comment 7 Jian Zhang 2019-12-03 09:55:09 UTC
Hi, Lance

Sorry for the late to reply. Thanks for your information. It works well now, details as follows:
mac:operator-registry jianzhang$ git branch
  bug-1751513
  etcd-env
  etcd-role
  etcd-upgrade
* master
mac:operator-registry jianzhang$ git log
commit 4aef889d760333aebac0f01459b3a8bf698fff0b (HEAD -> master, origin/master, origin/HEAD)
Merge: fb1e512 5bae611
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Nov 22 14:44:35 2019 +0100
...

mac:operator-registry jianzhang$ make build
go build   -o bin/appregistry-server ./cmd/appregistry-server
go build   -o bin/configmap-server ./cmd/configmap-server
go build   -o bin/initializer ./cmd/initializer
go build   -o bin/opm ./cmd/opm
go build   -o bin/registry-server ./cmd/registry-server

mac:operator-registry jianzhang$ cd bin/
mac:bin jianzhang$ ./initializer --manifests /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] loading Bundles                               dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=bundles
INFO[0000] found csv, loading bundle                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundles
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.crd.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.package.yaml load=bundle
INFO[0000] loading Packages and Entries                  dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=package

As we can see, we can get the `skiprange` operatorbundle table, LGTM, verify it. 
mac:bin jianzhang$ sqlite3 bundles.db 
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> .table
api                channel            package          
api_provider       channel_entry      related_image    
api_requirer       operatorbundle     schema_migrations
sqlite> select skiprange, version from operatorbundle;
> 0.0.1|0.0.3

Comment 9 errata-xmlrpc 2020-01-23 11:12:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062