Bug 1771522 - [operator-registry] Version and skiprange fields not populated during insert
Summary: [operator-registry] Version and skiprange fields not populated during insert
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3.0
Assignee: lgallett
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-12 14:28 UTC by lgallett
Modified: 2020-01-23 11:12 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-23 11:12:22 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github operator-framework operator-registry pull 122 0 'None' closed Bug 1771522: add version and skiprange to operatorbundle table 2020-04-08 03:38:34 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:12:32 UTC

Description lgallett 2019-11-12 14:28:09 UTC
Description of problem:

Version and skiprange fields in the operatorbundle table of the operator-registry database are not populated during insert.


Version-Release number of selected component (if applicable):


How reproducible:

Always

Steps to Reproduce:
1. Use any binary to insert operator bundles or manifests into the operator-registry db
2. Open the db using any sqlite db interface
3. Notice the version and skiprange fields that are set in the CSV are not populated in the operatorbundle table

Actual results:

Version and skiprange fields are null

Expected results:

Version and skiprange should contain the corresponding information from the CSV

Additional info:

Comment 2 Jian Zhang 2019-11-14 08:51:27 UTC
Hi, Lance

Could you help provide more details? Actually, I couldn't reproduce this issue. Below are the steps:

1, Build these binaries without the fixed PR, see below:
mac:bin jianzhang$ pwd
/Users/jianzhang/goproject/src/github.com/operator-framework/operator-registry/bin
mac:bin jianzhang$ git branch
* master

mac:bin jianzhang$ git log
commit 237e0d77dd18fee42be455f29ed6b9191f9a6160 (HEAD -> master, origin/master, origin/HEAD)
Merge: 5d342b2 ab7034f
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Nov 8 19:39:32 2019 +0100

    Merge pull request #121 from kevinrizza/fix-docker-image-unpack
    
    Make imagereader work for all layer paths

mac:bin jianzhang$ ls -l
total 441976
-rwxr-xr-x  1 jianzhang  staff  51692152 Nov 13 13:31 appregistry-server
-rw-r--r--  1 jianzhang  staff    118784 Nov 14 16:07 bundles.db
-rwxr-xr-x  1 jianzhang  staff  50706856 Nov 13 13:31 configmap-server
-rwxr-xr-x  1 jianzhang  staff  33671736 Nov 13 13:32 initializer
-rwxr-xr-x  1 jianzhang  staff  51425072 Nov 13 13:32 opm
-rwxr-xr-x  1 jianzhang  staff  34589936 Nov 13 13:32 registry-server

2, Add the "skipRange" in the CSV file, like https://github.com/jianzhangbjz/learn-operator/blob/multi-versions/manifests/learn/0.0.3/learn-operator.v0.0.3.clusterserviceversion.yaml#L5

3, Insert these bundles into SQlite by using the `initializer` binary.
mac:bin jianzhang$ ./initializer --manifests /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 
INFO[0000] loading Bundles                               dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=bundles
INFO[0000] found csv, loading bundle                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundles
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.crd.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.package.yaml load=bundle
INFO[0000] loading Packages and Entries                  dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=package

4, Check the operatorbundle data in this bundles.db.
mac:bin jianzhang$ sqlite3 bundles.db 
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> .tables
api                channel            package          
api_provider       channel_entry      related_image    
api_requirer       operatorbundle     schema_migrations

As you can see the `skipRange` and `version` fields are there:
sqlite> select * from operatorbundle;
learn-operator.v0.0.3|{"apiVersion":"operators.coreos.com/v1alpha1","kind":"ClusterServiceVersion","metadata":{"annotations":{"alm-examples":"[\n  {\n    \"apiVersion\": \"app.learn.com/v1\",\n    \"kind\": \"Learn\",\n    \"metadata\": {\n      \"name\": \"example-learn\"\n    },\n    \"spec\": {\n      \"size\": 2\n    }\n  }\n]","capabilities":"Basic Install","olm.skipRange":"\u003e 0.0.1"},"name":"learn-operator.v0.0.3","namespace":"learn"},"spec":{"apiservicedefinitions":{},"customresourcedefinitions":{"owned":[{"description":"An example to show how to make it managed by OLM","displayName":"Jian's Learn Operator","kind":"Learn","name":"learns.app.learn.com","version":"v1"}]},"description":"Placeholder description","displayName":"Learn Operator","icon":null,"install":{"spec":{"deployments":[{"name":"learn-operator","spec":{"replicas":1,"selector":{"matchLabels":{"name":"learn-operator"}},"strategy":{},"template":{"metadata":{"labels":{"name":"learn-operator"}},"spec":{"containers":[{"command":["learn-operator"],"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.annotations['olm.targetNamespaces']"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"OPERATOR_NAME","value":"learn-operator"}],"image":"quay.io/jiazha/learn-operator-image","imagePullPolicy":"Always","name":"learn-operator","resources":{}}],"serviceAccountName":"learn-operator"}}}}],"permissions":[{"rules":[{"apiGroups":[""],"resources":["pods","services","services/finalizers","endpoints","persistentvolumeclaims","events","configmaps","secrets"],"verbs":["*"]},{"apiGroups":["apps"],"resources":["deployments","daemonsets","replicasets","statefulsets"],"verbs":["*"]},{"apiGroups":["monitoring.coreos.com"],"resources":["servicemonitors"],"verbs":["get","create"]},{"apiGroups":["apps"],"resourceNames":["learn-operator"],"resources":["deployments/finalizers"],"verbs":["update"]},{"apiGroups":[""],"resources":["pods"],"verbs":["get"]},{"apiGroups":["apps"],"resources":["replicasets","deployments"],"verbs":["get"]},{"apiGroups":["app.learn.com"],"resources":["*"],"verbs":["*"]}],"serviceAccountName":"learn-operator"}]},"strategy":"deployment"},"installModes":[{"supported":true,"type":"OwnNamespace"},{"supported":true,"type":"SingleNamespace"},{"supported":false,"type":"MultiNamespace"},{"supported":false,"type":"AllNamespaces"}],"maturity":"alpha","provider":{},"version":"0.0.3"}}
|{"apiVersion":"operators.coreos.com/v1alpha1","kind":"ClusterServiceVersion","metadata":{"annotations":{"alm-examples":"[\n  {\n    \"apiVersion\": \"app.learn.com/v1\",\n    \"kind\": \"Learn\",\n    \"metadata\": {\n      \"name\": \"example-learn\"\n    },\n    \"spec\": {\n      \"size\": 2\n    }\n  }\n]","capabilities":"Basic Install","olm.skipRange":"\u003e 0.0.1"},"name":"learn-operator.v0.0.3","namespace":"learn"},"spec":{"apiservicedefinitions":{},"customresourcedefinitions":{"owned":[{"description":"An example to show how to make it managed by OLM","displayName":"Jian's Learn Operator","kind":"Learn","name":"learns.app.learn.com","version":"v1"}]},"description":"Placeholder description","displayName":"Learn Operator","icon":null,"install":{"spec":{"deployments":[{"name":"learn-operator","spec":{"replicas":1,"selector":{"matchLabels":{"name":"learn-operator"}},"strategy":{},"template":{"metadata":{"labels":{"name":"learn-operator"}},"spec":{"containers":[{"command":["learn-operator"],"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.annotations['olm.targetNamespaces']"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"OPERATOR_NAME","value":"learn-operator"}],"image":"quay.io/jiazha/learn-operator-image","imagePullPolicy":"Always","name":"learn-operator","resources":{}}],"serviceAccountName":"learn-operator"}}}}],"permissions":[{"rules":[{"apiGroups":[""],"resources":["pods","services","services/finalizers","endpoints","persistentvolumeclaims","events","configmaps","secrets"],"verbs":["*"]},{"apiGroups":["apps"],"resources":["deployments","daemonsets","replicasets","statefulsets"],"verbs":["*"]},{"apiGroups":["monitoring.coreos.com"],"resources":["servicemonitors"],"verbs":["get","create"]},{"apiGroups":["apps"],"resourceNames":["learn-operator"],"resources":["deployments/finalizers"],"verbs":["update"]},{"apiGroups":[""],"resources":["pods"],"verbs":["get"]},{"apiGroups":["apps"],"resources":["replicasets","deployments"],"verbs":["get"]},{"apiGroups":["app.learn.com"],"resources":["*"],"verbs":["*"]}],"serviceAccountName":"learn-operator"}]},"strategy":"deployment"},"installModes":[{"supported":true,"type":"OwnNamespace"},{"supported":true,"type":"SingleNamespace"},{"supported":false,"type":"MultiNamespace"},{"supported":false,"type":"AllNamespaces"}],"maturity":"alpha","provider":{},"version":"0.0.3"}}
{"apiVersion":"apiextensions.k8s.io/v1beta1","kind":"CustomResourceDefinition","metadata":{"name":"learns.app.learn.com"},"spec":{"group":"app.learn.com","names":{"kind":"Learn","listKind":"LearnList","plural":"learns","singular":"learn"},"scope":"Namespaced","subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"Learn is the Schema for the learns API","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds","type":"string"},"metadata":{"type":"object"},"spec":{"description":"LearnSpec defines the desired state of Learn","type":"object"},"status":{"description":"LearnStatus defines the observed state of Learn","type":"object"}},"type":"object"}},"version":"v1","versions":[{"name":"v1","served":true,"storage":true}]}}

Comment 3 Jian Zhang 2019-11-18 08:07:54 UTC
Per comment 2, change the status to ASSIGNED first. Please feel free to change it back.

Comment 4 lgallett 2019-11-18 12:45:11 UTC
From what you posted above, I can only see the "olm.skipRange" field that is part of the CSV. What I meant, is the table attributes "skipRange" and "version" of the operatorbundle table are not populated using that same insert flow as above.

Comment 5 lgallett 2019-11-19 18:54:14 UTC
can you run `select skiprange, version from operatorbundle;`?

Comment 6 lgallett 2019-12-02 18:26:32 UTC
Hi Jian,

I don't have enough information in the output you sent above to validate whether the fields are missing. From what I have tested (that is running "select skiprange, version from operatorbundle;" with the same flow you described above), these fields are there - so I will move this back to ON_QA for you to take a look at. Please let me know if I can assist with testing on your end.

Comment 7 Jian Zhang 2019-12-03 09:55:09 UTC
Hi, Lance

Sorry for the late to reply. Thanks for your information. It works well now, details as follows:
mac:operator-registry jianzhang$ git branch
  bug-1751513
  etcd-env
  etcd-role
  etcd-upgrade
* master
mac:operator-registry jianzhang$ git log
commit 4aef889d760333aebac0f01459b3a8bf698fff0b (HEAD -> master, origin/master, origin/HEAD)
Merge: fb1e512 5bae611
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Nov 22 14:44:35 2019 +0100
...

mac:operator-registry jianzhang$ make build
go build   -o bin/appregistry-server ./cmd/appregistry-server
go build   -o bin/configmap-server ./cmd/configmap-server
go build   -o bin/initializer ./cmd/initializer
go build   -o bin/opm ./cmd/opm
go build   -o bin/registry-server ./cmd/registry-server

mac:operator-registry jianzhang$ cd bin/
mac:bin jianzhang$ ./initializer --manifests /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] loading Bundles                               dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=bundles
INFO[0000] found csv, loading bundle                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundles
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn-operator.v0.0.3.clusterserviceversion.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.crd.yaml load=bundle
INFO[0000] loading bundle file                           dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=learn.package.yaml load=bundle
INFO[0000] loading Packages and Entries                  dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3
INFO[0000] directory                                     dir=/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.3 file=0.0.3 load=package

As we can see, we can get the `skiprange` operatorbundle table, LGTM, verify it. 
mac:bin jianzhang$ sqlite3 bundles.db 
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> .table
api                channel            package          
api_provider       channel_entry      related_image    
api_requirer       operatorbundle     schema_migrations
sqlite> select skiprange, version from operatorbundle;
> 0.0.1|0.0.3

Comment 9 errata-xmlrpc 2020-01-23 11:12:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.