Bug 1883377

Summary: opm registry add fails with FOREIGN KEY constraint when bundle default channel is empty
Product: OpenShift Container Platform Reporter: Jesus M. Rodriguez <jesusr>
Component: OLMAssignee: Vu Dinh <vdinh>
OLM sub component: OLM QA Contact: Bruno Andrade <bandrade>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: bandrade, bluddy, krizza
Version: 4.6   
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:45:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jesus M. Rodriguez 2020-09-29 00:53:13 UTC
Description of problem:
opm registry add fails with FOREIGN KEY constraint when bundle with an empty default channel ("") is supplied.

Version-Release number of selected component (if applicable):


How reproducible:
bin/opm registry add -d database/index.db -b quay.io/vnarsing/memcached-operator:v1 --mode=semver

I updated my local copy to track down the problem, here is a bunch of debug logs:

https://gist.github.com/jmrodri/9b7c61f098922704aa55c8f5bf5cf304#file-empty-default-channel-L23-L24

Actual results:
[jesusr@transam operator-registry{debug}]$ bin/opm registry add -d database/index.db -b quay.io/vnarsing/memcached-operator:v1 --mode=semver
INFO[0000] adding to the registry                        bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0000] resolved name: quay.io/vnarsing/memcached-operator:v1 
INFO[0000] fetched                                       digest="sha256:af6d5164bc7f2d6cbaffa6512e6f75c2a8f006023abae3a99464e2cac72dc79f"
INFO[0000] fetched                                       digest="sha256:9e28e44ee9e02f864a5979bb1352a9f2cde5988b19d90d8291f792947d6473be"
INFO[0000] fetched                                       digest="sha256:cb0f79f77243fc0308b0f392dc8a3cf04b2f1efda4ca6047c1a56c7f3d741d34"
INFO[0000] fetched                                       digest="sha256:df87481a09e9bb57fb7aff5143fde82f2141e69c8f0de4843696da047bc15d98"
INFO[0000] fetched                                       digest="sha256:bed2176ad0d92c8a90c3faade1df343420bb7de1dc57add2b3d6ed60883563e1"
INFO[0000] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:9e28e44ee9e02f864a5979bb1352a9f2cde5988b19d90d8291f792947d6473be 2075 [] map[] <nil>} 
INFO[0000] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:bed2176ad0d92c8a90c3faade1df343420bb7de1dc57add2b3d6ed60883563e1 379 [] map[] <nil>} 
INFO[0000] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:cb0f79f77243fc0308b0f392dc8a3cf04b2f1efda4ca6047c1a56c7f3d741d34 476 [] map[] <nil>} 
INFO[0000] Could not find optional dependencies file     dir=bundle_tmp045653365 file=bundle_tmp045653365/metadata load=annotations
INFO[0000] found csv, loading bundle                     dir=bundle_tmp045653365 file=bundle_tmp045653365/manifests load=bundle
INFO[0000] loading bundle file                           dir=bundle_tmp045653365/manifests file=barktgx.example.comktgx_fooktgxes.yaml load=bundle
INFO[0000] loading bundle file                           dir=bundle_tmp045653365/manifests file=e2e-ktgx-metrics-reader_rbac.authorization.k8s.io_v1beta1_clusterrole.yaml load=bundle
INFO[0000] loading bundle file                           dir=bundle_tmp045653365/manifests file=e2e-ktgx.clusterserviceversion.yaml load=bundle
ERRO[0000] permissive mode disabled                      bundles="[quay.io/vnarsing/memcached-operator:v1]" error="loadManifestsSemver: error loading bundle into db: FOREIGN KEY constraint failed"
Error: loadManifestsSemver: error loading bundle into db: FOREIGN KEY constraint failed
Usage:
  opm registry add [flags]

Flags:
  -b, --bundle-images strings   comma separated list of links to bundle image
  -c, --container-tool string   tool to interact with container images (save, build, etc.). One of: [none, docker, podman] (default "none")
  -d, --database string         relative path to database file (default "bundles.db")
      --debug                   enable debug logging
  -h, --help                    help for add
      --mode string             graph update mode that defines how channel graphs are updated. One of: [replaces, semver, semver-skippatch] (default "replaces")
      --permissive              allow registry load errors

Global Flags:
      --skip-tls   skip TLS certificate verification for container image registries while pulling bundles or index

Expected results:

[jesusr@transam operator-registry{debug}]$ bin/opm registry add -d database/index.db -b quay.io/joelanford/example-operator-bundle:0.1.0 --mode=semver
INFO[0000] adding to the registry                        bundles="[quay.io/joelanford/example-operator-bundle:0.1.0]"
INFO[0000] resolved name: quay.io/joelanford/example-operator-bundle:0.1.0 
INFO[0000] fetched                                       digest="sha256:19666955b8400a6faaab9a1bc15510ce4d1681e33ec034b78831e890e9bdd510"
INFO[0000] fetched                                       digest="sha256:4a34196d00c2f6e00f8501b6da547575bda513061954b32dfea511478d1cdcfc"
INFO[0000] fetched                                       digest="sha256:1eb69b507afc07bc4fcf5ddc264a1b66511a9c96fef52a79afc3b69d29d8ffb6"
INFO[0000] fetched                                       digest="sha256:2d74147b6c4419b30c32fda6b9a30395098308e903190dc2e9fdf39818dbcd36"
INFO[0001] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:2d74147b6c4419b30c32fda6b9a30395098308e903190dc2e9fdf39818dbcd36 1695 [] map[] <nil>} 
INFO[0001] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:1eb69b507afc07bc4fcf5ddc264a1b66511a9c96fef52a79afc3b69d29d8ffb6 265 [] map[] <nil>} 
INFO[0001] Could not find optional dependencies file     dir=bundle_tmp026908200 file=bundle_tmp026908200/metadata load=annotations
INFO[0001] found csv, loading bundle                     dir=bundle_tmp026908200 file=bundle_tmp026908200/manifests load=bundle
INFO[0001] loading bundle file                           dir=bundle_tmp026908200/manifests file=example-operator.clusterserviceversion.yaml load=bundle
INFO[0001] loading bundle file                           dir=bundle_tmp026908200/manifests file=example.com_apps_crd.yaml load=bundle

Additional info:

I've tracked the problem down to updateDefaultChannel: https://github.com/operator-framework/operator-registry/blob/e8dbb711ecf73461c0b36599eaaf0f43297cb8c0/pkg/sqlite/loadprocs.go#L125 that fails with the FOREIGN KEY constraint

Comment 1 Jesus M. Rodriguez 2020-09-29 00:54:47 UTC
This will block: operator-sdk run bundle or anyone with a bundle and a blank default channel.

Comment 2 Vu Dinh 2020-09-30 16:03:58 UTC
Hey Jesus,

It seems you are adding the first bundle for a new package without default channel. Can you confirm that? The default channel is required when a new package is created (via the first bundle addition). So the first bundle needs to have default channel information. Then, subsequent bundles may not have default channel if there is no change in default channel setting.

I tried to add bundles in semver mode using proper order (first bundle with default channel and second one without) and it worked just fine. So this is not a bug.

What I can improve here is to handle the error a bit better so it won't go too far down to the DB transaction level and add a default channel inference that is similar in replace mode (if you have only one channel in channel list then we will pick the that channel as default if you add the first bundle in the package).

Vu

Comment 3 Jesus M. Rodriguez 2020-09-30 16:14:20 UTC
It is the first bundle and likely the only bundle. One of our use cases is to run a bundle without specifying an index image. We use the builder to create an ephemeral index.

Comment 4 Jesus M. Rodriguez 2020-09-30 18:51:33 UTC
based on a lengthy slack discussion:

the best thing to do here is, in this case, if the field isn't set just pick it lexicographically like what Vu described in comment #2.

Comment 6 Bruno Andrade 2020-10-02 11:45:01 UTC
Marking as VERIFIED.

$ bin/opm version                                                                                   
Version: version.Version{OpmVersion:"v1.14.3-5-gf6e5d92", GitCommit:"f6e5d92", BuildDate:"2020-10-02T11:29:08Z", GoOs:"linux", GoArch:"amd6

$ bin/opm registry add -d index.db -b quay.io/vnarsing/memcached-operator:v1 --mode=semver -c podman
INFO[0000] adding to the registry                        bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0000] running /usr/bin/podman pull quay.io/vnarsing/memcached-operator:v1  bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0005] running podman create                         bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0005] running podman cp                             bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0005] running podman rm                             bundles="[quay.io/vnarsing/memcached-operator:v1]"
INFO[0005] Could not find optional dependencies file     dir=bundle_tmp214709305 file=bundle_tmp214709305/metadata load=annotations
INFO[0005] found csv, loading bundle                     dir=bundle_tmp214709305 file=bundle_tmp214709305/manifests load=bundle
INFO[0005] loading bundle file                           dir=bundle_tmp214709305/manifests file=barlyvt.example.comlyvt_foolyvts.yaml load=bundle
INFO[0005] loading bundle file                           dir=bundle_tmp214709305/manifests file=e2e-lyvt-metrics-reader_rbac.authorization.k8s.io_v1beta1_clusterrole.yaml load=bundle
INFO[0005] loading bundle file                           dir=bundle_tmp214709305/manifests file=e2e-lyvt.clusterserviceversion.yaml load=bundle

Inference worked successfully.

Comment 9 errata-xmlrpc 2020-10-27 16:45:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196