Description of problem: The index image is created but, the database is empty when a package has issues and --permissive flag is added: opm index add --bundles quay.io/bandrade/lib-bucket-provisioner:1.0,quay.io/bandrade/planetscale:0.1.8 --tag quay.io/bandrade/multi-index:3.0 --permissive INFO[0000] building the index bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0001] resolved name: quay.io/bandrade/lib-bucket-provisioner:1.0 INFO[0001] fetched digest="sha256:bab1b404feef4c0ad2d2ffb51031ed03e3c1ca55d7255340bc0212c303a4fbe4" INFO[0001] fetched digest="sha256:b43fb7ebb5b4aaa53d4428bd375438805df68c2a721d8716cce965bda63a1f05" INFO[0001] fetched digest="sha256:791c595abfd01d142a6bb29018900fdd5698095b2fc95e3dfd5023b9ceefa3f6" INFO[0001] fetched digest="sha256:5c1e1bbf8aa344143cc2378d77e86c7f11908041d45e239ef6930916ff81d1bf" INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:5c1e1bbf8aa344143cc2378d77e86c7f11908041d45e239ef6930916ff81d1bf 130482 [] map[] <nil>} INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:b43fb7ebb5b4aaa53d4428bd375438805df68c2a721d8716cce965bda63a1f05 296 [] map[] <nil>} INFO[0003] resolved name: quay.io/bandrade/planetscale:0.1.8 INFO[0003] fetched digest="sha256:40ae61e2939f860fbf10a7001b81e08521a29f9ec8d9a73f866f1402b134c0cb" INFO[0004] fetched digest="sha256:83623a05adb1bf7271d5dcdbd19023f884a4720bdc9152a078d7a9534fd8baa9" INFO[0004] fetched digest="sha256:13c596886113a9a00ed8873ecda1bfd99c885a6a7a95f2eb8ed484721a418c0d" INFO[0004] fetched digest="sha256:d60f3b69d7a1e8d5830d4235ae101dacd4529a77b16c3aa4da6c9e7f91f12fe2" INFO[0005] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:d60f3b69d7a1e8d5830d4235ae101dacd4529a77b16c3aa4da6c9e7f91f12fe2 66159 [] map[] <nil>} INFO[0005] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:83623a05adb1bf7271d5dcdbd19023f884a4720bdc9152a078d7a9534fd8baa9 284 [] map[] <nil>} INFO[0005] Could not find optional dependencies file dir=bundle_tmp629791097 file=bundle_tmp629791097/metadata load=annotations INFO[0005] found csv, loading bundle dir=bundle_tmp629791097 file=bundle_tmp629791097/manifests load=bundle INFO[0005] loading bundle file dir=bundle_tmp629791097/manifests file=lib-bucket-provisioner.v1.0.0.clusterserviceversion.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp629791097/manifests file=objectbucketclaims.objectbucket.io.crd.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp629791097/manifests file=objectbuckets.objectbucket.io.crd.yaml load=bundle INFO[0005] Could not find optional dependencies file dir=bundle_tmp515466884 file=bundle_tmp515466884/metadata load=annotations INFO[0005] found csv, loading bundle dir=bundle_tmp515466884 file=bundle_tmp515466884/manifests load=bundle INFO[0005] loading bundle file dir=bundle_tmp515466884/manifests file=planetscale-operator.v0.1.8.clusterserviceversion.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp515466884/manifests file=psclusters.planetscale.com.crd.yaml load=bundle WARN[0005] permissive mode enabled bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" error="Invalid bundle planetscale-operator.v0.1.8, bundle specifies a non-existent replacement planetscale-operator.v0.1.7" INFO[0005] Generating dockerfile bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0005] writing dockerfile: index.Dockerfile001997167 bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0005] running podman build bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0005] [podman build --format docker -f index.Dockerfile001997167 -t quay.io/bandrade/multi-index:3.0 .] bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" While trying to serve the registry locally, I can see it without any packages podman create quay.io/bandrade/multi-index:3.0 57608f68b2c99beb4b2080b7f30e544b375f7bd7ecb2b6c6c31c9580c20c02aa podman start 57608f68b2c9 57608f68b2c99beb4b2080b7f30e544b375f7bd7ecb2b6c6c31c9580c20c02aa podman cp 57608f68b2c9:/database/index.db . opm registry serve -d index.db & time="2020-08-11T18:01:26Z" level=info msg="Keeping server open for infinite seconds" database=index.db port=50051 time="2020-08-11T18:01:26Z" level=info msg="serving registry" database=index.db port=50051 ./grpcurl -plaintext localhost:50051 api.Registry/ListPackage Version-Release number of selected component (if applicable): opm version Version: version.Version{OpmVersion:"v1.13.7", GitCommit:"018a040", BuildDate:"2020-08-11T14:23:30Z", GoOs:"linux", GoArch:"amd64"} podman version 1.6.4 How reproducible: Always Steps to Reproduce: 1.Use the above steps Actual results: Not able to add bundles to an index using permissive mode Expected results: Should be able to add bundles to an index using permissive mode
Hi Bruno, The `permissive` flag was added to handle certain edge case database conflicts, and turns errors into warnings to prevent the index from not being built. It does not force add an invalid bundle to the index, and what you're seeing is the expected behavior.
Hi Anik, As you said it turns errors into warnings and the index is built, but the problem is that no bundle is added into the index, even those that does not have any problems. I already used the permissive flag to load an operator registry and I remember that at least the bundles that does not have issues were loaded. Please let me know if that is expected as well. Thanks
On second look this may need a little bit of triaging. Keeping this open.
This needs to be looked at but shouldn't block 4.6, moving to 4.7
This issue has been fixed when the following PRs were merged as part of https://bugzilla.redhat.com/show_bug.cgi?id=1885425 https://github.com/operator-framework/operator-registry/pull/523 https://github.com/operator-framework/operator-registry/pull/503 Please verify. Additional info: It will be fixed in 4.6.z when https://bugzilla.redhat.com/show_bug.cgi?id=1898746 is fixed, which is the 4.6.z backport of https://bugzilla.redhat.com/show_bug.cgi?id=1885425
Hi Anik, I'm using the latest version of opm built from source. In order to reproduce this issue, I'm using an index which has a package in the index as shown below. opm version Version: version.Version{OpmVersion:"v1.15.2-4-gbeace541", GitCommit:"beace541", BuildDate:"2020-11-23T11:27:53Z", GoOs:"linux", GoArch:"amd64"} podman create quay.io/olmqe/etcd-prometheus-dependency-index:9.0 79243291fcf01d947c2435d28bcd6ca1591459595775ff1719a1dfc07cc646a6 podman start 79243291fcf01d947c2435d28bcd6ca1591459595775ff1719a1dfc07cc646a6 79243291fcf01d947c2435d28bcd6ca1591459595775ff1719a1dfc07cc646a6 podman cp 79243291fcf01d947c2435d28bcd6ca1591459595775ff1719a1dfc07cc646a6:/database/index.db . sqlite3 index.db SQLite version 3.33.0 2020-08-14 13:23:32 Enter ".help" for usage hints. sqlite> select * from operatorbundle; etcdoperator.v0.9.4|{"apiVersion":"operators.coreos.com/ v1alpha1","kind":"ClusterServiceVersion","metadata":{"annotations":{"alm-examples":"[\n {\n \"apiV.... Considering that, I'm trying to add a broken operatorbundle, it was expected that the other operatorbundle is preserved, but it wasn't as shown below. opm index add --bundles quay.io/bandrade/lib-bucket-provisioner:1.0,quay.io/bandrade/planetscale:0.1.8 --tag quay.io/olmqe/etcd-prometheus-dependency-index:9.0 --permissive INFO[0000] building the index bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0000] resolved name: quay.io/bandrade/lib-bucket-provisioner:1.0 INFO[0000] fetched digest="sha256:d4c4c9a1372197994a025fd1c5b74fb5cc2e8a34f66c98db8c63e5d98015d76d" INFO[0001] fetched digest="sha256:bda50c7d2a2ade1e8af3176a5dc4bc3d4fbcc8978b95eed61ae631468b3962d9" INFO[0001] fetched digest="sha256:91214fa0b23b1fa34771f1743a1e4eced407f008438cb30a321d20eebd6cb323" INFO[0001] fetched digest="sha256:46de1129490d66c106f62172383902862aa0dab202183f4e87a9743ddbcd7650" INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:46de1129490d66c106f62172383902862aa0dab202183f4e87a9743ddbcd7650 130481 [] map[] <nil>} INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:bda50c7d2a2ade1e8af3176a5dc4bc3d4fbcc8978b95eed61ae631468b3962d9 447 [] map[] <nil>} INFO[0003] resolved name: quay.io/bandrade/planetscale:0.1.8 INFO[0003] fetched digest="sha256:40ae61e2939f860fbf10a7001b81e08521a29f9ec8d9a73f866f1402b134c0cb" INFO[0003] fetched digest="sha256:83623a05adb1bf7271d5dcdbd19023f884a4720bdc9152a078d7a9534fd8baa9" INFO[0003] fetched digest="sha256:13c596886113a9a00ed8873ecda1bfd99c885a6a7a95f2eb8ed484721a418c0d" INFO[0003] fetched digest="sha256:d60f3b69d7a1e8d5830d4235ae101dacd4529a77b16c3aa4da6c9e7f91f12fe2" INFO[0004] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:d60f3b69d7a1e8d5830d4235ae101dacd4529a77b16c3aa4da6c9e7f91f12fe2 66159 [] map[] <nil>} INFO[0004] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:83623a05adb1bf7271d5dcdbd19023f884a4720bdc9152a078d7a9534fd8baa9 284 [] map[] <nil>} INFO[0004] Could not find optional dependencies file dir=bundle_tmp813088815 file=bundle_tmp813088815/metadata load=annotations INFO[0004] found csv, loading bundle dir=bundle_tmp813088815 file=bundle_tmp813088815/manifests load=bundle INFO[0004] loading bundle file dir=bundle_tmp813088815/manifests file=planetscale-operator.v0.1.8.clusterserviceversion.yaml load=bundle INFO[0004] loading bundle file dir=bundle_tmp813088815/manifests file=psclusters.planetscale.com.crd.yaml load=bundle WARN[0004] permissive mode enabled bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" error="error converting YAML to JSON: yaml: line 5: found an indentation indicator equal to 0" INFO[0004] Generating dockerfile bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0004] writing dockerfile: index.Dockerfile077050542 bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0004] running podman build bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" INFO[0004] [podman build --format docker -f index.Dockerfile077050542 -t quay.io/olmqe/etcd-prometheus-dependency-index:9.0 .] bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" podman create quay.io/olmqe/etcd-prometheus-dependency-index:9.0 4e1ac6ad113aa39c3015782f87172e0cbf7204bb91a64fff3fdd6294080146f9 podman start 4e1ac6ad113aa39c3015782f87172e0cbf7204bb91a64fff3fdd6294080146f9 4e1ac6ad113aa39c3015782f87172e0cbf7204bb91a64fff3fdd6294080146f9 podman cp 4e1ac6ad113aa39c3015782f87172e0cbf7204bb91a64fff3fdd6294080146f9:/database/index.db . sqlite3 index.db SQLite version 3.33.0 2020-08-14 13:23:32 Enter ".help" for usage hints. sqlite> select * from operatorbundle; sqlite> Please let me know if you need more details. All images are public. Thanks
Hi Bruno, The `opm index add` command builds a new index with the bundles provided with `--bundles` from scratch. Considering that, since both of your bundles have errors: the planetscale CSV mentions a replaces that's non-existent the libbucket-provisioner csv has indentation error: from your log, WARN[0004] permissive mode enabled bundles="[quay.io/bandrade/lib-bucket-provisioner:1.0 quay.io/bandrade/planetscale:0.1.8]" error="error converting YAML to JSON: yaml: line 5: found an indentation indicator equal to 0" none of the bundles get added to the index, and an empty index is created. If you want to preserve the existing bundles, you can use the `--from-index` flag to include the bundles from an old index, eg: ``` $ opm index add --bundles quay.io/anik120/planetscale:v0.1.8,quay.io/anik120/lib-bucket-provisioner:v1.0.0 --from-index quay.io/olmqe/etcd-prometheus-dependency-index:9.0 --tag quay.io/anik120/index:fromanother --permissive --container-tool=docker INFO[0000] building the index bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0000] Pulling previous image quay.io/olmqe/etcd-prometheus-dependency-index:9.0 to get metadata bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0000] running /usr/bin/docker pull quay.io/olmqe/etcd-prometheus-dependency-index:9.0 bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0001] running /usr/bin/docker pull quay.io/olmqe/etcd-prometheus-dependency-index:9.0 bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] Getting label data from previous image bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] running docker inspect bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] running docker create bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] running docker cp bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] running docker rm bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0002] running /usr/bin/docker pull quay.io/anik120/planetscale:v0.1.8 bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0003] running docker create bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0004] running docker cp bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0004] running docker rm bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0004] running /usr/bin/docker pull quay.io/anik120/lib-bucket-provisioner:v1.0.0 bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] running docker create bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] running docker cp bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] running docker rm bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] Could not find optional dependencies file dir=bundle_tmp148268000 file=bundle_tmp148268000/metadata load=annotations INFO[0005] found csv, loading bundle dir=bundle_tmp148268000 file=bundle_tmp148268000/manifests load=bundle INFO[0005] loading bundle file dir=bundle_tmp148268000/manifests file=planetscale-operator.v0.1.8.clusterserviceversion.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp148268000/manifests file=psclusters.planetscale.com.crd.yaml load=bundle INFO[0005] Could not find optional dependencies file dir=bundle_tmp438876607 file=bundle_tmp438876607/metadata load=annotations INFO[0005] found csv, loading bundle dir=bundle_tmp438876607 file=bundle_tmp438876607/manifests load=bundle INFO[0005] loading bundle file dir=bundle_tmp438876607/manifests file=lib-bucket-provisioner.v1.0.0.clusterserviceversion.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp438876607/manifests file=objectbucketclaims.objectbucket.io.crd.yaml load=bundle INFO[0005] loading bundle file dir=bundle_tmp438876607/manifests file=objectbuckets.objectbucket.io.crd.yaml load=bundle WARN[0005] permissive mode enabled bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" error="Invalid bundle planetscale-operator.v0.1.8, replaces nonexistent bundle planetscale-operator.v0.1.7" INFO[0005] Generating dockerfile bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] writing dockerfile: index.Dockerfile003221502 bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] running docker build bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" INFO[0005] [docker build -f index.Dockerfile003221502 -t quay.io/anik120/index:fromanother .] bundles="[quay.io/anik120/planetscale:v0.1.8 quay.io/anik120/lib-bucket-provisioner:v1.0.0]" $ docker create quay.io/anik120/index:fromanother f6d0e91e15dae546b83f5999b9a6e5aa913c3cfb98b0c8ecf92fd426d18a537e $ docker start f6d0e91e15dae546b83f5999b9a6e5aa913c3cfb98b0c8ecf92fd426d18a537e f6d0e91e15dae546b83f5999b9a6e5aa913c3cfb98b0c8ecf92fd426d18a537e $ docker cp f6d0e91e15dae546b83f5999b9a6e5aa913c3cfb98b0c8ecf92fd426d18a537e:/database/index.db . $ opm registry serve -d index.db WARN[0000] unable to set termination log path error="open /dev/termination-log: permission denied" INFO[0000] Keeping server open for infinite seconds database=index.db port=50051 INFO[0000] serving registry database=index.db port=50051 $ grpcurl -plaintext localhost:50051 api.Registry/ListPackages { "name": "etcd-service-monitor" } { "name": "lib-bucket-provisioner" } ``` You can see here that the new index preserved the bundle from your index, i.e the etcd bundle, and added the bundle I mentioned, i.e lib-bucket-provisioner
Hi Anik, That makes sense, thanks for your explanation, marking as VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633