Bug 1883905

Summary: race condition in opm index add --overwrite-latest
Product: OpenShift Container Platform Reporter: lgallett
Component: OLMAssignee: lgallett
OLM sub component: OLM QA Contact: Jian Zhang <jiazha>
Status: CLOSED ERRATA Docs Contact:
Severity: low    
Priority: unspecified CC: krizza
Version: 4.6   
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-24 15:21:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description lgallett 2020-09-30 13:44:25 UTC 
Description of problem:

using the golang race detector you can see concurrent writes are being made to the same map.

Version-Release number of selected component (if applicable):

latest

How reproducible:

always

Steps to Reproduce:
1. go build -race -mod=vendor -tags=json1 -o bin/opm ./cmd/opm
2. add some bundles in overwrite-latest mode to an index

Actual results:

golang complains

Expected results:

golang doesn't complain

Additional info:
Comment 3 Jian Zhang 2020-10-09 08:05:05 UTC 
1, Pull the latest master code
[root@preserve-olm-env operator-registry]# git log
commit c76735d6f644791867f0d5ba49587e0f9e5aeb55
Merge: b773a3c dc08166
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Thu Oct 8 13:44:18 2020 -0400

    Merge pull request #464 from gallettilance/fix-race-condition
    
    Bug 1883905: (fix) concurrent writes during image pulls

commit b773a3c30ff7eed4a2d0822c3f85ffc55c3aad14
Merge: 27f8daf 868892a
...
[root@preserve-olm-env operator-registry]# git branch
  1.12.6
* master
...

2, Compile the opm binary.
[root@preserve-olm-env operator-registry]# go build -race -mod=vendor -tags=json1 -o bin/opm ./cmd/opm
[root@preserve-olm-env operator-registry]#
[root@preserve-olm-env operator-registry]# ./bin/opm  version
Version: version.Version{OpmVersion:"unknown", GitCommit:"", BuildDate:"", GoOs:"linux", GoArch:"amd64"}

3, No the Data Race errors below, LGTM, verify it.
WARNING: DATA RACE
Read at 0x00c000a2b5f8 by main goroutine:

Details:
[root@preserve-olm-env operator-registry]# ./bin/opm index add --overwrite-latest -b quay.io/olmqe/etcd-bundle:0.9.2 -f quay.io/olmqe/etcd-index:0.9.2 
INFO[0000] building the index                            bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0000] Pulling previous image quay.io/olmqe/etcd-index:0.9.2 to get metadata  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0000] resolved name: quay.io/olmqe/etcd-index:0.9.2  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:ee23a1fd8a76e1ed95219577fe764c843ae932735181f26d7d75ae268c13526e"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:1d2d37b5593129000c6954da07b8e12af13280a6d7c0ee8499f37c57a333c1c2"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:03260e9c3f626c4f2822688eda12971a44cd0bedeb21c194b5e22cd478db1126"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:df20fa9351a15782c64e6dddb2d4a6f50bf6d3688060a34c4014b0d9a752eb4c"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:1d224440001736408010810e42f9c8c67bdcc1fd4b948776a760afe4de6c9f47"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:91add1d93084bc4dbb357a84948dcd8a81671a72b84a514c06082cea5125deda"
INFO[0000] fetched                                       bundles="[quay.io/olmqe/etcd-bundle:0.9.2]" digest="sha256:d5ffc5a98b7ec75898a51010c39c6ea3dba5115f6d9776469b3623d36e703dfc"
WARN[0001] {"architecture":"amd64","config":{"Hostname":"","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"50051/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["registry","serve","--database","/database/index.db"],"Image":"sha256:e60b760c38c62c30c9378aa36b824d9b956e46a3a2edbc45228a370e126d5a6f","Volumes":null,"WorkingDir":"","Entrypoint":["/bin/opm"],"OnBuild":null,"Labels":{"operators.operatorframework.io.index.database.v1":"/database/index.db"}},"container":"c17af6eba6c90b6f676d5fcae80f94f758e9e561b3ee7ece4b2c9de11b3c1b5a","container_config":{"Hostname":"c17af6eba6c9","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"50051/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/bin/sh","-c","#(nop) ","CMD [\"registry\" \"serve\" \"--database\" \"/database/index.db\"]"],"Image":"sha256:e60b760c38c62c30c9378aa36b824d9b956e46a3a2edbc45228a370e126d5a6f","Volumes":null,"WorkingDir":"","Entrypoint":["/bin/opm"],"OnBuild":null,"Labels":{"operators.operatorframework.io.index.database.v1":"/database/index.db"}},"created":"2020-08-21T12:16:45.66342323Z","docker_version":"19.03.9","history":[{"created":"2020-05-29T21:19:46.192045972Z","created_by":"/bin/sh -c #(nop) ADD file:c92c248239f8c7b9b3c067650954815f391b7bcb09023f984972c082ace2a8d0 in / "},{"created":"2020-05-29T21:19:46.363518345Z","created_by":"/bin/sh -c #(nop)  CMD [\"/bin/sh\"]","empty_layer":true},{"created":"2020-06-02T23:42:55.241252016Z","created_by":"/bin/sh -c apk update \u0026\u0026 apk add ca-certificates"},{"created":"2020-08-07T23:42:42.899020602Z","created_by":"/bin/sh -c #(nop) COPY file:de6403f21b5a03b63d0172e94713c29c53713d42f9ed3646c3e36f8707f2dabf in /bin/opm "},{"created":"2020-08-07T23:42:43.142507531Z","created_by":"/bin/sh -c #(nop) COPY file:d152b88c7d1e4f825edb1173337c2994c80fce349331e1378947a054943f0c10 in /bin/grpc_health_probe "},{"created":"2020-08-14T07:03:03.527040626Z","created_by":"/bin/sh -c #(nop)  LABEL operators.operatorframework.io.index.database.v1=/database/index.db","empty_layer":true},{"created":"2020-08-21T12:16:45.08006747Z","created_by":"/bin/sh -c #(nop) ADD file:1452b17416d583f48f4a4846257955ba448c38a91d7fbfe5a1911d59c8c0eeb4 in /database/index.db "},{"created":"2020-08-21T12:16:45.2597659Z","created_by":"/bin/sh -c #(nop)  EXPOSE 50051","empty_layer":true},{"created":"2020-08-21T12:16:45.459340031Z","created_by":"/bin/sh -c #(nop)  ENTRYPOINT [\"/bin/opm\"]","empty_layer":true},{"created":"2020-08-21T12:16:45.66342323Z","created_by":"/bin/sh -c #(nop)  CMD [\"registry\" \"serve\" \"--database\" \"/database/index.db\"]","empty_layer":true}],"os":"linux","rootfs":{"type":"layers","diff_ids":["sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a","sha256:4150c4f2e6dfcb80618a9323b447f6feefcc3c44cdc811d910b0a04879a51e56","sha256:cd87e5dc2bef2fe93539009558feb207d9ecebd14881cdcdfb66122d35ec1b34","sha256:ba840fdc36a3c0109086e12aef7d7ba674ed776c3f7da12817b681a892bcb9b6","sha256:3c04c6b108e0d63935b0d517d5f95ae5e1aa0dab994e0e1eae84cda5779b6c1f"]}}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0001] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:df20fa9351a15782c64e6dddb2d4a6f50bf6d3688060a34c4014b0d9a752eb4c 2797541 [] map[] <nil>}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:03260e9c3f626c4f2822688eda12971a44cd0bedeb21c194b5e22cd478db1126 2058239 [] map[] <nil>}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0002] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:1d224440001736408010810e42f9c8c67bdcc1fd4b948776a760afe4de6c9f47 16043331 [] map[] <nil>}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0010] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:91add1d93084bc4dbb357a84948dcd8a81671a72b84a514c06082cea5125deda 3776661 [] map[] <nil>}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0012] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:d5ffc5a98b7ec75898a51010c39c6ea3dba5115f6d9776469b3623d36e703dfc 19960 [] map[] <nil>}  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0012] resolved name: quay.io/olmqe/etcd-bundle:0.9.2 
INFO[0012] fetched                                       digest="sha256:d2989cb046cc8fce69003851f17dae54d4f53091e3d15075247edb2cf6e7e6c3"
INFO[0012] fetched                                       digest="sha256:7a853460998c00e93e994ec887ce1b06bc2dc94b3233673a7fd03295ae3548c8"
INFO[0012] fetched                                       digest="sha256:952889cd26279c35b24474b0a976a33182bf4d0df8ac50172a1d39513cf15d89"
INFO[0012] fetched                                       digest="sha256:314fcb6eb4f3b5665b7543bd41edca6cff8b5547e5bc158b3835284906022fde"
INFO[0013] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:952889cd26279c35b24474b0a976a33182bf4d0df8ac50172a1d39513cf15d89 8281 [] map[] <nil>} 
INFO[0013] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:314fcb6eb4f3b5665b7543bd41edca6cff8b5547e5bc158b3835284906022fde 283 [] map[] <nil>} 
INFO[0013] Could not find optional dependencies file     dir=bundle_tmp016707069 file=bundle_tmp016707069/metadata load=annotations
INFO[0013] found csv, loading bundle                     dir=bundle_tmp016707069 file=bundle_tmp016707069/manifests load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdbackups.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdclusters.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdoperator.v0.9.2.clusterserviceversion.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdrestores.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] resolved name: quay.io/olmqe/etcd-bundle:0.9.4 
INFO[0013] fetched                                       digest="sha256:23443e20fcb8106ff61ce862775251f12f6d862a0aec07fbd8dc20ea008126ba"
INFO[0013] fetched                                       digest="sha256:5dbe61aafc8fcb18afd27bba1f518d97c0990278f1a9120a9ad28608cc04c0b7"
INFO[0013] fetched                                       digest="sha256:45a1de674a84d0cfe5630176600f53868b0e29e350648786430f11175f59d899"
INFO[0013] fetched                                       digest="sha256:ef7a6117e82ba792f0d8d7ff2ef6b0867f012dc07d8027a60ae1c0aa3142d9d3"
INFO[0013] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:ef7a6117e82ba792f0d8d7ff2ef6b0867f012dc07d8027a60ae1c0aa3142d9d3 8331 [] map[] <nil>} 
INFO[0013] unpacking layer: {application/vnd.docker.image.rootfs.diff.tar.gzip sha256:45a1de674a84d0cfe5630176600f53868b0e29e350648786430f11175f59d899 285 [] map[] <nil>} 
INFO[0013] Could not find optional dependencies file     dir=bundle_tmp016707069 file=bundle_tmp016707069/metadata load=annotations
INFO[0013] found csv, loading bundle                     dir=bundle_tmp016707069 file=bundle_tmp016707069/manifests load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdbackups.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdclusters.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdoperator.v0.9.2.clusterserviceversion.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp016707069/manifests file=etcdrestores.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] Could not find optional dependencies file     dir=bundle_tmp434689848 file=bundle_tmp434689848/metadata load=annotations
INFO[0013] found csv, loading bundle                     dir=bundle_tmp434689848 file=bundle_tmp434689848/manifests load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp434689848/manifests file=etcdbackups.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp434689848/manifests file=etcdclusters.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp434689848/manifests file=etcdoperator.v0.9.4.clusterserviceversion.yaml load=bundle
INFO[0013] loading bundle file                           dir=bundle_tmp434689848/manifests file=etcdrestores.etcd.database.coreos.com.crd.yaml load=bundle
INFO[0013] Generating dockerfile                         bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0013] writing dockerfile: index.Dockerfile450677843  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0013] running podman build                          bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
INFO[0013] [podman build --format docker -f index.Dockerfile450677843 -t operator-registry-index:latest .]  bundles="[quay.io/olmqe/etcd-bundle:0.9.2]"
Comment 6 errata-xmlrpc 2021-02-24 15:21:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633