Bug 1729714 - Image registry operator panic on upgrade from 4.1.4 to a recent 4.2 release
Summary: Image registry operator panic on upgrade from 4.1.4 to a recent 4.2 release
Keywords:
Status: VERIFIED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Corey Daley
QA Contact: wewang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-13 21:13 UTC by Corey Daley
Modified: 2019-08-14 07:57 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Priority Status Summary Last Updated
Github openshift cluster-image-registry-operator pull 320 None None None 2019-07-14 00:43:38 UTC

Description Corey Daley 2019-07-13 21:13:00 UTC 
I0712 14:24:57.468264       1 main.go:20] Cluster Image Registry Operator Version: 4.0.0-237-g67ff989-dirty
I0712 14:24:57.468352       1 main.go:21] Go Version: go1.12.5
I0712 14:24:57.468360       1 main.go:22] Go OS/Arch: linux/amd64
I0712 14:24:57.471624       1 controller.go:474] waiting for informer caches to sync
I0712 14:24:59.075005       1 controller.go:483] started events processor
E0712 14:24:59.075246       1 controller.go:278] unable to sync: unable to sync storage configuration: secret "cluster-config-v1" not found, requeuing
E0712 14:24:59.085652       1 runtime.go:69] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:76
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:522
/usr/local/go/src/runtime/panic.go:82
/usr/local/go/src/runtime/signal_unix.go:390
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:68
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:116
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:253
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:272
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/resource/generator.go:122
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/resource/generator.go:179
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:115
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:176
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:276
/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:283
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153
/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88
/usr/local/go/src/runtime/asm_amd64.s:1337
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x153ff3c]

goroutine 306 [running]:
github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x105
panic(0x17b3f00, 0x2ee0af0)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/openshift/cluster-image-registry-operator/pkg/storage/s3.GetConfig(0xc00000bc20, 0xc000288500, 0xc0000c92b8, 0x1da9640, 0x17bd640)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:68 +0xac
github.com/openshift/cluster-image-registry-operator/pkg/storage/s3.(*driver).getS3Service(0xc0009162c0, 0x0, 0x0, 0x0)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:116 +0x5d
github.com/openshift/cluster-image-registry-operator/pkg/storage/s3.(*driver).bucketExists(0xc0009162c0, 0xc00098c8c0, 0x3e, 0x0, 0xc00074d270)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:253 +0x3d
github.com/openshift/cluster-image-registry-operator/pkg/storage/s3.(*driver).StorageExists(0xc0009162c0, 0xc0002cd340, 0xc000288500, 0x1d81de0, 0xc0009162c0)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/storage/s3/s3.go:272 +0x52
github.com/openshift/cluster-image-registry-operator/pkg/resource.(*Generator).syncStorage(0xc0002ca000, 0xc0002cd340, 0x203000, 0x203000)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/resource/generator.go:122 +0x10a
github.com/openshift/cluster-image-registry-operator/pkg/resource.(*Generator).Apply(0xc0002ca000, 0xc0002cd340, 0x0, 0xc93e03)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/resource/generator.go:179 +0x4d
github.com/openshift/cluster-image-registry-operator/pkg/operator.(*Controller).createOrUpdateResources(0xc00043a000, 0xc0002cd340, 0x7, 0xc0003ab001)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:115 +0x1a9
github.com/openshift/cluster-image-registry-operator/pkg/operator.(*Controller).sync(0xc00043a000, 0x1d90e70, 0xc0002ca0c0)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:176 +0x1560
github.com/openshift/cluster-image-registry-operator/pkg/operator.(*Controller).eventProcessor.func1(0xc00043a000, 0x1719900, 0x1cfca10)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:276 +0x98
github.com/openshift/cluster-image-registry-operator/pkg/operator.(*Controller).eventProcessor(0xc00043a000)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:283 +0x45
github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000bd41f0)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x54
github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000bd41f0, 0x3b9aca00, 0x0, 0x1, 0xc00059c360)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8
Comment 1 Corey Daley 2019-07-13 21:14:19 UTC 
Oleg, I am going to be out for a week, could you please take a look at this?
Comment 3 Corey Daley 2019-08-12 14:59:25 UTC 
I believe that this should be fixed now.
Comment 4 Wenjing Zheng 2019-08-14 07:57:00 UTC 
Upgrade from 4.1.4 to 4.2 (4.2.0-0.nightly-2019-08-13-183722) succeeds.
Note You need to log in before you can comment on or make changes to this bug.