Description of problem: If you create an application and subscription within ACM and the secret for the git repository is not available it causes the subscription hub to go into a crash loop. This could cause cascading issues because it breaks app deployment and updating. Version-Release number of selected component (if applicable): 2.1 How reproducible: Deploy all application components (channel, app, placement, subscription) using a private repository and don't provide a secret. Steps to Reproduce: 1. create private repo 2. create acm application without secret 3. get logs of pod oc logs -f multicluster-operators-hub-subscription-769d89fb95-mfhv2 -n open-cluster-management Actual results: E0121 15:04:49.018617 1 gitrepo.go:198] Secret "github-gitops-clusters" not foundUnable to get secret from local cluster. E0121 15:04:49.018630 1 hub_git.go:323] subscription-hub-reconciler "msg"="failed to register subscription to git watcher register" "error"="Secret \"github-gitops-clusters\" not found" I0121 15:04:49.018648 1 mcmhub_controller.go:705] subscription-hub-reconciler "msg"="Enter finalCommit..." I0121 15:04:49.018684 1 mcmhub_controller.go:733] subscription-hub-reconciler "msg"="spec or metadata of scribe-system/scribe is updated" I0121 15:04:49.018703 1 mcmhub_controller.go:769] subscription-hub-reconciler "msg"="no post hooks, exit the reconcile." I0121 15:04:49.018722 1 mcmhub_controller.go:770] subscription-hub-reconciler "msg"="Eixt finalCommit..." I0121 15:04:49.018738 1 panic.go:679] subscription-hub-reconciler/scribe-system/scribe "msg"="exist Hub Reconciling subscription: scribe-system/scribe" E0121 15:04:49.018800 1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) goroutine 3621 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x2dcc520, 0x4a6bb70) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:74 +0xa3 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:48 +0x82 panic(0x2dcc520, 0x4a6bb70) /usr/lib/golang/src/runtime/panic.go:679 +0x1b2 github.com/open-cluster-management/multicloud-operators-subscription/pkg/controller/mcmhub.(*HubGitOps).GetLatestCommitID(0xc000aa9500, 0xc000315440, 0x3166707, 0x25, 0xc000fbb2d8, 0x1) /remote-source/app/pkg/controller/mcmhub/hub_git.go:457 +0x176 github.com/open-cluster-management/multicloud-operators-subscription/pkg/controller/mcmhub.(*ReconcileSubscription).UpdateGitDeployablesAnnotation(0xc0008a8000, 0xc000315440, 0xc0005791e0, 0x6, 0x0) /remote-source/app/pkg/controller/mcmhub/gitrepo_sync.go:75 +0x1f4 github.com/open-cluster-management/multicloud-operators-subscription/pkg/controller/mcmhub.(*ReconcileSubscription).doMCMHubReconcile(0xc0008a8000, 0xc000315440, 0x0, 0x0) /remote-source/app/pkg/controller/mcmhub/hub.go:72 +0x37c github.com/open-cluster-management/multicloud-operators-subscription/pkg/controller/mcmhub.(*ReconcileSubscription).Reconcile(0xc0008a8000, 0xc00116afd0, 0xd, 0xc00116afb4, 0x6, 0xc000a3e400, 0x0, 0x0, 0x0) /remote-source/app/pkg/controller/mcmhub/mcmhub_controller.go:565 +0xec0 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0006e0990, 0x2e80c20, 0xc0009d9460, 0x10e7c00) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:235 +0x27d sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0006e0990, 0x0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:209 +0xcb sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc0006e0990) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:188 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00125a2a0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155 +0x5e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00125a2a0, 0x35ad420, 0xc000a51770, 0x1, 0xc0000d4600) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00125a2a0, 0x3b9aca00, 0x0, 0x1, 0xc0000d4600) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133 +0xaa k8s.io/apimachinery/pkg/util/wait.Until(0xc00125a2a0, 0x3b9aca00, 0xc0000d4600) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:170 +0x431 panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x28ab226] Expected results: Application is just skipped and error logged to logs and potentially console Additional info:
Fix will be available in 2.1.3 to contain the accurate error message in subscription status.
You can wait for ACM2.1.3 to verify it. If you need the fix earlier, you this doc https://github.com/open-cluster-management/multicloud-operators-subscription/blob/master/docs/patching_subscription_image.md and use community-2.1 tag instead of community-latest.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat Advanced Cluster Management 2.1.3 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:0607