Bug 1782301
| Summary: | Multus admission controller not detecting syntax issues in pod and net-attach-def CRDs | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Ben Bennett <bbennett> | |
| Component: | Networking | Assignee: | Douglas Smith <dosmith> | |
| Networking sub component: | multus | QA Contact: | Weibin Liang <weliang> | |
| Status: | CLOSED WONTFIX | Docs Contact: | ||
| Severity: | high | |||
| Priority: | high | CC: | aconstan, anusaxen, bbennett, dosmith, scuppett, weliang, zzhao | |
| Version: | 4.3.0 | |||
| Target Milestone: | --- | |||
| Target Release: | 4.3.z | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | SDN-CI-IMPACT,SDN-BP,SDN-STALE | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1758702 | |||
| : | 1837638 (view as bug list) | Environment: | ||
| Last Closed: | 2020-07-01 13:05:50 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1837638 | |||
| Bug Blocks: | ||||
|
Comment 1
Anurag saxena
2019-12-11 14:50:08 UTC
I'm currently able to replicate the issue. $ cat def.yaml apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: macvlan-bridge spec: config: 'asdf' $ oc create -f def.yaml networkattachmentdefinition.k8s.cni.cncf.io/macvlan-bridge created $ oc get pods --all-namespaces | grep -i multus [...snip...] $ oc logs multus-admission-controller-5gkdd --namespace=openshift-multus I1212 17:32:27.482683 1 main.go:53] starting net-attach-def-admission-controller webhook server I1212 17:32:27.493962 1 localmetrics.go:50] UPdating net-attach-def metrics for any with value 0 I1212 17:32:27.494006 1 localmetrics.go:50] UPdating net-attach-def metrics for sriov with value 0 I1212 17:32:27.494854 1 controller.go:179] Starting net-attach-def-admission-controller I1212 17:32:27.595213 1 controller.go:188] net-attach-def-admission-controller synced and ready I1212 18:48:17.380940 1 webhook.go:111] validating network config spec: asdf 2019/12/12 18:48:17 http: panic serving 10.129.0.1:57938: assignment to entry in nil map goroutine 1118 [running]: net/http.(*conn).serve.func1(0xc4201e4960) /usr/local/go/src/net/http/server.go:1726 +0xd0 panic(0x10ea600, 0x1337eb0) /usr/local/go/src/runtime/panic.go:502 +0x229 github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.preprocessCNIConfig(0xc4206c2c50, 0xe, 0xc4206c2e38, 0x4, 0x8, 0x8, 0x0, 0x69ec80, 0xc420887888, 0x121cb00) /go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:90 +0x1be github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.validateNetworkAttachmentDefinition(0xc420723ba0, 0x1b, 0xc420723b80, 0x12, 0xc4206c2c50, 0xe, 0x0, 0x0, 0xc4206c2c09, 0x7, ...) /go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:118 +0x24c github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.ValidateHandler(0x1352020, 0xc4201e22a0, 0xc420778b00) /go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:403 +0x112 net/http.HandlerFunc.ServeHTTP(0x12cb9d0, 0x1352020, 0xc4201e22a0, 0xc420778b00) /usr/local/go/src/net/http/server.go:1947 +0x44 net/http.(*ServeMux).ServeHTTP(0x1becdc0, 0x1352020, 0xc4201e22a0, 0xc420778b00) /usr/local/go/src/net/http/server.go:2340 +0x130 net/http.serverHandler.ServeHTTP(0xc42045c000, 0x1352020, 0xc4201e22a0, 0xc420778b00) /usr/local/go/src/net/http/server.go:2697 +0xbc net/http.(*conn).serve(0xc4201e4960, 0x1353060, 0xc420688e80) /usr/local/go/src/net/http/server.go:1830 +0x651 created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:2798 +0x27b Also, using release 4.3.0-0.ci-2019-12-05-183852 I'm still trying to isolate the cause, however, interestingly enough -- some errors are appropriately trapped, such as:
```
$ cat def2.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: macvlan-bridge-2
spec:
config: '{}'
$ oc create -f def2.yaml
Error from server: error when creating "def2.yaml": admission webhook "multus-validating-config.k8s.io" denied the request: invalid config: missing 'type' in cni config
$ oc logs multus-admission-controller-5gkdd --namespace=openshift-multus
I1212 17:32:27.482683 1 main.go:53] starting net-attach-def-admission-controller webhook server
I1212 17:32:27.493962 1 localmetrics.go:50] UPdating net-attach-def metrics for any with value 0
I1212 17:32:27.494006 1 localmetrics.go:50] UPdating net-attach-def metrics for sriov with value 0
I1212 17:32:27.494854 1 controller.go:179] Starting net-attach-def-admission-controller
I1212 17:32:27.595213 1 controller.go:188] net-attach-def-admission-controller synced and ready
I1212 18:48:17.380940 1 webhook.go:111] validating network config spec: asdf
2019/12/12 18:48:17 http: panic serving 10.129.0.1:57938: assignment to entry in nil map
goroutine 1118 [running]:
net/http.(*conn).serve.func1(0xc4201e4960)
/usr/local/go/src/net/http/server.go:1726 +0xd0
panic(0x10ea600, 0x1337eb0)
/usr/local/go/src/runtime/panic.go:502 +0x229
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.preprocessCNIConfig(0xc4206c2c50, 0xe, 0xc4206c2e38, 0x4, 0x8, 0x8, 0x0, 0x69ec80, 0xc420887888, 0x121cb00)
/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:90 +0x1be
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.validateNetworkAttachmentDefinition(0xc420723ba0, 0x1b, 0xc420723b80, 0x12, 0xc4206c2c50, 0xe, 0x0, 0x0, 0xc4206c2c09, 0x7, ...)
/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:118 +0x24c
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook.ValidateHandler(0x1352020, 0xc4201e22a0, 0xc420778b00)
/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/webhook/webhook.go:403 +0x112
net/http.HandlerFunc.ServeHTTP(0x12cb9d0, 0x1352020, 0xc4201e22a0, 0xc420778b00)
/usr/local/go/src/net/http/server.go:1947 +0x44
net/http.(*ServeMux).ServeHTTP(0x1becdc0, 0x1352020, 0xc4201e22a0, 0xc420778b00)
/usr/local/go/src/net/http/server.go:2340 +0x130
net/http.serverHandler.ServeHTTP(0xc42045c000, 0x1352020, 0xc4201e22a0, 0xc420778b00)
/usr/local/go/src/net/http/server.go:2697 +0xbc
net/http.(*conn).serve(0xc4201e4960, 0x1353060, 0xc420688e80)
/usr/local/go/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2798 +0x27b
I1212 19:00:00.162334 1 webhook.go:111] validating network config spec: {}
```
(note that the log for processing the value of '{}' comes after the previous error)
Looks like we needed to validate that the configuration string was in JSON format before continuing with the rest of the validation. Upstream pull request @ https://github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pull/37 Merged upstream, downstream PRs: Into master @ https://github.com/openshift/multus-admission-controller/pull/21 For backport to 4.3 @ https://github.com/openshift/multus-admission-controller/pull/22 Hi Has this been merged? In such a case: can it be closed? -Alex This has a tangle of BZ dependencies for the BZ bot. The PR is still pending (https://github.com/openshift/multus-admission-controller/pull/22), and I need to try to detangle the BZ deps |