Bug 1794182
| Summary: | WMCB panics due to empty verbosity argument | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Sebastian Soto <ssoto> |
| Component: | Windows Containers | Assignee: | Sebastian Soto <ssoto> |
| Status: | CLOSED ERRATA | QA Contact: | gaoshang <sgao> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 4.4 | CC: | aos-bugs, aravindh, gmarkley, rgudimet |
| Target Milestone: | --- | ||
| Target Release: | 4.4.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-05-04 11:26:35 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This bug has been verified and passed on 4.4.0-0.nightly-2020-02-02-225006 with AWS, move status to VERIFIED, thanks.
Version:
# oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.4.0-0.nightly-2020-02-02-225006 True False 130m Cluster version is 4.4.0-0.nightly-2020-02-02-225006
windows-machine-config-operator # git show
commit d0f8ddb70946f7707d65564fb87245f309d7ada2
Steps to Reproduce:
1. Install OCP with ovn kubenetes
2. Launch windows instance and run WSU
3. Check WSU run succeed and windows node added
# ansible-playbook -i hosts ~/go/src/windows-machine-config-operator/tools/ansible/tasks/wsu/main.yaml -v
...
PLAY RECAP **********************************************************************************************
18.224.165.72 : ok=34 changed=29 unreachable=0 failed=0 skipped=4 rescued=0 ignored=1
localhost : ok=10 changed=9 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
# oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-132-216.us-east-2.compute.internal Ready master 156m v1.17.1
ip-10-0-140-110.us-east-2.compute.internal Ready worker 149m v1.17.1
ip-10-0-146-221.us-east-2.compute.internal Ready worker 149m v1.17.1
ip-10-0-153-52.us-east-2.compute.internal Ready master 155m v1.17.1
ip-10-0-172-40.us-east-2.compute.internal Ready master 155m v1.17.1
ip-10-0-175-184.us-east-2.compute.internal Ready worker 148m v1.17.1
ip-10-0-31-99.us-east-2.compute.internal Ready worker 5m33s v1.16.2
4. Check kubelet service in windows node, arguments -v=3 and no error in c:\k\log\kubelet.log
PS C:\k\log> Get-Item -path HKLM:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\kubelet
Hive: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services
Name Property
---- --------
kubelet Type : 16
Start : 2
ErrorControl : 1
ImagePath : c:\k\kubelet.exe --windows-service
--node-labels=node.openshift.io/os_id=Windows
--cni-conf-dir=c:\k\cni\config
--bootstrap-kubeconfig=c:\k\bootstrap-kubeconfig
--kubeconfig=c:\k\kubeconfig --cert-dir=c:\var\lib\kubelet\pki\
--register-with-taints=os=Windows:NoSchedule
--config=c:\k\kubelet.conf --network-plugin=cni
--cni-bin-dir=c:\k\cni --log-file=c:\k\log\kubelet.log
--cloud-provider=aws --v=3 --resolv-conf=""
--pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.2.0
--logtostderr=false
ObjectName : LocalSystem
Description : OpenShift Kubelet
FailureActions : {88, 2, 0, 0...}
DisplayName : kubelet
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581 |
Description of problem: Log file created at: 2020/01/22 19:49:56 Running on machine: EC2AMAZ-B19V8TR Binary: Built with gc go1.12.10 for windows/amd64 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg F0122 19:49:56.930381 4676 server.go:154] invalid argument "" for "-v, --v" flag: strconv.Atoi: parsing "": invalid syntax goroutine 1 [running]: k8s.io/kubernetes/vendor/k8s.io/klog.stacks(0xc0000b6001, 0xc000582000, 0x93d, 0x2710) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/klog.go:883 +0xb8 k8s.io/kubernetes/vendor/k8s.io/klog.(*loggingT).output(0x75aa9c0, 0xc000000003, 0xc0005f4a10, 0x745ec6e, 0x9, 0x9a, 0x0) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/klog.go:837 +0x14d k8s.io/kubernetes/vendor/k8s.io/klog.(*loggingT).printDepth(0x75aa9c0, 0x3, 0x1, 0xc00073bd80, 0x1, 0x1) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/klog.go:706 +0x132 k8s.io/kubernetes/vendor/k8s.io/klog.(*loggingT).print(...) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/klog.go:697 k8s.io/kubernetes/vendor/k8s.io/klog.Fatal(...) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/klog.go:1264 k8s.io/kubernetes/cmd/kubelet/app.NewKubeletCommand.func1(0xc0005f1180, 0xc0000c6010, 0xc, 0xf) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:154 +0xd69 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc0005f1180, 0xc0000c6010, 0xc, 0xf, 0xc0005f1180, 0xc0000c6010) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:830 +0x2b5 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0005f1180, 0x42a0918, 0x75aa600, 0xc00073bf88) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:914 +0x303 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(...) /workspace/anago-v1.16.2-beta.0.19+c97fe5036ef3df/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:864 main.main() _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:41 +0xd0 Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Run WSU Actual results: wmcb panics Expected results: wmcb doesnt panic Additional info: Looking at the service arguments i see the issue: c:\k\kubelet.exe --config=c:\k\kubelet.conf --bootstrap-kubeconfig=c:\k\bootstrap-kubeconfig --kubeconfig=c:\k\kubeconfig --pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.2.0 --cert-dir=c:\var\lib\kubelet\pki\ --windows-service --logtostderr=false --log-file=c:\k\kubelet.log --register-with-taints=os=Windows:NoSchedule --node-labels=node.openshift.io/os_id=Windows --cloud-provider=aws --v= and heres the worker.ign: --v=\"${KUBELET_LOG_LEVEL}\"\n\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target\n","enabled":true,"name":"kubelet.service"},{"contents":"[Unit]\nBefore=pivot.service\nConditionFirstBoot=true\n[Service]\nExecStart=/bin/sh -c 'mkdir /run/pivot \u0026\u0026 touch /run/pivot/reboot-needed'\n[Install]\nWantedBy=multi-user.target\n","enabled":true,"name":"mcd-write-pivot-reboot.service"}]}} note: v=\"${KUBELET_LOG_LEVEL}