-------------------------------------------------- Description of problem: -------------------------------------------------- as a part of the new KubeVirt rate limiting testing, I tried to increase the number of virt-api pods using this which had no issues before, I suspect the new auto virt-api auto-scaler overwrites the manual patch. ================================================================================================ apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: annotations: deployOVS: "false" hco.kubevirt.io/tuningPolicy: '{"qps":200,"burst":400}' kubevirt.kubevirt.io/jsonpatch: '[{"op": "add", "path": "/spec/customizeComponents/patches", "value": [{"resourceType": "Deployment", "resourceName": "virt-api", "type": "json", "patch": "[{\"op\": \"replace\", \"path\": \"/spec/replicas\", \"value\": 6}]"}]}]' ================================================================================================= oc get deployment virt-api -o yaml readyReplicas: 2 replicas: 2 updatedReplicas: 2 ================================================================================================= oc -n openshift-cnv get kubevirt kubevirt -o yaml customizeComponents: patches: - patch: '[{"op": "replace", "path": "/spec/replicas", "value": 6}]' resourceName: virt-api resourceType: Deployment type: json ================================================================================================= Boaz Ben Shabat, 10:29 AM [root@e29-h19-740xd storms_tool]# oc get pod -n openshift-cnv|grep virt-api virt-api-8795754c5-npd5m 1/1 Running 1 30h virt-api-8795754c5-v8tn5 1/1 Running 1 30h ================================================================================================= -------------------------------------------------- Version-Release number of selected component (if applicable): -------------------------------------------------- openshift-cnv 4.13.0-1689 openshift-local-storage 4.12.0-202302061702 openshift-storage 4.12.0 -------------------------------------------------- How reproducible: -------------------------------------------------- every time. -------------------------------------------------- Steps to Reproduce: -------------------------------------------------- apply the following patch: oc patch hco -n openshift-cnv kubevirt-hyperconverged --type=merge -p '{"metadata":{"annotations":{"kubevirt.kubevirt.io/jsonpatch":"[{\"op\": \"add\", \"path\": \"/spec/customizeComponents/patches\", \"value\": [{\"resourceType\": \"Deployment\", \"resourceName\": \"virt-api\", \"type\": \"json\", \"patch\": \"[{\\\"op\\\": \\\"replace\\\", \\\"path\\\": \\\"/spec/replicas\\\", \\\"value\\\": 6}]\"}]}]"}}}' -------------------------------------------------- Expected results: -------------------------------------------------- the number of virt-api pods should increase to 6
Taking the severity of the bug and capacity of the team we plan to target this to CNV 4.14
oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged \ kubevirt.kubevirt.io/jsonpatch='[ { "op": "add", "path": "/spec/customizeComponents/patches", "value": [{ "patch": "[{\"op\":\"replace\",\"path\":\"/spec/replicas\",\"value\":6}]", "resourceName": "virt-api", "resourceType": "Deployment", "type": "json" }] } ]'
HCO JSON Patch might be working in few situations and might not be working in some situations. 1) The working examples a) oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged \ kubevirt.kubevirt.io/jsonpatch='[ { "op": "add", "path": "/spec/customizeComponents/patches", "value": [{ "patch": "[{\"op\":\"add\",\"path\":\"/spec/template/spec/containers/0/resources/requests\",\"value\":{\"cpu\": \"10m\",\"memory\": \"400Mi\"}}]", "resourceName": "virt-controller", "resourceType": "Deployment", "type": "json" }] } ]' b) oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[{ "op": "add", "path": "/spec/configuration/developerConfiguration/cpuAllocationRatio", "value": 20 }]' 2) The not working examples a) oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged \ kubevirt.kubevirt.io/jsonpatch='[ { "op": "add", "path": "/spec/configuration/additionalGuestMemoryOverheadRatio", "value": 1.5 } ]' b) oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged \ kubevirt.kubevirt.io/jsonpatch='[ { "op": "add", "path": "/spec/customizeComponents/patches", "value": [{ "patch": "[{\"op\":\"replace\",\"path\":\"/spec/replicas\",\"value\":6}]", "resourceName": "virt-api", "resourceType": "Deployment", "type": "json" }] } ]'
(In reply to Kedar Bidarkar from comment #4) > HCO JSON Patch might be working in few situations and might not be working > in some situations. > > > 1) The working examples > a) > > oc annotate --overwrite -n openshift-cnv hyperconverged > kubevirt-hyperconverged \ > kubevirt.kubevirt.io/jsonpatch='[ > { > "op": "add", > "path": "/spec/customizeComponents/patches", > "value": [{ > "patch": > "[{\"op\":\"add\",\"path\":\"/spec/template/spec/containers/0/resources/ > requests\",\"value\":{\"cpu\": \"10m\",\"memory\": \"400Mi\"}}]", > "resourceName": "virt-controller", > "resourceType": "Deployment", > "type": "json" > }] > } > ]' > > b) oc annotate --overwrite -n openshift-cnv hyperconverged > kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[{ > "op": "add", > "path": > "/spec/configuration/developerConfiguration/cpuAllocationRatio", > "value": 20 > }]' > > 2) The not working examples > > a) oc annotate --overwrite -n openshift-cnv hyperconverged > kubevirt-hyperconverged \ > kubevirt.kubevirt.io/jsonpatch='[ > { > "op": "add", > "path": "/spec/configuration/additionalGuestMemoryOverheadRatio", > "value": 1.5 > } > ]' > > b) oc annotate --overwrite -n openshift-cnv hyperconverged > kubevirt-hyperconverged \ > kubevirt.kubevirt.io/jsonpatch='[ > { > "op": "add", > "path": "/spec/customizeComponents/patches", > "value": [{ > "patch": > "[{\"op\":\"replace\",\"path\":\"/spec/replicas\",\"value\":6}]", > "resourceName": "virt-api", > "resourceType": "Deployment", > "type": "json" > }] > } > ]' Indeed there is a bug with the calculation of virt-api replicas probably by the following Change: https://github.com/kubevirt/kubevirt/pull/7401 The following PR should resolve it: https://github.com/kubevirt/kubevirt/pull/10027 as for > a) oc annotate --overwrite -n openshift-cnv hyperconverged > kubevirt-hyperconverged \ > kubevirt.kubevirt.io/jsonpatch='[ > { > "op": "add", > "path": "/spec/configuration/additionalGuestMemoryOverheadRatio", > "value": 1.5 > } > ]' Please try using a string val it worked for me and i can see the kv has been modified. my command: oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[{ "op": "add", "path": "/spec/configuration/additionalGuestMemoryOverheadRatio", "value": "1.5" }]'
https://bugzilla.redhat.com/show_bug.cgi?id=2222190 https://bugzilla.redhat.com/show_bug.cgi?id=2222191 created BZ for 4.12 and 4.11 and backported
test with build: CNV-v4.13.3.rhel9-160 step : 1. apply patch: oc patch hco -n openshift-cnv kubevirt-hyperconverged --type=merge -p '{"metadata":{"annotations":{"kubevirt.kubevirt.io/jsonpatch":"[{\"op\": \"add\", \"path\": \"/spec/customizeComponents/patches\", \"value\": [{\"resourceType\": \"Deployment\", \"resourceName\": \"virt-api\", \"type\": \"json\", \"patch\": \"[{\\\"op\\\": \\\"replace\\\", \\\"path\\\": \\\"/spec/replicas\\\", \\\"value\\\": 6}]\"}]}]"}}}' check: $oc get deployment virt-api -n openshift-cnv -o yaml replicas: 6 availableReplicas: 6 readyReplicas: 6 replicas: 6 updatedReplicas: 6 $oc get pod -n openshift-cnv|grep virt-api $ oc get pods -n openshift-cnv | grep virt-api virt-api-5689868576-djjwh 1/1 Running 0 4m55s virt-api-5689868576-hpqdl 1/1 Running 0 4m55s virt-api-5689868576-jlnz2 1/1 Running 0 4m55s virt-api-5689868576-mphkx 1/1 Running 0 4m55s virt-api-5689868576-tzl8w 1/1 Running 0 2d3h virt-api-5689868576-xfltr 1/1 Running 0 2d3h 2: apply patch: oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[{ "op": "add", "path": "/spec/configuration/additionalGuestMemoryOverheadRatio", "value": "1.5" }]' check: $ oc get kv -n openshift-cnv -o yaml ... additionalGuestMemoryOverheadRatio: "1.5" move to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Virtualization 4.13.3 Images security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:4664