Bug 2058679 - [4.8z] Pod density test causing problems when using kube-burner
Summary: [4.8z] Pod density test causing problems when using kube-burner
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.z
Assignee: Tim Rozet
QA Contact: Sai Sindhur Malleni
URL:
Whiteboard:
Depends On: 2058670
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-02-25 15:29 UTC by Tim Rozet
Modified: 2022-03-22 17:29 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2058670
Environment:
Last Closed: 2022-03-22 17:29:11 UTC
Target Upstream Version:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 979 0 None open Bug 2058679: Bumps ovn2.13-20.12.0-195.el8fdp 2022-03-01 16:55:08 UTC
Red Hat Product Errata RHBA-2022:0872 0 None None None 2022-03-22 17:29:31 UTC

Comment 5 Sai Sindhur Malleni 2022-03-16 15:58:15 UTC 
[smalleni@localhost kubelet-density]$ ./kube-burner init -c kubelet-density.yml 
INFO[2022-03-16 10:53:12] 🔥 Starting kube-burner (0.15.2@1a59017bd3e97f98c5fdcca80b906be31250b465) with UUID 0998a288-5515-4125-b747-c2ea2ed406fa 
true
INFO[2022-03-16 10:53:12] 📁 Creating indexer: elastic                  
INFO[2022-03-16 10:53:13] 📈 Creating measurement factory               
INFO[2022-03-16 10:53:13] Registered measurement: podLatency           
INFO[2022-03-16 10:53:13] Preparing create job: kubelet-density        
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Pod replicas 
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Service replicas 
INFO[2022-03-16 10:53:13] Triggering job: kubelet-density              
INFO[2022-03-16 10:53:13] Creating Pod latency watcher for kubelet-density 
INFO[2022-03-16 10:53:14] QPS: 50                                      
INFO[2022-03-16 10:53:14] Burst: 50                                    
INFO[2022-03-16 10:53:14] Running job kubelet-density                  
INFO[2022-03-16 10:56:07] Waiting 3h0m0s for actions to be completed   
INFO[2022-03-16 10:56:17] Actions in namespace kubelet-density completed 
INFO[2022-03-16 10:56:17] Finished the create job in 183.128181019 seconds 
INFO[2022-03-16 10:56:17] Verifying created objects                    
INFO[2022-03-16 10:56:19] pods found: 2000 Expected: 2000              
INFO[2022-03-16 10:56:20] services found: 2000 Expected: 2000          
INFO[2022-03-16 10:56:20] Stopping measurement: podLatency             
INFO[2022-03-16 10:56:20] Evaluating latency thresholds                
WARN[2022-03-16 10:56:20] Avg Ready latency (4625ms) higher than configured threshold: 4s 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency.json 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency-summary.json 
INFO[2022-03-16 10:56:20] Indexing [2000] documents in kube-burner     
INFO[2022-03-16 10:56:23] Indexing finished in 2.524s: created=2000    
INFO[2022-03-16 10:56:23] Indexing [4] documents in kube-burner        
INFO[2022-03-16 10:56:23] Indexing finished in 308ms: created=4        
INFO[2022-03-16 10:56:23] Job kubelet-density took 190.69 seconds      
INFO[2022-03-16 10:56:23] Writing to: collected-metrics/kubelet-density-metadata.json 
INFO[2022-03-16 10:56:23] Indexing metadata information for job: kubelet-density 
INFO[2022-03-16 10:56:23] Indexing [1] documents in ripsaw-kube-burner 
INFO[2022-03-16 10:56:24] Indexing finished in 83ms: created=1         
INFO[2022-03-16 10:56:24] Finished execution with UUID: 0998a288-5515-4125-b747-c2ea2ed406fa 
INFO[2022-03-16 10:56:24] 👋 Exiting kube-burner                        
[smalleni@localhost kubelet-density]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
[smalleni@localhost kubelet-density]$ ./kube-burner init -c kubelet-density.yml 
INFO[2022-03-16 10:53:12] 🔥 Starting kube-burner (0.15.2@1a59017bd3e97f98c5fdcca80b906be31250b465) with UUID 0998a288-5515-4125-b747-c2ea2ed406fa 
true
INFO[2022-03-16 10:53:12] 📁 Creating indexer: elastic                  
INFO[2022-03-16 10:53:13] 📈 Creating measurement factory               
INFO[2022-03-16 10:53:13] Registered measurement: podLatency           
INFO[2022-03-16 10:53:13] Preparing create job: kubelet-density        
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Pod replicas 
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Service replicas 
INFO[2022-03-16 10:53:13] Triggering job: kubelet-density              
INFO[2022-03-16 10:53:13] Creating Pod latency watcher for kubelet-density 
INFO[2022-03-16 10:53:14] QPS: 50                                      
INFO[2022-03-16 10:53:14] Burst: 50                                    
INFO[2022-03-16 10:53:14] Running job kubelet-density                  
INFO[2022-03-16 10:56:07] Waiting 3h0m0s for actions to be completed   
INFO[2022-03-16 10:56:17] Actions in namespace kubelet-density completed 
INFO[2022-03-16 10:56:17] Finished the create job in 183.128181019 seconds 
INFO[2022-03-16 10:56:17] Verifying created objects                    
INFO[2022-03-16 10:56:19] pods found: 2000 Expected: 2000              
INFO[2022-03-16 10:56:20] services found: 2000 Expected: 2000          
INFO[2022-03-16 10:56:20] Stopping measurement: podLatency             
INFO[2022-03-16 10:56:20] Evaluating latency thresholds                
WARN[2022-03-16 10:56:20] Avg Ready latency (4625ms) higher than configured threshold: 4s 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency.json 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency-summary.json 
INFO[2022-03-16 10:56:20] Indexing [2000] documents in kube-burner     
INFO[2022-03-16 10:56:23] Indexing finished in 2.524s: created=2000    
INFO[2022-03-16 10:56:23] Indexing [4] documents in kube-burner        
INFO[2022-03-16 10:56:23] Indexing finished in 308ms: created=4        
INFO[2022-03-16 10:56:23] Job kubelet-density took 190.69 seconds      
INFO[2022-03-16 10:56:23] Writing to: collected-metrics/kubelet-density-metadata.json 
INFO[2022-03-16 10:56:23] Indexing metadata information for job: kubelet-density 
INFO[2022-03-16 10:56:23] Indexing [1] documents in ripsaw-kube-burner 
INFO[2022-03-16 10:56:24] Indexing finished in 83ms: created=1         
INFO[2022-03-16 10:56:24] Finished execution with UUID: 0998a288-5515-4125-b747-c2ea2ed406fa 
INFO[2022-03-16 10:56:24] 👋 Exiting kube-burner                        
[smalleni@localhost kubelet-density]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
[smalleni@localhost kubelet-density]$ ./kube-burner init -c kubelet-density.yml 
INFO[2022-03-16 10:53:12] 🔥 Starting kube-burner (0.15.2@1a59017bd3e97f98c5fdcca80b906be31250b465) with UUID 0998a288-5515-4125-b747-c2ea2ed406fa 
true
INFO[2022-03-16 10:53:12] 📁 Creating indexer: elastic                  
INFO[2022-03-16 10:53:13] 📈 Creating measurement factory               
INFO[2022-03-16 10:53:13] Registered measurement: podLatency           
INFO[2022-03-16 10:53:13] Preparing create job: kubelet-density        
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Pod replicas 
INFO[2022-03-16 10:53:13] Job kubelet-density: 2000 iterations with 1 Service replicas 
INFO[2022-03-16 10:53:13] Triggering job: kubelet-density              
INFO[2022-03-16 10:53:13] Creating Pod latency watcher for kubelet-density 
INFO[2022-03-16 10:53:14] QPS: 50                                      
INFO[2022-03-16 10:53:14] Burst: 50                                    
INFO[2022-03-16 10:53:14] Running job kubelet-density                  
INFO[2022-03-16 10:56:07] Waiting 3h0m0s for actions to be completed   
INFO[2022-03-16 10:56:17] Actions in namespace kubelet-density completed 
INFO[2022-03-16 10:56:17] Finished the create job in 183.128181019 seconds 
INFO[2022-03-16 10:56:17] Verifying created objects                    
INFO[2022-03-16 10:56:19] pods found: 2000 Expected: 2000              
INFO[2022-03-16 10:56:20] services found: 2000 Expected: 2000          
INFO[2022-03-16 10:56:20] Stopping measurement: podLatency             
INFO[2022-03-16 10:56:20] Evaluating latency thresholds                
WARN[2022-03-16 10:56:20] Avg Ready latency (4625ms) higher than configured threshold: 4s 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency.json 
INFO[2022-03-16 10:56:20] Writing latency metrics in collected-metrics/kubelet-density-podLatency-summary.json 
INFO[2022-03-16 10:56:20] Indexing [2000] documents in kube-burner     
INFO[2022-03-16 10:56:23] Indexing finished in 2.524s: created=2000    
INFO[2022-03-16 10:56:23] Indexing [4] documents in kube-burner        
INFO[2022-03-16 10:56:23] Indexing finished in 308ms: created=4        
INFO[2022-03-16 10:56:23] Job kubelet-density took 190.69 seconds      
INFO[2022-03-16 10:56:23] Writing to: collected-metrics/kubelet-density-metadata.json 
INFO[2022-03-16 10:56:23] Indexing metadata information for job: kubelet-density 
INFO[2022-03-16 10:56:23] Indexing [1] documents in ripsaw-kube-burner 
INFO[2022-03-16 10:56:24] Indexing finished in 83ms: created=1         
INFO[2022-03-16 10:56:24] Finished execution with UUID: 0998a288-5515-4125-b747-c2ea2ed406fa 
INFO[2022-03-16 10:56:24] 👋 Exiting kube-burner                        
[smalleni@localhost kubelet-density]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2022-03-15-185859   True        False         51m     Cluster version is 4.8.0-0.nightly-2022-03-15-185859
Comment 7 errata-xmlrpc 2022-03-22 17:29:11 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.35 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0872
Note You need to log in before you can comment on or make changes to this bug.