Created attachment 1760394[details]
sync log
Description of problem:
Slow start of jobs in Jenkins after starting the build in OpenShift.
The Global Concurrency Limit for Kubernetes is 100 on Jenkins Configure Clouds. There's no concurrency limitation on jenkins slave containers.
The slow is detected in some projects and not in others. Some projects builds always stay in "No stage have started." state for a long time while other projects are quickly triggered on Jenkins witout any issue.
Version-Release number of selected component (if applicable):
- OpenShift 4.6
- OpenShift Sync 1.0.45
How reproducible:
Always
Attaching an excerpt of the log (see full log attached):
2021-02-25 16:32:00 SEVERE hudson.triggers.SafeTimerTask run Timer task io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask@7d18fb7d failed
java.lang.IllegalArgumentException
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:623)
at jenkins.util.InterceptingScheduledExecutorService.scheduleAtFixedRate(InterceptingScheduledExecutorService.java:59)
at io.fabric8.jenkins.openshiftsync.BaseWatcher.start(BaseWatcher.java:63)
at io.fabric8.jenkins.openshiftsync.ConfigMapWatcher.start(ConfigMapWatcher.java:93)
at io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask.intializeAndStartWatchers(GlobalPluginConfigurationTimerTask.java:55)
at io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask.doRun(GlobalPluginConfigurationTimerTask.java:40)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:91)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Marking this one as a duplicated of https://bugzilla.redhat.com/show_bug.cgi?id=1925524.
This is getting fixed with the release of 1.0.47 of Jenkins plugin and subsequent versions which better manage scheduling of Tasks especially during high loads.
With OCP 4.5 and higher, this bug is likely to happen more frequently because of the introduction and enablement of the Priority and Fairness feature gate on OCP which limits and throttles the number of connections Jenkins can have to the API Server.
*** This bug has been marked as a duplicate of bug 1925524 ***
Created attachment 1760394 [details] sync log Description of problem: Slow start of jobs in Jenkins after starting the build in OpenShift. The Global Concurrency Limit for Kubernetes is 100 on Jenkins Configure Clouds. There's no concurrency limitation on jenkins slave containers. The slow is detected in some projects and not in others. Some projects builds always stay in "No stage have started." state for a long time while other projects are quickly triggered on Jenkins witout any issue. Version-Release number of selected component (if applicable): - OpenShift 4.6 - OpenShift Sync 1.0.45 How reproducible: Always Attaching an excerpt of the log (see full log attached): 2021-02-25 16:32:00 SEVERE hudson.triggers.SafeTimerTask run Timer task io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask@7d18fb7d failed java.lang.IllegalArgumentException at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:623) at jenkins.util.InterceptingScheduledExecutorService.scheduleAtFixedRate(InterceptingScheduledExecutorService.java:59) at io.fabric8.jenkins.openshiftsync.BaseWatcher.start(BaseWatcher.java:63) at io.fabric8.jenkins.openshiftsync.ConfigMapWatcher.start(ConfigMapWatcher.java:93) at io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask.intializeAndStartWatchers(GlobalPluginConfigurationTimerTask.java:55) at io.fabric8.jenkins.openshiftsync.GlobalPluginConfigurationTimerTask.doRun(GlobalPluginConfigurationTimerTask.java:40) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:91) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)