Description of problem: Using jenkins-2-rhel7 image to do jenkinspipeline build, multi builds triggered from openshift client and jenkins webconsole don't sync consistently. Some builds are not existing in jenkins webconsole, but exist in openshift client, and build status keep new. Version-Release number of selected component (if applicable): brew-pulp.../jenkins-2-rhel7 (0bee80fcce6b) openshift-sync-0.0.13 openshift v3.3.1.2 kubernetes v1.3.0+52492b4 etcd 2.3.0+git How reproducible: Always Steps to Reproduce: 1.Create pipeline bc and other resources $ oc new-app -f https://raw.githubusercontent.com/openshift/origin/master/examples/jenkins/pipeline/samplepipeline.json $ oc new-app jenkins-ephemeral 2.Start more than 10 builds one by one from openshift client 3.Check build status from openshift client and jenkins webconsole 4.Start more than 10 builds one by one from jenkins webconsole 5.Check build status from openshift client and jenkins webconsole Actual results: Step3,5:The build numbers are not consistent.Some builds are not existing in jenkins webconsole, but exist in openshift client, and build status keep new. Expected results: Builds should sync well between openshift client and jenkins webconsole Additional info: $oc get pod sample-pipeline-3 JenkinsPipeline Complete 41 minutes ago 23s sample-pipeline-4 JenkinsPipeline Complete 21 minutes ago 25s sample-pipeline-5 JenkinsPipeline Complete 7 minutes ago 19s sample-pipeline-6 JenkinsPipeline Complete 21 minutes ago 26s sample-pipeline-7 JenkinsPipeline Complete 20 minutes ago 42s sample-pipeline-8 JenkinsPipeline Complete 20 minutes ago 59s sample-pipeline-9 JenkinsPipeline New sample-pipeline-10 JenkinsPipeline Complete 20 minutes ago 1m4s sample-pipeline-11 JenkinsPipeline Complete 20 minutes ago 1m9s sample-pipeline-12 JenkinsPipeline Complete 20 minutes ago 1m14s sample-pipeline-13 JenkinsPipeline New sample-pipeline-14 JenkinsPipeline Complete 14 minutes ago 22s sample-pipeline-15 JenkinsPipeline New sample-pipeline-16 JenkinsPipeline Complete 14 minutes ago 39s sample-pipeline-17 JenkinsPipeline Complete 14 minutes ago 49s sample-pipeline-18 JenkinsPipeline Complete 14 minutes ago 56s sample-pipeline-19 JenkinsPipeline New sample-pipeline-20 JenkinsPipeline Complete 14 minutes ago 1m3s sample-pipeline-21 JenkinsPipeline Complete 14 minutes ago 1m11s sample-pipeline-22 JenkinsPipeline Complete 13 minutes ago 1m11s sample-pipeline-23 JenkinsPipeline New sample-pipeline-24 JenkinsPipeline Complete 13 minutes ago 1m28s sample-pipeline-25 JenkinsPipeline Complete 6 minutes ago 22s
This is either a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1390293 or could be a side effect of https://bugzilla.redhat.com/show_bug.cgi?id=1356037.
Yeah I'm confident this is a side effect of not supporting build run policies properly so is effectively a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1390293. However, to confirm that it is I'd like to leave this open separately until QA confirm that it is fixed once the next release goes out.
Moving to ON_QA to test with latest image.
Test with brew-pulp.../openshift3/jenkins-2-rhel7 d126876bf60a brew-pulp.../openshift3/jenkins-1-rhel7 dc04c4ed570b openshift-login 0.9 openshift-pipeline 1.0.33 openshift-sync 0.0.17 openshift v3.4.0.26+f7e109e kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0 Since the pipeline build strategy default runpolicy is serial now, jenkins pipeline build sync consistently between openshift client and jenkins webconsole. But when change the runPolicy to Parallel, this issue still can be reproduced.
@tdawson Released sync plugin 0.1.0 ready for packaging please (no dependency changes).
this is blocked by whatever is causing jenkins to not startup in this PR https://github.com/openshift/jenkins/pull/204 once that PR has a sync plugin that works and gets merged, we can have Troy package the new sync plugin and spin a new rhel image accordingly.
Should be ready for QE now. rpm: jenkins-plugin-openshift-sync-0.1.1-1.el7 Images: openshift3/jenkins-1-rhel7:1.651.2-40 openshift3/jenkins-2-rhel7:2.19-7
Test with brew-pulp.../openshift3/jenkins-1-rhel7 6e8c6a3c7a01 brew-pulp.../openshift3/jenkins-2-rhel7 ca15e7519b57 openshift-login 0.9 openshift-pipeline 1.0.36 openshift-sync 0.1.1 openshift v3.4.0.29+ca980ba kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0 Steps: 1.Edit pipeline buildConfig to use Parallel runPolicy 2.Start more than 10 builds one by one from openshift client 3.Check build status from openshift client and jenkins webconsole 4.Start more than 10 builds one by one from jenkins webconsole 5.Check build status from openshift client and jenkins webconsole Actual results: 3&5.Will create triple jenkins jobs, and trigger triple related builds (nodejs-mongodb-example-N) for one pipeline build
When you say triple Jenkins jobs you mean there are 3 Jenkins jobs for the one BC?
I mean there are 3 jenkins builds under sample-pipeline jenkins project for one pipeline build (sample-pipeline-1)
I've managed to repro this, although not consistently. Will work on a fix today.
Should be fixed in v0.1.4 of sync plugin. Assigning to @tdawson for packaging.
rpms built: jenkins-plugin-openshift-pipeline-1.0.37-1.el7 jenkins-plugin-openshift-sync-0.1.4-1.el7 puddles created and synced to mirrors images created and pushed to registry.ops: openshift3/jenkins-1-rhel7:1.651.2-41 openshift3/jenkins-2-rhel7:2.19-8
Thanks @tdawson!
Test with brew-pulp.../openshift3/jenkins-1-rhel7 bdda1c3ed6fd brew-pulp.../openshift3/jenkins-2-rhel7 40ed5bbd7488 openshift-login 0.9 openshift-pipeline 1.0.37 openshift-sync 0.1.5 openshift v3.4.0.30+e10cc28 kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0