Description of problem: When using OpenShift Pipeline DSL to interact with the OpenShift API server, special shell characters are not passed to `oc` but explained by Bash (on a Linux slave) or Command Prompt (on Windows slave). We hit this issue when using Pipeline DSL to manage the CI/CD workloads for our projects. For example, if an option passed to a selector contains whitespaces or special shell character (like `#`, `;`, `$`), those characters will be handled by shell so that the `oc` process will not receive the option. Furthermore, this also lead to a vulnerability: There are customers run pipeline jobs behind a self-service portal. Crackers can construct malicious strings to hack into the pipeline build and run arbitrary commands on the slave unless they are secured filtered. Version-Release number of selected component (if applicable): 3.11 How reproducible: I create a BuildConfig to demo this shell injection. Steps to Reproduce: 1. Create a new app of ruby-hello-world: oc new-app https://github.com/openshift/ruby-hello-world 2. Create this BuildConfig: kind: "BuildConfig" apiVersion: "v1" metadata: name: "bug-shell-injection" spec: strategy: jenkinsPipelineStrategy: jenkinsfile: |- def label = "mypod-${UUID.randomUUID().toString()}" podTemplate(cloud: 'openshift', label: label, containers: [containerTemplate(name: 'jnlp', image: 'docker.io/openshift/jenkins-slave-base-centos7:latest')], ) { node(label) { stage('Options with Special Characters') { openshift.withCluster() { try { // because arguments contain special bash characters, the following step will fail. openshift.set('env', 'dc/ruby-hello-world', 'VAR1=PR#123', 'VAR2=hello world') echo 'Environment variables updated.' // will never run } catch (Exception e) { echo "Error running openshift.set" } } } stage('Shell Injection') { openshift.withCluster() { // ping will be run. Using `exit 1` to make the step Failed try { openshift.selector('po', '|| ping google.com -c 5 > out.txt ; echo {} | tee --').object() } catch (Exception e) { echo "Error running openshift.selector().object()" } pingResult = readFile 'out.txt' echo pingResult } } } } 3. Start a new build: oc start-build bug-shell-injection Actual results: stage('Options with Special Characters'): Cannot be finished because special characters are not correctly passed to `oc`. stage('Shell Injection') is a malicious stage: It inject a command `ping google.com` to the pipeline build step. ping results can be found from the build log. Expected results: stage('Options with Special Characters') should print 'Environment variables updated.'. stage('Shell Injection') should not run the malicious command `ping google.com`. Additional info: A possible fix is to run `oc` command using the `Launcher` Jenkins API rather than DurableTask. I've create a PR to show this solution: https://github.com/openshift/jenkins-client-plugin/pull/176
The PR from Yuxiang Zhu has merged. Thanks !! For now, given the depth of the change, only targeting the v4.0 jenkins images. As the change gains traction upstream, based on results, we can revisit backporting to older releases.
PR https://github.com/openshift/jenkins/pull/697 is updating the centos image
Job https://buildvm.openshift.eng.bos.redhat.com:8443/job/devex/job/devex%252Fjenkins-plugins/66/ has been submitted to update the 4.0 plugin rpm with v1.0.17 of the client plugin. Moving to post ... I don't think 4.0 images are showing up in brew just yet. Between checking periodically and monitoring the various mailing lists, I'll move to QA once a RHEL image is ready to test. The centos image should be available in a few hours.
Forgot to move this out of post ... the client plugin in the 4 .0 images is well past 1.0.17 now moving to ON QA
Hi Gabe, Tested in client plugin v1.0.21, met another issue：default sa cannot get dc in the same project, not sure if authentication issue, or I need add steps for it. steps: 1. Create a new app of ruby-hello-world: $oc new-app https://github.com/openshift/ruby-hello-world 2. Create this BuildConfig:bug-shell-injection Updated openshift.set('env', 'dc/ruby-hello-world', 'VAR1=PR#123', 'VAR2=hello world') to openshift.set('env', 'dc/ruby-hello-world', 'VAR1=PR#123'), seems oc set env did not support first format. 4. Create jenkins instance $oc new-app jenkins-persistent 3. Start a new pipeline build: $oc start-build bug-shell-injection 4.Check Logs: https://url.corp.redhat.com/d6d6034
Hey Wen Wang, Sorry for the delay in getting to this. The thanksgiving holiday and 4.0 are my excuses. Unfortunately, my delay resulted in https://url.corp.redhat.com/d6d6034 no longer being available. That said, if the sa cannot get to the dc, that is a config / setup issue. A couple of points: 1) You said "default sa" ... the jenkins templates like jenkins-persistent create a "jenkins" service account that has the permissions needed Did you change something so the jenkins pod would not use the jenkins SA? 2) were you using a 4.0 openshift ? ... I only ask because we removed the auto provisioning in 4.0, and if you were using 4.0, creating the build config before calling new-app on jenkins-persistent is OK. If you tried on 3.11, I would tell you to new-app jenkins-persistent *BEFORE* creating the pipeline build config 3) could you provide the complete build config yaml for the bug-shell-injection build config ? ... at worst, I can try to reproduce myself if need be thanks
I didn't change it, yes I use 4.0 openshift(v4.0.0-0.81.0) here's log:https://url.corp.redhat.com/5600f40, seems role and rolebinding related issue, tested in 3.11, did not meet the problem.
I still get ` Could not getting paste data: Paste does not exist, has expired or has been deleted.` when I try to look at your log URL. But I'll try to reproduce on 4.0 myself and we can go from there.
OK I think I figured it out. "system:serviceaccount:jenkins2:default" is incorrect it should be using "system:serviceaccount:jenkins2:jenkins" when provisioning via our jenkins templates. For our maven and nodejs sample pod templates, we set up the serviceAccount to be the "jenkins" SA during startup. See https://github.com/openshift/jenkins/blob/master/2/contrib/jenkins/kube-slave-common.sh#L81-L158 But I know realized after you re-reading your description that you are defining your own pod template using the base slave image: podTemplate(cloud: 'openshift', label: label, containers: [containerTemplate(name: 'jnlp', image: 'docker.io/openshift/jenkins-slave-base-centos7:latest')], ) On your 3.11 cluster, either you are using a different pod template, or you have authorized the default SA in the namespace on that cluster to have access in that namespace, etc. To really understand, you should pull the pod template definition from the working 3.11 jenkins instance and compare with the pod template definition from the 4.0 jenkins instance. But that is secondary to moving forward with verifying this fix. You need to either switch to one of our sample pod templates, or you need to augment the pod template you are defining to fill in all the fields we fill in with https://github.com/openshift/jenkins/blob/master/2/contrib/jenkins/kube-slave-common.sh#L81-L158 Do one of those options and try again please.
Thanks so much Gabe, you always understood qe's confusion. I add a step: oc policy add-role-to-user edit system:serviceaccount:wen:default for my Comment5, and it works now, the bug verfied. also thanks Yuxiang Zhu here's log: Running on jenkins-slave-spsm8-vzq99 in /home/jenkins/workspace/wen/wen-bug-shell-injection [Pipeline] { [Pipeline] stage [Pipeline] { (Options with Special Characters) [Pipeline] echo [Pipeline] _OcContextInit [Pipeline] readFile [Pipeline] readFile [Pipeline] _OcAction [Pipeline] echo Environment variables updated. //expected result,hooray! [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Shell Injection) [Pipeline] _OcContextInit [Pipeline] readFile [Pipeline] readFile [Pipeline] _OcAction Error: unknown shorthand flag: 'c' in -c
glad it is sorted out Wen!!
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758