Bug 2113015 - Confusion around disable-home-env-overwrite and disable-working-directory-overwrite [NEEDINFO]
Summary: Confusion around disable-home-env-overwrite and disable-working-directory-ove...
Keywords:
Status: NEW
Alias: None
Product: Red Hat OpenShift Pipelines
Classification: Red Hat
Component: pipelines
Version: 1.7
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Vincent Demeester
QA Contact: Ruchir Garg
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-08-01 17:45 UTC by Luiz Carvalho
Modified: 2023-07-21 23:14 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
lucarval: needinfo? (vdemeest)


Attachments (Terms of Use)

Description Luiz Carvalho 2022-08-01 17:45:43 UTC
Description of problem:
There's a Red Hat solution (https://access.redhat.com/solutions/6956010) providing a remediation to the error:

warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied

However, this error does not occur when using the "pipeline" ServiceAccount. It's not clear why that is the case, but it appears to be related to the SecurityContextConstraints associated with the ServiceAccount (and the created Pod).

There is no explanation in the solution, nor in the docs (https://docs.openshift.com/container-platform/4.10/cicd/pipelines/understanding-openshift-pipelines.html#about-tasks_understanding-openshift-pipelines), as to this subtlety. This can lead to confusion for Task authors and users, especially because the ServiceAccount used is a property of the TaskRun, not the Task.

Version-Release number of selected component (if applicable):

$ oc version
Client Version: 4.8.0-202201210133.p0.g88e7eba.assembly.stream-88e7eba
Server Version: 4.10.22
Kubernetes Version: v1.23.5+3afdacb

$ tkn version
Client version: 0.21.0
Pipeline version: v0.33.2
Triggers version: v0.19.0

How reproducible:
Always.

Steps to Reproduce:
1. Create a simple task and two taskruns from it. One using the "pipeline" ServiceAccount, the other using the "default" ServiceAccount:

oc create -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: reproducer
spec:
  steps:
  - name: reproducer
    image: registry.access.redhat.com/ubi8/ubi:latest
    script: |
      echo 'hello'
---
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: reproducer-default-sa
spec:
  serviceAccountName: default
  taskRef:
    kind: Task
    name: reproducer
---
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: reproducer-pipeline-sa
spec:
  serviceAccountName: pipeline
  taskRef:
    kind: Task
    name: reproducer
EOF

Actual results:
The TaskRun reproducer-pipeline-sa succeeds while the TaskRun reproducer-default-sa fails.

Expected results:
Goth TaskRuns should either fail or succeed.

Additional info:

The main issue here is confusion. If we can make both cases behave the same, that's great. But I think it's also acceptable to document this oddity.

Comment 1 Luiz Carvalho 2022-08-01 17:50:11 UTC
Also important to mention that adding the "pipelines-scc" SecurityContextConstraint to whichever ServiceAccount is used by the Pod succeeds. With the example from the description in mind, the following command makes both use cases succeed:

$ oc policy add-role-to-user pipelines-scc-clusterrole -z default

where "default" is the name of whichever ServiceAccount is being used by the TaskRun.

Comment 2 Vincent Demeester 2022-08-04 06:29:33 UTC
@lucarval note that this is a "warning" and not necessarily an error. It won't make the Task fail on its own – but the Task could fail later due to authentication because it didn't find the files.
That said, this mainly happens when the Task and/or the image doesn't set the `HOME` environment variable. In that case (no HOME set), the "creds-init" behavior (that tekton does when using auth with ServiceAccount), it will "still" try to write to `$HOME/.docker` (for example for docker), which becomes `/.docker` which is, usually, not writable by any user but root (or not even by root).

If you apply the `pipelines-scc` SecurityContextConstraint, you are basically applying something very close to anyuid. The anyuid SCC is gonna run Pods as root if not otherwise specified (in the Pod definition *or* with a `USER` in the image). This is the reason why applying it fixes the issue.

This is one of the reason why the "best practices" tendency on authentications and secrets with Tekton points toward being more explicit with secrets bound to workspace (see https://docs.openshift-pipelines.xyz/pipeline/auth.html#secrets-and-workspaces). It's explicit, you can choose exactly where you "mount" them in the containers so that your Task content can do its thing.

Comment 3 Luiz Carvalho 2022-08-04 14:11:37 UTC
@vdemeest, thank you for the clarification. It makes sense. It would be nice to include this information in the solution I linked earlier. Something like this:

> This behavior may not occur if the Task is executed with a ServiceAccount which uses a less restricted SecurityContextConstraint. For example, by default tasks are executed with the "pipeline" ServiceAccount which uses the "pipelines-scc"" SecurityContextConstraint. As such, the ServiceAccount has enough access to not trigger the warning.

I was made aware of this issue because one of the tasks I wrote did not work properly on a certain environment. If the text above was in the solution, it would've saved me a few hours of investigation to narrow down why I couldn't reproduce the issue initially. Wdyt?


Note You need to log in before you can comment on or make changes to this bug.