Description of problem: Due to 'Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/e502a13b9ce0e665bf77b912fb0a4d7fc46fd83ad8519858b7fc97f3bebceb7e: permission denied' error, can't create git server pod Works before in int env(older than openshift master:v3.2.0.40) Version-Release number of selected component (if applicable): dev-preview-stg (openshift master:v3.2.0.44) openshift/origin-gitserver (imageid:28042b7dc0fd) How reproducible: Steps to Reproduce: 1.Create the Git Server $oc create -f https://raw.githubusercontent.com/openshift/origin/master/examples/gitserver/gitserver.yaml 2.Grant edit access to the git service account $oc policy add-role-to-user openshift-online:edit -z git 3.Check pod # oc get pods NAME READY STATUS RESTARTS AGE git-1-deploy 0/1 Error 0 31m git-2-deploy 1/1 Running 0 <invalid> git-2-w5dfg 0/1 RunContainerError 0 <invalid> Actual results: Events: FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------------- ------- <invalid> <invalid> 1 {default-scheduler } Normal Scheduled Successfully assigned git-2-w5dfg to ip-172-31-9-165.ec2.internal <invalid> <invalid> 1 {kubelet ip-172-31-9-165.ec2.internal} spec.containers{git} WarningFailed Failed to create docker container with error: API error (500): mkdir /var/lib/docker/volumes/045e603cba13564651cf3b95f1689468c125a1dfcd023e6b7042ffb86d02fc7f: permission denied <invalid> <invalid> 1 {kubelet ip-172-31-9-165.ec2.internal} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "git" with RunContainerError: "runContainer: API error (500): mkdir /var/lib/docker/volumes/045e603cba13564651cf3b95f1689468c125a1dfcd023e6b7042ffb86d02fc7f: permission denied\n" Expected results: Gitserver pod should be running. Additional info: Have added emptyDir in gitserver.yaml. volumeMounts: - mountPath: /var/lib/git/ name: git volumes: - name: git emptyDir: {} triggers: - type: ConfigChange
The gitserver image currently declares 2 volumes: 1) /var/lib/origin (inherited from openshift/origin) 2) /var/lib/git Currently, it's possible to get it running on openshift online by creating separate pvc's for them: oc volume --add --type=pvc --name=git --claim-size=1G --overwrite dc/git oc volume --add --type=pvc --name=origin --claim-size=1G --mount-path=/var/lib/origin dc/git There's really no use for the origin volume, so I will change the parent image of the gitserver to be openshift/origin-base instead of openshift/origin. I will also state in the instructions that if you want to create the git server online, you must do it with a pvc claim... after running 'oc create -f gitserver.yaml', you'll need to run: oc volume --add --type=pvc --name=git --claim-size=1G --overwrite dc/git
envs: dev-preview-stg (openshift master:v3.2.0.44) dev-preview-int (openshift master:v3.2.0.40) The workaround doesn't work.Still meet the "API error (500): mkdir /var/lib/docker/volumes/045e603cba13564651cf3b95f1689468c125a1dfcd023e6b7042ffb86d02fc7f: permission denied\n" error $oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES AGE pvc-x9hh3 Bound pv-aws-ujuze 1Gi RWO 21m $oc get dc git -o yaml | grep pvc claimName: pvc-x9hh3 $ oc get pods NAME READY STATUS RESTARTS AGE git-1-deploy 0/1 DeadlineExceeded 0 23m git-2-deploy 0/1 DeadlineExceeded 0 21m git-3-deploy 0/1 DeadlineExceeded 0 17m git-4-deploy 0/1 Error 0 12m git-5-78ttx 0/1 RunContainerError 0 2m git-5-deploy 1/1 Running 0 2m
Sorry, should create pvc for /var/lib/origin too.Git server pod could be running now. $oc create -f gitserver.yaml $oc volume --add --type=pvc --name=git --claim-size=1G --overwrite dc/git $oc volume --add --type=pvc --name=origin --claim-size=1G --mount-path=/var/lib/origin dc/git
Fix in https://github.com/openshift/origin/pull/8902
@Cesar Pod could be running in online envs with gitserver-persistent.yaml. But online envs (dev-preview-stg and dev-preview-int) don't allow docker build.So will fail to create bc with docker strategy when push initial code to gitserver repo. remote: error: buildconfigs "ruby-hello-world" is forbidden: build strategy Docker is not allowed remote: deploymentconfig "ruby-hello-world" created remote: service "ruby-hello-world" created remote: --> Failed To http://git-xiuwang.b795.dev-preview-stg.openshiftapps.com/ruby-hello-world.git * [new branch] master -> master Can we change the docker build to s2i build for gitserver?
@XiuJuan I've submitted PR https://github.com/openshift/origin/pull/9031 which allows you to set an environment variable on the git server deployment that controls your default strategy. For the online case, we would need to set it to 'docker'.
Sorry, for online, you need to set it to 'source'
@cewong After seting the env to source, the created app is still using dockerstrategy when push initial code to git server repo. $ oc get dc -o yaml | grep -A 1 BUILD - name: BUILD_STRATEGY value: source remote: * An image stream will be created as "ruby-22-centos7:latest" that will track the source image remote: * A Docker build using source code from http://git:8080/ruby-hello-world.git will be created remote: * The resulting image will be pushed to image stream "ruby-hello-world:latest" remote: * Every time "ruby-22-centos7:latest" changes a new build will be triggered remote: * This image will be deployed in deployment config "ruby-hello-world" remote: * Port 8080 will be load balanced by service "ruby-hello-world" remote: * Other containers can access this service through the hostname "ruby-hello-world" remote: remote: --> Creating resources with label app=ruby-hello-world ... remote: imagestream "ruby-22-centos7" created remote: imagestream "ruby-hello-world" created remote: error: buildconfigs "ruby-hello-world" is forbidden: build strategy Docker is not allowed Delay so long time to check this,my bad
@XiuJuan it looks like the latest version of origin-gitserver has not been pushed to Docker hub. I pulled the openshift/origin-gitserver:latest and its /var/lib/git-hooks/post-receive doesn't include my latest change: https://github.com/openshift/origin/blob/master/examples/gitserver/hooks/post-receive#L30 I will check later today to see if a new image is pushed that contains my change.
Looks like the gitserver image has been updated. Please try again with the latest image.
@cewong, works now. You can move this bug to on-qa now.Thanks
template: https://raw.githubusercontent.com/openshift/origin/master/examples/gitserver/gitserver-persistent.yaml Could create git server pod with above template when set BUILD_STRATEGY=source in online env. Move this bug to verified.