Description of problem: set role: 'system:image-pruner' to user, and run the oadm prune command, it appears panic error. openshift v3.5.0.32-1+4f84c83 kubernetes v1.5.2+43a9be4 etcd 3.1.0 How reproducible: Always Steps to Reproduce: 1. On master node, login openshift with user successfully #oc login ec2-52-xx-xx-xx.compute-1.amazonaws.com:443 --config=/tmp/config 2. swith to system:admin user and set image-prune role to user: # oadm policy add-cluster-role-to-user system:image-pruner geliu cluster role "system:image-pruner" added: "geliu" 3. switch back to user and import or tag a imagestream 4. run the image-prune command, it appears panic error: # oadm prune images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0xeba7e0] goroutine 1 [running]: panic(0x3b44ba0, 0xc4200140a0) /usr/lib/golang/src/runtime/panic.go:500 +0x1a1 github.com/openshift/origin/pkg/cmd/admin/prune.getClients(0xc420f5c9c0, 0x7ffc7402664e, 0xb, 0xc4203cb360, 0x4c4708, 0x421fee9, 0x5, 0x391f820, 0x414383) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:497 +0x4e0 github.com/openshift/origin/pkg/cmd/admin/prune.(*PruneImagesOptions).Complete(0xc4206af710, 0xc420f5c9c0, 0xc420fa8480, 0xc420fc2e10, 0x0, 0x3, 0x92cd980, 0xc420026010, 0xc420b23c70, 0x4c2c66) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:157 +0x211 github.com/openshift/origin/pkg/cmd/admin/prune.NewCmdPruneImages.func1(0xc420fa8480, 0xc420fc2e10, 0x0, 0x3) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:108 +0x93 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc420fa8480, 0xc420fc2cf0, 0x3, 0x3, 0xc420fa8480, 0xc420fc2cf0) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x439 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420f5e900, 0xc420026008, 0xc420026018, 0xc420f5e900) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x367 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420f5e900, 0x4, 0xc420f5e900) /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b main.main() /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x196 Actual results: appear panic error, runtime error: invalid memory address or nil pointer dereference Expected results: image prune command should be suceed or prompt reasonable error msg
Ge Liu what's the contents of your /tmp/config file? It looks like it is not complete cert.
Maciej, the /tmp/config created by step 1. above: #oc login ec2-52-xx-xx-xx.compute-1.amazonaws.com:443 --config=/tmp/config # cat /tmp/config apiVersion: v1 clusters: - cluster: insecure-skip-tls-verify: true server: https://ec2-xx-xx-1xx-xx.compute-1.amazonaws.com:8443 name: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443 contexts: - context: cluster: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443 user: geliu/ec2-52-xx-xx-xx-compute-1-amazonaws-com:8443 name: /ec2-52-90-xx-xx-compute-1-amazonaws-com:8443/geliu - context: cluster: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443 namespace: lgproj user: geliu/ec2-52-xx-xx-213-compute-1-amazonaws-com:8443 name: lgproj/ec2-52-90-1xx-xx-compute-1-amazonaws-com:8443/geliu current-context: lgproj/ec2-52-xx-xx-xx-compute-1-amazonaws-com:8443/geliu kind: Config preferences: {} users: - name: geliu/ec2-52-90-xx-xx-compute-1-amazonaws-com:8443 user: token: PDIs3IoVHgHWTXVBmkhj7bHStNwUiKfRi4lF8287UeY
Master PR: https://github.com/openshift/origin/pull/13072 1.5 PR: https://github.com/openshift/origin/pull/13073
Commits pushed to master at https://github.com/openshift/origin https://github.com/openshift/origin/commit/174e3dd2174a1289700b3efabc59b5463a7a8141 Bug 1425706 - protect from nil tlsConfig. In very special case (client config does not contain ca cert nor auth cert nor set tls-insecure), tlsConfig can be nil and no error is returned from TLSConfigFor either, thus we need to check for nil on tlsConfig before proceeding in image pruning. https://github.com/openshift/origin/commit/a6633738e69d433736c6677ee5931bad47ef8ea5 Merge pull request #13072 from soltysh/bug1425706 Merged by openshift-bot
Verified in ocp 3.5: # openshift version openshift v3.5.0.34 kubernetes v1.5.2+43a9be4 etcd 3.1.0 # oadm prune images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config Dry run enabled - no modifications will be made. Add --confirm to remove images [root@host-8-172-x master]# oadm prune images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config --confirm [root@host-8-172-x master]# The panic error fixed, but it looks like the the image data still exist after run the prune command, I'm not sure whether the command line above it correct? -Before run prune: # oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample _layers _manifests _uploads # oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample/_manifests/revisions/sha256 091a11048fd3e796d4501b0c473bce09074ee39abe51c275ea4169f537c88b92 -After run prune: # oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample _layers _manifests _uploads # oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample/_manifests/revisions/sha256 091a11048fd3e796d4501b0c473bce09074ee39abe51c275ea4169f537c88b92
@maszulik, could u pls take a look the issue in my comment 5? anyway, the panic issue have been fixed, so close this bug.
@geliu, there are multiple options why this layer was not removed, ie. this could be a shared layer with other image (eg. the builder), or it was not even considered eligible for pruning. It's hard to say, please perform thorough tests (increasing loglevel to see what's being removed helps here) and open a bug if you feel like there's a problem with pruning.
Since this bug never reached customers, I am closing it.