Bug 1425706 - oadm prune image appears error "panic: runtime error: invalid memory address or nil pointer dereference"
Summary: oadm prune image appears error "panic: runtime error: invalid memory address ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 3.5.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Maciej Szulik
QA Contact: Xingxing Xia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-22 08:01 UTC by ge liu
Modified: 2017-03-02 22:11 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-03-02 22:11:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description ge liu 2017-02-22 08:01:16 UTC
Description of problem:
set role: 'system:image-pruner' to user, and run the oadm prune command, it appears panic error.

openshift v3.5.0.32-1+4f84c83
kubernetes v1.5.2+43a9be4
etcd 3.1.0

How reproducible:
Always

Steps to Reproduce:
1. On master node, login openshift with user successfully

#oc login ec2-52-xx-xx-xx.compute-1.amazonaws.com:443 --config=/tmp/config


2. swith to system:admin user and set image-prune role to user:

# oadm policy add-cluster-role-to-user system:image-pruner  geliu
cluster role "system:image-pruner" added: "geliu"

3. switch back to user and import or tag a imagestream

4. run the image-prune command, it appears panic error:

#  oadm prune  images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0xeba7e0]

goroutine 1 [running]:
panic(0x3b44ba0, 0xc4200140a0)
    /usr/lib/golang/src/runtime/panic.go:500 +0x1a1
github.com/openshift/origin/pkg/cmd/admin/prune.getClients(0xc420f5c9c0, 0x7ffc7402664e, 0xb, 0xc4203cb360, 0x4c4708, 0x421fee9, 0x5, 0x391f820, 0x414383)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:497 +0x4e0
github.com/openshift/origin/pkg/cmd/admin/prune.(*PruneImagesOptions).Complete(0xc4206af710, 0xc420f5c9c0, 0xc420fa8480, 0xc420fc2e10, 0x0, 0x3, 0x92cd980, 0xc420026010, 0xc420b23c70, 0x4c2c66)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:157 +0x211
github.com/openshift/origin/pkg/cmd/admin/prune.NewCmdPruneImages.func1(0xc420fa8480, 0xc420fc2e10, 0x0, 0x3)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/pkg/cmd/admin/prune/images.go:108 +0x93
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc420fa8480, 0xc420fc2cf0, 0x3, 0x3, 0xc420fa8480, 0xc420fc2cf0)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x439
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420f5e900, 0xc420026008, 0xc420026018, 0xc420f5e900)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x367
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420f5e900, 0x4, 0xc420f5e900)
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
    /builddir/build/BUILD/atomic-openshift-git-0.4f84c83/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x196


Actual results:

appear panic error, runtime error: invalid memory address or nil pointer dereference

Expected results:

image prune command should be suceed or prompt reasonable error msg

Comment 1 Maciej Szulik 2017-02-22 16:21:47 UTC
Ge Liu what's the contents of your /tmp/config file? It looks like it is not complete cert.

Comment 2 ge liu 2017-02-23 07:32:53 UTC
Maciej, the /tmp/config created by step 1. above: #oc login ec2-52-xx-xx-xx.compute-1.amazonaws.com:443 --config=/tmp/config 


# cat /tmp/config
apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: https://ec2-xx-xx-1xx-xx.compute-1.amazonaws.com:8443
  name: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443
contexts:
- context:
    cluster: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443
    user: geliu/ec2-52-xx-xx-xx-compute-1-amazonaws-com:8443
  name: /ec2-52-90-xx-xx-compute-1-amazonaws-com:8443/geliu
- context:
    cluster: ec2-52-90-xx-xx-compute-1-amazonaws-com:8443
    namespace: lgproj
    user: geliu/ec2-52-xx-xx-213-compute-1-amazonaws-com:8443
  name: lgproj/ec2-52-90-1xx-xx-compute-1-amazonaws-com:8443/geliu
current-context: lgproj/ec2-52-xx-xx-xx-compute-1-amazonaws-com:8443/geliu
kind: Config
preferences: {}
users:
- name: geliu/ec2-52-90-xx-xx-compute-1-amazonaws-com:8443
  user:
    token: PDIs3IoVHgHWTXVBmkhj7bHStNwUiKfRi4lF8287UeY

Comment 4 openshift-github-bot 2017-02-24 12:50:23 UTC
Commits pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/174e3dd2174a1289700b3efabc59b5463a7a8141
Bug 1425706 - protect from nil tlsConfig.

In very special case (client config does not contain ca cert nor auth cert nor
set tls-insecure), tlsConfig can be nil and no error is returned from TLSConfigFor
either, thus we need to check for nil on tlsConfig before proceeding in
image pruning.

https://github.com/openshift/origin/commit/a6633738e69d433736c6677ee5931bad47ef8ea5
Merge pull request #13072 from soltysh/bug1425706

Merged by openshift-bot

Comment 5 ge liu 2017-02-27 03:26:07 UTC
Verified in ocp 3.5:

# openshift version
openshift v3.5.0.34
kubernetes v1.5.2+43a9be4
etcd 3.1.0

# oadm prune  images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config
Dry run enabled - no modifications will be made. Add --confirm to remove images
[root@host-8-172-x master]# oadm prune  images --all=true --registry-url=https://172.xx.xx.xx:5000 --certificate-authority=/tmp/config --confirm
[root@host-8-172-x master]# 

The panic error fixed, but it looks like the the image data still exist after run the prune command, I'm not sure whether the command line above it correct?


-Before run prune:

# oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample
_layers  _manifests  _uploads

# oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample/_manifests/revisions/sha256
091a11048fd3e796d4501b0c473bce09074ee39abe51c275ea4169f537c88b92

-After run prune:

# oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample
_layers  _manifests  _uploads

# oc rsh docker-registry-5-6c69f ls /registry/docker/registry/v2/repositories/lgproj/origin-ruby-sample/_manifests/revisions/sha256
091a11048fd3e796d4501b0c473bce09074ee39abe51c275ea4169f537c88b92

Comment 6 ge liu 2017-02-27 03:28:06 UTC
@maszulik, could u pls take a look the issue in my comment 5? anyway, the panic issue have been fixed, so close this bug.

Comment 7 Maciej Szulik 2017-02-27 11:12:12 UTC
@geliu, there are multiple options why this layer was not removed, ie. this could be a shared layer with other image (eg. the builder), or it was not even considered eligible for pruning. It's hard to say, please perform thorough tests (increasing loglevel to see what's being removed helps here) and open a bug if you feel like there's a problem with pruning.

Comment 8 Troy Dawson 2017-03-02 22:11:36 UTC
Since this bug never reached customers, I am closing it.


Note You need to log in before you can comment on or make changes to this bug.