Bug 2034889

Summary: `oc adm prune deployments` does not work
Product: OpenShift Container Platform Reporter: zhou ying <yinzhou>
Component: ocAssignee: Ross Peoples <rpeoples>
oc sub component: oc QA Contact: zhou ying <yinzhou>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: aos-bugs, maszulik, mfojtik
Version: 4.10   
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: EmergencyRequest
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-11 18:15:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhou ying 2021-12-22 11:52:53 UTC
Description of problem:
`oc adm prune deployments` does not work 

Version-Release number of selected component (if applicable):
oc version --client
Client Version: 4.10.0-202112201514.p0.g9214112.assembly.stream-9214112

How reproducible:
always

Steps to Reproduce:
1. Create dc and rollout more than 10 times;
[root@localhost ~]# oc get rc
NAME      DESIRED   CURRENT   READY   AGE
mydc-10   0         0         0       3h47m
mydc-11   0         0         0       3h26m
mydc-12   1         1         1       107s
mydc-2    0         0         0       6h44m
mydc-3    0         0         0       6h42m
mydc-4    0         0         0       6h39m
mydc-5    0         0         0       5h30m
mydc-6    0         0         0       5h21m
mydc-7    0         0         0       5h19m
mydc-8    0         0         0       5h7m
mydc-9    0         0         0       5h

2. Run command to prune the RC:
`oc adm prune deployments --confirm=true --keep-complete=2 --keep-failed=1 --keep-younger-than=1m`
3. Create deploy and rollout more times;
4. Run the command to prune the RS:
`oc adm prune deployments  --keep-complete=2 --keep-failed=1 --keep-younger-than=1m  --replica-sets=true`

Actual results:
2. Nothing was pruned:
[root@localhost ~]# oc adm prune deployments --confirm=true --keep-complete=2 --keep-failed=1 --keep-younger-than=1m  -v 6
I1222 19:50:20.754829   94321 loader.go:372] Config loaded from file:  /root/kubeconfig
I1222 19:50:21.752349   94321 round_trippers.go:553] GET https://api.yinzhou-ipv6b.qe.devcluster.openshift.com:6443/apis/apps.openshift.io/v1/deploymentconfigs 200 OK in 996 milliseconds
I1222 19:50:22.367199   94321 round_trippers.go:553] GET https://api.yinzhou-ipv6b.qe.devcluster.openshift.com:6443/api/v1/replicationcontrollers 200 OK in 305 milliseconds
I1222 19:50:22.985825   94321 prune.go:58] Creating deployment pruner with keepYoungerThan=1m0s, orphans=false, replicaSets=false, keepComplete=2, keepFailed=1

[root@localhost ~]# oc get rc
NAME      DESIRED   CURRENT   READY   AGE
mydc-10   0         0         0       4h48m
mydc-11   0         0         0       4h27m
mydc-12   1         1         1       62m
mydc-2    0         0         0       7h44m
mydc-3    0         0         0       7h43m
mydc-4    0         0         0       7h39m
mydc-5    0         0         0       6h30m
mydc-6    0         0         0       6h22m
mydc-7    0         0         0       6h20m
mydc-8    0         0         0       6h7m
mydc-9    0         0         0       6h1m


4. Hit painc error:
[root@localhost ~]# oc adm prune deployments  --keep-complete=2 --keep-failed=1 --keep-younger-than=1m  --replica-sets=true 
panic: runtime error: index out of range [17] with length 17

goroutine 1 [running]:
github.com/openshift/oc/pkg/cli/admin/prune/deployments.PruneDeploymentsOptions.Run({0x0, 0x0, 0x1, 0xdf8475800, 0x2, 0x1, {0x0, 0x0}, {0x40167e0, 0xc000bf5310}, ...})
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/deployments.go:180 +0xaf7
github.com/openshift/oc/pkg/cli/admin/prune/deployments.NewCmdPruneDeployments.func1(0xc000ae0f00, {0xc000b5f440, 0x0, 0x4})
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/deployments.go:78 +0x118
github.com/spf13/cobra.(*Command).execute(0xc000ae0f00, {0xc000b5f400, 0x4, 0x4})
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:860 +0x5f8
github.com/spf13/cobra.(*Command).ExecuteC(0xc00085ac80)
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:902
k8s.io/component-base/cli.Run(0xc00085ac80)
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/vendor/k8s.io/component-base/cli/run.go:105 +0x389
main.main()
	/builddir/build/BUILD/openshift-clients-4.10.0/__gopath/src/github.com/openshift/oc/cmd/oc/oc.go:78 +0x385

Expected results:


Additional info:

Comment 1 Michal Fojtik 2021-12-22 12:22:31 UTC
** A NOTE ABOUT USING URGENT **

This BZ has been set to urgent severity and priority. When a BZ is marked urgent priority Engineers are asked to stop whatever they are doing, putting everything else on hold.
Please be prepared to have reasonable justification ready to discuss, and ensure your own and engineering management are aware and agree this BZ is urgent. Keep in mind, urgent bugs are very expensive and have maximal management visibility.

NOTE: This bug was automatically assigned to an engineering manager with the severity reset to *unspecified* until the emergency is vetted and confirmed. Please do not manually override the severity.

** INFORMATION REQUIRED **

Please answer these questions before escalation to engineering:

1. Has a link to must-gather output been provided in this BZ? We cannot work without. If must-gather fails to run, attach all relevant logs and provide the error message of must-gather.
2. Give the output of "oc get clusteroperators -o yaml".
3. In case of degraded/unavailable operators, have all their logs and the logs of the operands been analyzed [yes/no]
4. List the top 5 relevant errors from the logs of the operators and operands in (3).
5. Order the list of degraded/unavailable operators according to which is likely the cause of the failure of the other, root-cause at the top.
6. Explain why (5) is likely the right order and list the information used for that assessment.
7. Explain why Engineering is necessary to make progress.

Comment 3 zhou ying 2021-12-27 02:56:25 UTC
Checked with latest oc client, still hit error :

[root@localhost ~]#  oc adm prune deployments  --keep-complete=2 --keep-failed=1 --keep-younger-than=1m  --replica-sets=true 
Dry run enabled - no modifications will be made. Add --confirm to remove deployments
panic: interface conversion: interface {} is *v1.Deployment, not *v1.DeploymentConfig

goroutine 1 [running]:
github.com/openshift/oc/pkg/cli/admin/prune/deployments.(*dataSet).ListDeployments(0xc000b47f18)
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/data.go:169 +0x149
github.com/openshift/oc/pkg/cli/admin/prune/deployments.(*perDeploymentResolver).Resolve(0xc000d11de0)
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/resolvers.go:109 +0x42
github.com/openshift/oc/pkg/cli/admin/prune/deployments.(*mergeResolver).Resolve(0x1)
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/resolvers.go:28 +0xa2
github.com/openshift/oc/pkg/cli/admin/prune/deployments.(*pruner).Prune(0x401a640, {0x4016d20, 0xc0006d8e10})
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/prune.go:88 +0x2f
github.com/openshift/oc/pkg/cli/admin/prune/deployments.PruneDeploymentsOptions.Run({0x0, 0x0, 0x1, 0xdf8475800, 0x2, 0x1, {0x0, 0x0}, {0x4016780, 0xc0006f8a40}, ...})
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/deployments.go:205 +0xaaa
github.com/openshift/oc/pkg/cli/admin/prune/deployments.NewCmdPruneDeployments.func1(0xc000bc8280, {0xc000c86240, 0x0, 0x4})
	/go/src/github.com/openshift/oc/pkg/cli/admin/prune/deployments/deployments.go:78 +0x118
github.com/spf13/cobra.(*Command).execute(0xc000bc8280, {0xc000c86200, 0x4, 0x4})
	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:860 +0x5f8
github.com/spf13/cobra.(*Command).ExecuteC(0xc0008f2a00)
	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:902
k8s.io/component-base/cli.Run(0xc0008f2a00)
	/go/src/github.com/openshift/oc/vendor/k8s.io/component-base/cli/run.go:105 +0x389
main.main()
	/go/src/github.com/openshift/oc/cmd/oc/oc.go:78 +0x385
[root@localhost ~]# oc version 
Client Version: 4.10.0-0.nightly-2021-12-25-025639
Server Version: 4.10.0-0.nightly-2021-12-23-153012
Kubernetes Version: v1.22.1+6859754
[root@localhost ~]# oc version  --client -o yaml 
clientVersion:
  buildDate: "2021-12-24T22:49:12Z"
  compiler: gc
  gitCommit: 5fe7203294827ca91e2c94d1dacbf23477553ba3
  gitTreeState: clean
  gitVersion: 4.10.0-202112242226.p0.g5fe7203.assembly.stream-5fe7203
  goVersion: go1.17.2
  major: ""
  minor: ""
  platform: linux/amd64
releaseClientVersion: 4.10.0-0.nightly-2021-12-25-025639

[root@localhost oc]# git log
commit 5fe7203294827ca91e2c94d1dacbf23477553ba3 (HEAD -> master, origin/release-4.11, origin/release-4.10, origin/master, origin/HEAD)
Merge: b7970db36 bbfcffdef
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Dec 24 21:26:08 2021 +0100

    Merge pull request #1005 from deejross/bz2034889-fix-prune-panic
    
    Bug 2034889: Fix prune deployment panic

Comment 5 zhou ying 2022-01-04 05:59:07 UTC
The panic has fixed , but the prune function still has issue , will verify this one and open a new bug to follow the prune function. 

oc adm prune deployments  --keep-complete=2 --keep-failed=1 --keep-younger-than=1m  --replica-sets=true 
Dry run enabled - no modifications will be made. Add --confirm to remove deployments

[root@localhost ~]# oc version --client
Client Version: 4.10.0-202201031825.p0.g25851bd.assembly.stream-25851bd