Bug 1748173 - [CPMA] cpmn utility doesn't transform Quota and ClusterQuota configurations from ocp3.7
Summary: [CPMA] cpmn utility doesn't transform Quota and ClusterQuota configurations f...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Migration Tooling
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.2.0
Assignee: Gilles Dubreuil
QA Contact: Xin jiang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-03 05:38 UTC by Xin jiang
Modified: 2019-10-30 04:45 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-30 04:44:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:3151 0 None None None 2019-10-30 04:45:04 UTC

Description Xin jiang 2019-09-03 05:38:13 UTC
Description of problem:

Quota and ClusterQuota configurations were added into ocp3.7, and then using cpma utility transformed Quota and ClusterQuota configurations from ocp3.7 but quota related CRs was not generated


Version-Release number of selected component (if applicable):
OCP3.7
cpma commit id:commit bbf1f97aaf646f94691c46ee520c823520bb73c1

How reproducible:
always


Steps to Reproduce:
1.1. Apply Quota and ClusterQuota Configuration in OCP 3 cluster
$ oc create -f https://raw.githubusercontent.com/XinRedhat/cpma-test/master/quota/quota-demo.yaml 

2.Execute cpma utility to generate CR Manifests
$ ./bin/cpma

Actual results:
It didn't generate quota related CRs.
$ tree data
data
├── 10.0.151.250
│   └── etc
│       ├── containers
│       │   └── registries.conf
│       ├── etcd
│       │   └── etcd.conf
│       └── origin
│           └── master
│               ├── htpasswd
│               └── master-config.yaml
├── manifests
│   ├── 100_CPMA-cluster-config-image.yaml
│   ├── 100_CPMA-cluster-config-oauth.yaml
│   ├── 100_CPMA-cluster-config-project.yaml
│   ├── 100_CPMA-cluster-config-sdn.yaml
│   ├── 100_CPMA-cluster-config-secret-github-secret.yaml
│   ├── 100_CPMA-cluster-config-secret-gitlab-secret.yaml
│   └── 100_CPMA-cluster-config-secret-htpasswd-secret.yaml
├── report.html
└── report.json

Expected results:
It should generate such like below CRs under manifests directory.

│   ├── 100_CPMA-cluster-quota-resource-clusterquotatest.yaml
│   └── 100_CPMA-object-quota-test-resource-quota-object-quota-demo.yaml


Additional info:
$ cat ~/cpma.yaml:
clustername: ci-vm-10-0-151-250-hosted-upshift-rdu2-redhat-com:8443
configsource: remote
crioconfigfile: /etc/crio/crio.conf
debug: false
etcdconfigfile: /etc/etcd/etcd.conf
fetchfromremote: true
home: /Users/xinjiang
hostname: 10.0.151.250
insecurehostkey: false
manifests: true
masterconfigfile: /etc/origin/master/master-config.yaml
nodeconfigfile: /etc/origin/node/node-config.yaml
registriesconfigfile: /etc/containers/registries.conf
reporting: true
saveconfig: true
silent: false
sshlogin: root
sshport: 22
sshprivatekey: /Users/xinjiang/ocp/scripts/libra.pem
workdir: data

Comment 1 Gilles Dubreuil 2019-09-06 09:23:59 UTC
Hi Xin,

I can't reproduce it when applying the same test set mentioned in #c0:

$ oc create -f https://raw.githubusercontent.com/XinRedhat/cpma-test/master/quota/quota-demo.yaml
namespace/object-quota-test created
resourcequota/object-quota-demo created
clusterresourcequota.quota.openshift.io/clusterquotatest created

$ bin/cpma 
.../...          
INFO[06 Sep 19 10:54 CEST] Flushing manifests to disk                   
INFO[06 Sep 19 10:54 CEST] CRD:Added: data/manifests/100_CPMA-cluster-quota-resource-clusterquotatest.yaml 
INFO[06 Sep 19 10:54 CEST] CRD:Added: data/manifests/100_CPMA-object-quota-test-resource-quota-object-quota-demo.yaml 
.../...

Maybe there was some delay in your case for the resources creation.
Could you please re-test, and after creating the resource please verify first the objects are in place before executing cpma?

$ export TOKEN=$(oc whoami -t)
$ export CLUSTER="https://openshift.gildub1.lab.pnq2.cee.redhat.com:443"

# Cluster Resource Quotas
$curl -X GET -H "Authorization: Bearer $TOKEN" --insecure ${CLUSTER}/apis/quota.openshift.io/v1/clusterresourcequotas

# Resource Quotas
$curl -X GET -H "Authorization: Bearer $TOKEN" --insecure ${CLUSTER}/api/v1/namespaces/object-quota-test/resourcequotas/

Comment 2 Gilles Dubreuil 2019-09-06 09:35:43 UTC
Hi Xin,

I'm missed the fact that this is occurring on OCP 3.7.
Therefore I'm going to test accordingly and will get back to you shortly.

Comment 3 Gilles Dubreuil 2019-09-06 14:11:13 UTC
I confirm the issue against OCP 3.7.

Actually it's the whole Cluster transform that fails.

Because the Quotas (ClusterQuotas and ResourceQuotas) are the only manifests generated during the Cluster transform, the rest being reporting only.

The culprit is the DeploymentList API request that fails:
Error message is:

-------------------------------------------------------------------------------------------------------------------------
> github.com/fusor/cpma/pkg/transform.ClusterTransform.Extract() ./pkg/transform/cluster_transform.go:139 (PC: 0x188d914)
   134:                 }
   135:                 namespaceResources.RouteList = routesList
   136:
   137:                 deploymentList, err := api.ListDeployments(namespace.Name)
   138:                 if err != nil {
=> 139:                         return nil, err
   140:                 }
   141:                 namespaceResources.DeploymentList = deploymentList
   142:
   143:                 daemonSetList, err := api.ListDaemonSets(namespace.Name)
   144:                 if err != nil {
(dlv) n
> github.com/fusor/cpma/pkg/transform.(*ClusterTransform).Extract() <autogenerated>:1 (PC: 0x189e15d)
Values returned:
        ~r0: github.com/fusor/cpma/pkg/transform.Extraction nil
        ~r1: error(*k8s.io/apimachinery/pkg/api/errors.StatusError) *{
                ErrStatus: k8s.io/apimachinery/pkg/apis/meta/v1.Status {
                        TypeMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.TypeMeta")(0xc0003c8cf0),
                        ListMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.ListMeta")(0xc0003c8d10),
                        Status: "Failure",
                        Message: "the server could not find the requested resource",
                        Reason: "NotFound",
                        Details: *(*"k8s.io/apimachinery/pkg/apis/meta/v1.StatusDetails")(0xc0002b75c0),
                        Code: 404,},}
-------------------------------------------------------------------------------------------------------------------------


OCP 3.7, which uses Kubernetes 1.7, do support deployment resource as betav1:
-------------------------------------------------------------------------------------------------------------------
$ curl -X GET -H "Authorization: Bearer $TOKEN" --insecure ${CLUSTER}/apis/apps/v1beta1/deployments
{
  "kind": "DeploymentList",
  "apiVersion": "apps/v1beta1",
  "metadata": {
    "selfLink": "/apis/apps/v1beta1/deployments",
    "resourceVersion": "16975"
  },
  "items": []
}
$ curl -X GET -H "Authorization: Bearer $TOKEN" --insecure ${CLUSTER}/apis/apps/v1/deployments
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "the server could not find the requested resource",
  "reason": "NotFound",
  "details": {},
  "code": 404
-------------------------------------------------------------------------------------------------------------------

Comment 4 Gilles Dubreuil 2019-09-09 08:22:56 UTC
This has been addressed by using API v1beta1 libraries for handling Deployment and DeamonSets requests for any version.

The corresponding patch https://github.com/fusor/cpma/pull/388 has been merged to upstream master branch.

Comment 5 Gilles Dubreuil 2019-09-20 15:48:40 UTC
Backported to release-1.0 branch and available in current respective build

Comment 6 Xin jiang 2019-09-23 03:23:41 UTC
Verified and this issue is gone.

Git commit id :commit 006c5698376dda59438d6b25e78f00ad1dd630a4 (HEAD -> release-1.0, origin/release-1.0)

Comment 8 errata-xmlrpc 2019-10-30 04:44:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3151


Note You need to log in before you can comment on or make changes to this bug.