Created attachment 1791630 [details] Topology view Description of the problem: I am trying to deploy a simple application using ACM. The application is described by this git repository: https://github.com/larsks/example-k8s-app There are only a small number of resources: - A deployment - A PVC - A service - A secret I was able to deploy this successfully earlier today, although it took a surprisingly long time for it to reconcile. I deleted the secret from the repository, expecting to see the secret removed from the managed cluster, but instead all of the application resources disappeared from the topology view (see the attached screenshot). Looking at the appsub resource on the managed cluster, I see the following status: ansiblejobs: {} lastUpdateTime: '2021-06-16T23:45:04Z' message: Active phase: Subscribed statuses: /: packages: larsks-example-k8s-app-channel-Deployment-example: lastUpdateTime: '2021-06-16T23:45:04Z' phase: Subscribed resourceStatus: availableReplicas: 1 conditions: - lastTransitionTime: '2021-06-16T16:02:14Z' lastUpdateTime: '2021-06-16T16:02:14Z' message: Deployment has minimum availability. reason: MinimumReplicasAvailable status: 'True' type: Available - lastTransitionTime: '2021-06-16T16:01:49Z' lastUpdateTime: '2021-06-16T23:27:06Z' message: ReplicaSet "example-6f5d85858" has successfully progressed. reason: NewReplicaSetAvailable status: 'True' type: Progressing observedGeneration: 2 readyReplicas: 1 replicas: 1 updatedReplicas: 1 larsks-example-k8s-app-channel-PersistentVolumeClaim-example-data-claim: lastUpdateTime: '2021-06-16T16:19:51Z' phase: Subscribed resourceStatus: accessModes: - ReadWriteOnce capacity: storage: 1Gi phase: Bound larsks-example-k8s-app-channel-Service-example: lastUpdateTime: '2021-06-16T16:19:51Z' phase: Subscribed resourceStatus: loadBalancer: ingress: - ip: 192.12.185.35 It looks as if the state of the resources on the managed cluster is correct. I see similar status in the appsub resource on the hub cluster: lastUpdateTime: '2021-06-16T23:27:00Z' message: zero:Active phase: Propagated statuses: zero: packages: larsks-example-k8s-app-channel-Deployment-example: lastUpdateTime: '2021-06-16T16:19:51Z' phase: Subscribed resourceStatus: availableReplicas: 1 conditions: - lastTransitionTime: '2021-06-16T16:02:14Z' lastUpdateTime: '2021-06-16T16:02:14Z' message: Deployment has minimum availability. reason: MinimumReplicasAvailable status: 'True' type: Available - lastTransitionTime: '2021-06-16T16:01:49Z' lastUpdateTime: '2021-06-16T16:02:14Z' message: ReplicaSet "example-85d55455cc" has successfully progressed. reason: NewReplicaSetAvailable status: 'True' type: Progressing observedGeneration: 1 readyReplicas: 1 replicas: 1 updatedReplicas: 1 larsks-example-k8s-app-channel-PersistentVolumeClaim-example-data-claim: lastUpdateTime: '2021-06-16T16:19:51Z' phase: Subscribed resourceStatus: accessModes: - ReadWriteOnce capacity: storage: 1Gi phase: Bound larsks-example-k8s-app-channel-Service-example: lastUpdateTime: '2021-06-16T16:19:51Z' phase: Subscribed resourceStatus: loadBalancer: ingress: - ip: 192.12.185.35 But despite this, the resources don't show up in the topology view. What's going on? Release version: acm 2.2.3 Operator snapshot version: OCP version: - 4.6.31 (hub cluster) - 4.7.13 (managed cluster)
G2Bsync 864290490 comment xiangjingli Fri, 18 Jun 2021 21:48:26 UTC G2Bsync Verified the following appsub on ACM 2.2 and 2.3 but can't reproduce it. ``` apiVersion: v1 kind: Namespace metadata: name: lars-sandbox --- apiVersion: apps.open-cluster-management.io/v1 kind: Channel metadata: name: larsks-example-k8s-app-channel namespace: lars-sandbox spec: pathname: https://github.com/larsks/example-k8s-app type: GitHub --- apiVersion: apps.open-cluster-management.io/v1 kind: PlacementRule metadata: annotations: apps.open-cluster-management.io/git-commit: e55dfaf702ae648bf530743d9999649aa9484aa0 name: lars-acm-example-placement-1 namespace: lars-sandbox spec: clusterSelector: {} --- apiVersion: apps.open-cluster-management.io/v1 kind: Subscription metadata: name: lars-acm-example-subscription-1 namespace: lars-sandbox spec: channel: lars-sandbox/larsks-example-k8s-app-channel placement: placementRef: kind: PlacementRule name: lars-acm-example-placement-1 ``` By applying the appsub, the two appsub annotations are set as expected. These two annotations are used by topology view for getting the deployed resources list. ``` annotations: apps.open-cluster-management.io/deployables: apps.open-cluster-management.io/topo ``` It looks like a env issue on customer cluster. I would suggest to clean up the lars-sandbox NS first, make sure the NS is removed and there is not any deployable resources there. then re-run the appsub as above. 1. oc delete namespace lars-sandbox --wait=false --ignore-not-found 2. re-applyu the appsub
Mike, This appears to be completely reproducible. After deleting the "lars-sandbox" namespace on both the hub cluster and the target cluster, I then: 1. Submitted https://github.com/larsks/example-acm-app using `kustomize build | oc apply -f-`. This creates the app in ACM, which then successfully deploys the resources from https://github.com/larsks/example-k8s-app. 2. I then commit a change to https://github.com/larsks/example-k8s-app that removes the secret resource. At this point, ACM appears to remove the resources from the target cluster, and ends up in the state pictures in the existing attachment.
Hi Mike, I'm going to try this with our latest code and see if there are any improvements, if not, we should raise this with dev.