Description of the problem: ACM 2.5 cannot create known_hosts file when pulling from ssh git repo Steps to reproduce: Create an application from ACM using ssh key Additional info: After installing ACM & configuring a multicluster hub, we do the following steps: ``` cat <<EOF | oc apply -f - apiVersion: v1 kind: Namespace metadata: name: bit-ros-acm-policies EOF cat <<EOFSEC | oc apply -n bit-ros-acm-policies -f - apiVersion: v1 kind: Secret metadata: name: repo-ros-acm-policies namespace: bit-ros-acm-policies type: Opaque data: sshKey: $(echo -n "${ssh_privatekey}" | base64 -w 0) EOFSEC cat <<EOFCHAN | oc apply -n bit-ros-acm-policies -f - apiVersion: apps.open-cluster-management.io/v1 kind: Channel metadata: annotations: apps.open-cluster-management.io/reconcile-rate: high name: ros-policies-channel namespace: bit-ros-acm-policies spec: secretRef: name: repo-ros-acm-policies pathname: ssh://git.ch/repo/ros-cluster.git type: Git EOFCHAN cat <<EOFSUB | oc apply -n bit-ros-acm-policies -f - apiVersion: apps.open-cluster-management.io/v1 kind: Subscription metadata: annotations: apps.open-cluster-management.io/git-branch: master apps.open-cluster-management.io/git-path: acm-policies/overlays/${CLUSTER_NAME} name: ros-policies-subscription namespace: bit-ros-acm-policies spec: channel: bit-ros-acm-policies/ros-policies-channel placement: local: true EOFSUB ``` However no policies are synced out of the git-path overlay into ACM. The Channel has the following status: ``` apiVersion: apps.open-cluster-management.io/v1 kind: Subscription metadata: annotations: apps.open-cluster-management.io/git-branch: master apps.open-cluster-management.io/git-path: acm-policies/overlays/CLUSTER_NAME creationTimestamp: "2022-06-29T11:16:34Z" generation: 1 name: ros-policies-subscription namespace: bit-ros-acm-policies spec: channel: bit-ros-acm-policies/ros-policies-channel placement: local: true status: ansiblejobs: {} appstatusReference: kubectl get appsubstatus -n bit-ros-acm-policies ros-policies-subscription lastUpdateTime: "2022-06-29T15:17:54Z" message: Active phase: Subscribed ``` If we check the logs of the pod multicluster-operators-standalone-subscription-6d78d97d4c-2pqz9 I see the following logs: ``` I0629 15:17:48.708681 1 helm_subscriber_item.go:361] Checking to see if the HelmRelease open-cluster-management/volsync-addon-controller-7c984 exists I0629 15:17:48.710387 1 helm_subscriber_item.go:380] Checking to see if the HelmRelease open-cluster-management/volsync-addon-controller-7c984 status is populated I0629 15:17:54.424990 1 git_subscriber_item.go:169] Re-try #1: subcribing to the Git repo I0629 15:17:54.425017 1 git_subscriber_item.go:185] enter doSubscription: bit-ros-acm-policies/ros-policies-subscription I0629 15:17:54.430958 1 git_subscriber_item.go:203] Subscribing ...ros-policies-subscription I0629 15:17:54.449881 1 gitrepo.go:210] Connecting to Git server via SSH I0629 15:17:54.449905 1 gitrepo.go:404] sshhostname = bitbucket.example.ch I0629 15:17:54.449911 1 gitrepo.go:405] sshhostport = I0629 15:17:54.449916 1 gitrepo.go:407] Getting public SSH host key for bitbucket.example.ch I0629 15:17:54.520901 1 gitrepo.go:422] SSH host key: bitbucket.example.ch.ch ssh-rsa AAAACensored E0629 15:17:54.520929 1 gitrepo.go:425] failed to write known_hosts file: open /tmp/bit-ros-acm-policies/ros-policies-subscription/known_hosts: no such file or directory E0629 15:17:54.520941 1 gitrepo.go:263] Failed to get Git clone options with the primary channel. Trying the secondary channel. E0629 15:17:54.520947 1 gitrepo.go:176] no secondary channel to try E0629 15:17:54.520958 1 git_subscriber_item.go:265] failed to build git connection optionsUnable to clone the git repo ssh://git.ch/repo/ros-cluster.git I0629 15:17:54.520966 1 git_subscriber_item.go:268] exit doSubscription: bit-ros-acm-policies/ros-policies-subscription E0629 15:17:54.520972 1 git_subscriber_item.go:173] failed to build git connection optionsSubscription error. ``` If we connect to the pod, we can see that the directory is missing, which might be the issue: ``` $ oc rsh multicluster-operators-standalone-subscription-6d78d97d4c-2pqz9 sh-4.4$ ls -l /tmp/ total 0 drwxr-x---. 9 1001 root 202 Jun 29 11:13 hr-charts drwxr-xr-x. 3 1001 root 55 Jun 29 14:58 open-cluster-management sh-4.4$ ls -l /tmp/open-cluster-management/ total 0 drwxr-xr-x. 2 1001 root 6 Jun 29 14:58 hive-clusterimagesets-subscription-fast-0 sh-4.4$ ls -l /tmp/hr-charts/ total 0 drwxr-x---. 3 1001 root 37 Jun 29 11:13 cluster-lifecycle-9ea7f drwxr-x---. 3 1001 root 37 Jun 29 11:13 console-chart-9718b drwxr-x---. 3 1001 root 37 Jun 29 11:13 grc-33032 drwxr-x---. 3 1001 root 37 Jun 29 11:13 management-ingress-fdbd5 drwxr-x---. 3 1001 root 37 Jun 29 11:13 policyreport-00422 drwxr-x---. 3 1001 root 37 Jun 29 11:13 search-prod-8684b drwxr-x---. 3 1001 root 37 Jun 29 11:13 volsync-addon-controller-7c984 ``` After creating the directory within the pod: ``` sh-4.4$ mkdir -p /tmp/bit-ros-acm-policies/ros-policies-subscription ``` The git repository can be synced and the policies appear in ACM ``` E0629 15:22:54.803314 1 git_subscriber_item.go:160] failed to build git connection optionsSubscription error. ##### here we created the directory in /tmp I0629 15:23:54.803766 1 git_subscriber_item.go:169] Re-try #1: subcribing to the Git repo I0629 15:23:54.803797 1 git_subscriber_item.go:185] enter doSubscription: bit-ros-acm-policies/ros-policies-subscription I0629 15:23:54.811902 1 git_subscriber_item.go:203] Subscribing ...ros-policies-subscription I0629 15:23:54.830170 1 gitrepo.go:210] Connecting to Git server via SSH I0629 15:23:54.830196 1 gitrepo.go:404] sshhostname = bitbucket.example.ch I0629 15:23:54.830202 1 gitrepo.go:405] sshhostport = I0629 15:23:54.830207 1 gitrepo.go:407] Getting public SSH host key for bitbucket.example.ch I0629 15:23:54.895361 1 gitrepo.go:422] SSH host key: bitbucket.example.ch ssh-rsa AAAABCensored I0629 15:23:54.895511 1 gitrepo.go:461] Using SSH known host keys E0629 15:23:54.895684 1 gitrepo.go:176] no secondary channel to try I0629 15:23:54.895700 1 gitrepo.go:291] Cloning ssh://git.ch/repo/ros-cluster.git into /tmp/bit-ros-acm-policies/ros-policies-subscription I0629 15:23:54.895707 1 gitrepo.go:293] cloneOptions.DestDir = /tmp/bit-ros-acm-policies/ros-policies-subscription I0629 15:23:54.895711 1 gitrepo.go:294] cloneOptions.Branch = refs/heads/master I0629 15:23:54.895718 1 gitrepo.go:295] cloneOptions.CommitHash = I0629 15:23:54.895722 1 gitrepo.go:296] cloneOptions.RevisionTag = I0629 15:23:54.895726 1 gitrepo.go:297] cloneOptions.CloneDepth = 1 I0629 15:23:55.193210 1 gitrepo.go:330] Successfully cloned the repo and the current branch is master I0629 15:23:55.193271 1 git_subscriber_item.go:271] Git commit: 01bCensored ```
Issue will be fixed in 2.5.2 https://github.com/stolostron/multicloud-operators-subscription/pull/780
The fix will be in 2.5.2
In 2.5.2 the Application successfully creates host_files and deploys the application using the SSH key. Logs show successful creation and generates a git commit