>: failed to reconcile. failed to add ceph rbd mirror peer: failed to import bootstrap peer token: failed to add rbd-mirror peer token for pool "ocs-storagecluster-cephblockpool". . 2022-03-25T19:24:13.048+0000 7f829 36dc2c0 -1 librbd::api::Mirror: peer_bootstrap_import: failed to connect to peer cluster: (110) Connection timed out This is mostly due to the network issue, rbd is not able to import the bootstrap token from the remote clusters. Mon on ocpa cluster a=172.40.126.252:6789,b=172.40.195.16:6789, c=172.40.111.218:6789 Mon on ocpb cluster a=172.30.186.194:6789,b=172.30.244.89:6789,c=172.30.161.165:6789 1) We need to check is networking is proper and we are able to reach remote mon endpoints from the other cluster 2) Can we get the secret content of 5f4bdf98d073ced07cbbddf249d9f434aea2df0 (its a secret name) (which contains remote cluster details) from the ocpb cluster Moving it to Rook, as Rook is responsible for setting up the mirroring.
They are some questions coming from the support team which have not been answered yet. Depending on the response no eng investigation would be needed. What kind of help are you expecting from eng that we did not get from support yet? This BZ seems a bit premature to me. Has anyone done some simple connectivity tests between both clusters? Thanks!
The ping is not helping unfortunately, we need to test the connectivity to the mons directly so either using curl or telnet like: curl <mon-ip>:<port> And see we get a reply. Thanks!
Are the telnet command executed from inside the rook-ceph operator pod? It is the one adding the peers so it's important. Thanks.
Thanks Kelson, Steve, great to see a connection could be established but it seems that the peer was not added. Can we get: * complete rook-ceph-operator log with DEBUG mode (you can edit the rook-ceph-operator-config and set ROOK_LOG_LEVEL: DEBUG) * the output of "oc get cephblockpool -n openshift-storage -o yaml" * logs from the rbd-mirror daemon pod Thanks.
Thanks Steve, one more thing can you attach the content of the secret "0c0e4c098c43c0ddd72f93031f92addff6cfc3d"? Thanks. I wonder if the information in the secret are correct. Also, have you tried adding the peer "manually" while logging inside the rook operator? Thanks
Kelson, Can you update the secret from the UI? The Operator would reconcile that afterward. Thanks!
Not sure if Ramen is the right sub-component but that's DR. This is related to the component exchanging connection details between cluster (for peer addition). In this unsupported case, the secret content was invalid and we are trying to understand why. Thanks!