Bug 2170129

Summary:	[GSS] cephfs clones are failing
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	kelwhite
Component:	ceph	Assignee:	Kotresh HR <khiremat>
ceph sub component:	CephFS	QA Contact:	Elad <ebenahar>
Status:	CLOSED NOTABUG	Docs Contact:
Severity:	urgent
Priority:	unspecified	CC:	bniver, hnallurv, khiremat, mmuench, muagarwa, ocs-bugs, odf-bz-bot, sostapov, vshankar
Version:	4.10	Flags:	khiremat: needinfo-
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-03-23 16:24:38 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Comment 9 kelwhite 2023-02-22 21:12:01 UTC

Changing direction, how does csi do the mounting of cephfs? Found the section where this seems to happen but I know nothing about GO:

https://github.com/ceph/ceph-csi/tree/devel/internal/cephfs/mounter
https://github.com/ceph/ceph-csi/blob/devel/internal/cephfs/mounter/volumemounter.go

Seems to me we either need to use one of the options:

// Load available ceph mounters installed on system into availableMounters
// Called from driver.go's Run().
func LoadAvailableMounters(conf *util.Config) error {
	// #nosec
	fuseMounterProbe := exec.Command("ceph-fuse", "--version")
	// #nosec
	kernelMounterProbe := exec.Command("mount.ceph")

However, neither of these work based on the above outputs. Am I missing something stupidly easy?

Comment 10 kelwhite 2023-02-22 21:27:13 UTC

Well, I had the mon IP wrong and it seems you need to specify the user. Finally got the thing to mount:

# oc scale deployment -n openshift-storage rook-ceph-mgr-a --replicas=0
# oc debug node/<WORKER_NODE>
# chroot /host
# mkdir /mnt/cephfs
# mount -t ceph <MON_1_IP>:6789:/ /mnt/cephfs -o name=admin,secret=<CLIENT.ADMIN.SECRET>

You can get the MON_IP by running 
# ceph mon stat

You can get the CLIENT.ADMIN.SECRET by running:
# ceph auth get client.admin

sh-4.4# mount -t ceph 172.30.185.225:6789:/ /mnt/cephfs -o name=admin,secret=<redacted>
sh-4.4#

sh-4.4# mount |grep ceph
...
172.30.185.225:6789:/ on /var/mnt/cephfs type ceph (rw,relatime,seclabel,name=admin,secret=<hidden>,acl)

I had the mon ip wrong and didn't specify the name=admin :)

Comment 12 Venky Shankar 2023-02-28 04:47:00 UTC

Kotresh, PTAL.