Bug 2231834 - [4.13.z clone][Multus] rbd command hung in toolbox pod on Multus enabled OCS cluster
Summary: [4.13.z clone][Multus] rbd command hung in toolbox pod on Multus enabled OCS ...
Keywords:
Status: POST
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: ocs-operator
Version: 4.13
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Nikhil Ladha
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-08-14 11:02 UTC by Nikhil Ladha
Modified: 2023-08-17 10:35 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)

Description Nikhil Ladha 2023-08-14 11:02:39 UTC
This bug was initially created as a copy of Bug #1982721


Description of problem (please be detailed as possible and provide log
snippests):
In a Multus enabled OCS Internal mode cluster, all commands are not working as expected in toolbox pod.
Simple ceph commands works, but rbd commands get hung.

```
sh-4.4# ceph -s
  cluster:
    id:     465f536e-3fed-41ec-88c5-00584ad4f069
    health: HEALTH_OK
 
  services:
    mon: 3 daemons, quorum a,b,c (age 6h)
    mgr: a(active, since 6h)
    mds: ocs-storagecluster-cephfilesystem:1 {0=ocs-storagecluster-cephfilesystem-a=up:active} 1 up:standby-replay
    osd: 3 osds: 3 up (since 6h), 3 in (since 6h)
    rgw: 1 daemon active (ocs.storagecluster.cephobjectstore.a)
 
  data:
    pools:   10 pools, 176 pgs
    objects: 351 objects, 204 MiB
    usage:   3.5 GiB used, 1.5 TiB / 1.5 TiB avail
    pgs:     176 active+clean
 
  io:
    client:   852 B/s rd, 11 KiB/s wr, 1 op/s rd, 1 op/s wr
 
sh-4.4# rbd ls -p ocs-storagecluster-cephblockpool
>> hangs & no output after several minutes

```

It was observed that toolbox pod doesn't have Multus related annotaions/interfaces by default.
Operator should take care of this automatically and apply proper multus annotations to the toolbox.

Version of all relevant components (if applicable):
OCP: 4.8.0-0.nightly-2021-07-14-153019
OCS: ocs-operator.v4.8.0-452.ci

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
User won't be able to execute all commands via toolbox

Is there any workaround available to the best of your knowledge?
Yes,
Add annotation to "rook-ceph-tools" deployment similar to what MONs have
 and set hostNetwork to false.

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:
-

Steps to Reproduce:
1. Install OCS operator
2. Create Storage Cluster with Multus
3. Use this command to start toolbox pod
$ oc patch ocsinitialization ocsinit -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/enableCephTools", "value": true }]'

4. Run a simple rbd command
$ rbd ls -p ocs-storagecluster-cephblockpool

Actual results:
rbd command get hung in toolbox pod

Expected results:
all ceph & rbd commands should work same with and without multus

Additional info:


Note You need to log in before you can comment on or make changes to this bug.