Bug 2186142

Summary: [RHCS 6.1] [Deployment] Cephadm bootstrap failing with default image.
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Mohit Bisht <mobisht>
Component: ContainerAssignee: Guillaume Abrioux <gabrioux>
Status: CLOSED ERRATA QA Contact: Mohit Bisht <mobisht>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 6.1CC: adking, ceph-eng-bugs, cephqe-warriors, kdreyer, msaini, tserlin, vdas, vimishra
Target Milestone: ---   
Target Release: 6.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rhceph:6-129 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-06-15 16:01:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mohit Bisht 2023-04-12 08:24:22 UTC
Description of problem:
========================
[RHCS 6.1] [Deployment] Cephadm bootstrap failing with default image.

# rpm -qa|grep cephadm
cephadm-17.2.6-13.el9cp.noarch

cephadm bootstrap --mon-ip <mon_ip> --registry-url registry.redhat.io  --registry-username <username> --registry-password <password>
orch apply ceph-exporter
/usr/bin/ceph: stderr Error EINVAL: Usage:
/usr/bin/ceph: stderr   ceph orch apply -i <yaml spec> [--dry-run]
/usr/bin/ceph: stderr   ceph orch apply <service_type> [--placement=<placement_string>] [--unmanaged]
/usr/bin/ceph: stderr         
Traceback (most recent call last):
  File "/usr/sbin/cephadm", line 9657, in <module>
    main()
  File "/usr/sbin/cephadm", line 9645, in main
    r = ctx.func(ctx)
  File "/usr/sbin/cephadm", line 2205, in _default_image
    return func(ctx)
  File "/usr/sbin/cephadm", line 5774, in command_bootstrap
    prepare_ssh(ctx, cli, wait_for_mgr_restart)
  File "/usr/sbin/cephadm", line 5275, in prepare_ssh
    cli(['orch', 'apply', t])
  File "/usr/sbin/cephadm", line 5708, in cli
    return CephContainer(
  File "/usr/sbin/cephadm", line 4144, in run
    out, _, _ = call_throws(self.ctx, self.run_cmd(),
  File "/usr/sbin/cephadm", line 1853, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --authfile=/etc/ceph/podman-auth.json --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=registry.redhat.io/rhceph/rhceph-6-rhel9:latest -e NODE_NAME=mobisht-test-6-1-rhel9 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/c94e1446-d906-11ed-b5fe-fa163e194455:/var/log/ceph:z -v /tmp/ceph-tmpia99o5e0:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpfon75m7z:/etc/ceph/ceph.conf:z registry.redhat.io/rhceph/rhceph-6-rhel9:latest orch apply ceph-exporte

Workaround
==========
Passed with custom image using --image option
cephadm --image registry-proxy.engineering.redhat.com/rh-osbs/rhceph:6-118 bootstrap --mon-ip <mon_ip> --registry-username <username> --registry-password <password>
sudo /usr/sbin/cephadm shell --fsid 2e7d7b76-d836-11ed-8918-fa163e49aef7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring

Or, if you are only running a single cluster on this host:

    sudo /usr/sbin/cephadm shell 

Please consider enabling telemetry to help improve Ceph:

    ceph telemetry on

For more information see:

    https://docs.ceph.com/docs/master/mgr/telemetry/

Bootstrap complete.

ceph versions
{
    "mon": {
        "ceph version 17.2.6-4.el9cp (95c32f374bd8f94d27c87e8293f9f549e379c9f4) quincy (stable)": 1
    },
    "mgr": {
        "ceph version 17.2.6-4.el9cp (95c32f374bd8f94d27c87e8293f9f549e379c9f4) quincy (stable)": 1
    },
    "osd": {},
    "mds": {},
    "overall": {
        "ceph version 17.2.6-4.el9cp (95c32f374bd8f94d27c87e8293f9f549e379c9f4) quincy (stable)": 2
    }
}


Version-Release number of selected component (if applicable):
=============================================================
cephadm-17.2.6-13.el9cp.noarch

How reproducible:
=================
Always

Steps to Reproduce:
===================
1. Install cephadm (cephadm-17.2.6-13.el9cp.noarch)
2. Bootstrap cluster with defult image.
cephadm bootstrap --mon-ip <mon_ip> --registry-url registry.redhat.io  --registry-username <username> --registry-password <password>

Actual results:
===============
Bootstrap failed

Expected results:
=================
Bootstrap should pass

Comment 4 Manisha Saini 2023-04-12 13:33:19 UTC
*** Bug 2186151 has been marked as a duplicate of this bug. ***

Comment 15 Manisha Saini 2023-04-17 04:22:19 UTC
Clearing the needinfo as Mohit aready answered in comment #10

Comment 18 errata-xmlrpc 2023-06-15 16:01:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 6.1 Container security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:3642