Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.
First Last Prev Next    This bug is not in your last search results.
Bug 1582411 - [CEE/SD][containers][ceph-ansible] RGW container does not start with "radosgw_civetweb_num_threads" configuration , set via option in all.yml, in ceph.conf as "rgw frontends = civetweb port=10.10.92.91:8080 num_threads=512 "
[CEE/SD][containers][ceph-ansible] RGW container does not start with "radosgw...
Status: CLOSED ERRATA
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Container (Show other bugs)
3.0
Unspecified Unspecified
medium Severity medium
: rc
: 3.1
Assigned To: Guillaume Abrioux
Shreekar
Aron Gunn
:
: 1631086 (view as bug list)
Depends On:
Blocks: 1572368 1584264
  Show dependency tree / graph
 
Reported: 2018-05-25 03:04 EDT by Tomas Petr
Modified: 2018-09-26 15:17 EDT (History)
11 users (show)

See Also:
Fixed In Version: RHEL: ceph-ansible-3.1.0-0.1.rc17.el7cp Ubuntu: ceph-ansible_3.1.0~rc17-2redhat1 rhceph:ceph-3.1-rhel-7-containers-candidate-38485-20180810211451
Doc Type: Bug Fix
Doc Text:
.The containerized Object Gateway daemon will read options from the Ceph configuration file now When launching the Object Gateway daemon in a container, the daemon would override any `rgw_frontends` options. This made it impossible to add extra options, such as, the `radosgw_civetweb_num_threads` option. In this release, the Object Gateway daemon will read options found in the Ceph configuration file, by default, `/etc/ceph/ceph.conf`.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-09-26 15:16:42 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


External Trackers
Tracker ID Priority Status Summary Last Updated
Github ceph/ceph-ansible/pull/2929 None None None 2018-07-24 12:32 EDT
Github ceph/ceph-ansible/pull/2949 None None None 2018-07-31 08:19 EDT
Github ceph/ceph-container/pull/1126 None None None 2018-07-24 12:33 EDT
Github ceph/ceph-container/pull/1148 None None None 2018-08-07 09:33 EDT
Red Hat Product Errata RHBA-2018:2820 None None None 2018-09-26 15:17 EDT

  None (edit)
Description Tomas Petr 2018-05-25 03:04:04 EDT 
Description of problem:
I am fighting with setting performance tuning for RGW in containers,  in all.yml you can set the tuning parameter "radosgw_civetweb_num_threads":
radosgw_civetweb_num_threads: 512 #default 100

in non-container env it is part of rgw_frontends variable, that is correct:
"rgw_frontends": "civetweb port=10.10.92.91:8080 num_threads=512",

In container environment rgw_frontends configuration from ceph.conf is overridden (or not used, I am not familiar with the process flow) by container start of RGW, as there is no  num_threads option
-----------------------------
# find / -name "start_rgw.sh"
/var/lib/docker/overlay2/439572f5c98fe1a2b438171a4cc834e13745676116f914f1ae888375f673570b/diff/start_rgw.sh
/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b/merged/start_rgw.sh

[root@rgws-1 ~]# grep num_threads 
/var/lib/docker/overlay2/439572f5c98fe1a2b438171a4cc834e13745676116f914f1ae888375f673570b/diff/start_rgw.sh:  local rgw_frontends="civetweb port=$RGW_CIVETWEB_IP:$RGW_CIVETWEB_PORT"

/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b/merged/start_rgw.sh:  local rgw_frontends="civetweb port=$RGW_CIVETWEB_IP:$RGW_CIVETWEB_PORT"
--------------------

[root@rgws-1 ~]# docker exec ceph-rgw-rgws-1 ceph --admin-daemon /var/run/ceph/ceph-client.rgw.rgws-1.asok config diff
{
    "diff": {
        "current": {
            "admin_socket": "/var/run/ceph/ceph-client.rgw.rgws-1.asok",
            "cluster_network": "192.168.1.0/28",
            "err_to_stderr": "true",
            "fsid": "d5a1e891-afca-4e6f-b2dc-e7c08db07dbf",
            "internal_safe_to_start_threads": "true",
            "keyring": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring",
            "log_max_recent": "10000",
            "mds_data": "/var/lib/ceph/mds/ceph-rgw.rgws-1",
            "mgr_data": "/var/lib/ceph/mgr/ceph-rgw.rgws-1",
            "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log",
            "mon_data": "/var/lib/ceph/mon/ceph-rgw.rgws-1",
            "mon_debug_dump_location": "/var/log/ceph/ceph-client.rgw.rgws-1.tdump",
            "mon_host": "10.10.93.0,10.10.92.94,10.10.93.107",
            "osd_data": "/var/lib/ceph/osd/ceph-rgw.rgws-1",
            "osd_journal": "/var/lib/ceph/osd/ceph-rgw.rgws-1/journal",
            "public_network": "10.10.92.0/22",
            "rgw_data": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1",
            "rgw_frontends": "civetweb port=10.10.92.91:8080",  <-----
            "rgw_thread_pool_size": "512",
            "setgroup": "ceph",
            "setuser": "ceph"


cat /etc/ceph/ceph.conf
[client.rgw.rgws-1]
host = rgws-1
keyring = /var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring
log file = /var/log/ceph/ceph-rgw-rgws-1.log
rgw frontends = civetweb port=10.10.92.91:8080 num_threads=512 <-----
rgw_thread_pool_size = 512
---------------------------

editing these files is the only way how to set this parameter in container environment:
local rgw_frontends="civetweb port=$RGW_CIVETWEB_IP:$RGW_CIVETWEB_PORT num_threads=512"

[root@rgws-1 ~]# docker exec ceph-rgw-rgws-1 ceph --admin-daemon /var/run/ceph/ceph-client.rgw.rgws-1.asok config diff
{
    "diff": {
        "current": {
            "admin_socket": "/var/run/ceph/ceph-client.rgw.rgws-1.asok",
            "cluster_network": "192.168.1.0/28",
            "err_to_stderr": "true",
            "fsid": "d5a1e891-afca-4e6f-b2dc-e7c08db07dbf",
            "internal_safe_to_start_threads": "true",
            "keyring": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring",
            "log_max_recent": "10000",
            "mds_data": "/var/lib/ceph/mds/ceph-rgw.rgws-1",
            "mgr_data": "/var/lib/ceph/mgr/ceph-rgw.rgws-1",
            "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log",
            "mon_data": "/var/lib/ceph/mon/ceph-rgw.rgws-1",
            "mon_debug_dump_location": "/var/log/ceph/ceph-client.rgw.rgws-1.tdump",
            "mon_host": "10.10.93.0,10.10.92.94,10.10.93.107",
            "osd_data": "/var/lib/ceph/osd/ceph-rgw.rgws-1",
            "osd_journal": "/var/lib/ceph/osd/ceph-rgw.rgws-1/journal",
            "public_network": "10.10.92.0/22",
            "rgw_data": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1",
            "rgw_frontends": "civetweb port=10.10.92.91:8080 num_threads=512",
            "rgw_thread_pool_size": "512",
            "setgroup": "ceph",
            "setuser": "ceph"


Version-Release number of selected component (if applicable):
ceph-radosgw-12.2.4-6.el7cp.x86_64
ceph-ansible-3.0.33-1.el7cp.noarch
docker-1.13.1-53.git774336d.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. deploy Ceph with site-docker.yml
2.  in all.yml you  set the tuning parameter "radosgw_civetweb_num_threads"
3.

Actual results:


Expected results:


Additional info:
Comment 3 Tomas Petr 2018-05-25 03:05:23 EDT 
[root@rgws-1 ~]# docker inspect ceph-rgw-rgws-1
[
    {
        "Id": "0c47b95650de6f68773d16d1f7c86e25a094acc7960d2123f8993876c707d37e",
        "Created": "2018-05-22T09:29:03.671062759Z",
        "Path": "/entrypoint.sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 11080,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2018-05-22T09:29:03.937237779Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:9f92f1dc96eccd12eda1e809a3539e58f83faad6289a21beb1a6ebac05b91f42",
        "ResolvConfPath": "/var/lib/docker/containers/0c47b95650de6f68773d16d1f7c86e25a094acc7960d2123f8993876c707d37e/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/0c47b95650de6f68773d16d1f7c86e25a094acc7960d2123f8993876c707d37e/hostname",
        "HostsPath": "/var/lib/docker/containers/0c47b95650de6f68773d16d1f7c86e25a094acc7960d2123f8993876c707d37e/hosts",
        "LogPath": "",
        "Name": "/ceph-rgw-rgws-1",
        "RestartCount": 0,
        "Driver": "overlay2",
        "MountLabel": "system_u:object_r:svirt_sandbox_file_t:s0:c110,c264",
        "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c110,c264",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/var/lib/ceph:/var/lib/ceph",
                "/etc/ceph:/etc/ceph",
                "/etc/localtime:/etc/localtime:ro"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "journald",
                "Config": {}
            },
            "NetworkMode": "host",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": true,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "docker-runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 2147483648,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 200000,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 4294967296,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "overlay2",
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b-init/diff:/var/lib/docker/overlay2/439572f5c98fe1a2b438171a4cc834e13745676116f914f1ae888375f673570b/diff:/var/lib/docker/overlay2/7c2723bc210e2197ee5434cb1c40f3f0d937a48a3df282d4b37c7833aeb59592/diff:/var/lib/docker/overlay2/833f5e514da7ef110d2eddf9a33ec26284fcd0a3f8d129abfe1a50f3a14cfcbf/diff",
                "MergedDir": "/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b/merged",
                "UpperDir": "/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b/diff",
                "WorkDir": "/var/lib/docker/overlay2/263d778874297c8fee362cdd06cad80924c2b8bb312e06ab3251d1df2555776b/work"
            }
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/var/lib/ceph",
                "Destination": "/var/lib/ceph",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/ceph",
                "Destination": "/etc/ceph",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/localtime",
                "Destination": "/etc/localtime",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "4493aa87c8cd704288bbcd01665d900156aa25ba99b95e0d5844faaa10b5e328",
                "Source": "/var/lib/docker/volumes/4493aa87c8cd704288bbcd01665d900156aa25ba99b95e0d5844faaa10b5e328/_data",
                "Destination": "/etc/ganesha",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "rgws-1.container.quicklab.rdu2.cee.redhat.com",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "5000/tcp": {},
                "6789/tcp": {},
                "6800/tcp": {},
                "6801/tcp": {},
                "6802/tcp": {},
                "6803/tcp": {},
                "6804/tcp": {},
                "6805/tcp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "RGW_CIVETWEB_IP=10.10.92.91",
                "CEPH_DAEMON=RGW",
                "CLUSTER=ceph",
                "RGW_CIVETWEB_PORT=8080",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "container=oci",
                "CEPH_VERSION=luminous",
                "CEPH_POINT_RELEASE="
            ],
            "Cmd": null,
            "ArgsEscaped": true,
            "Image": "registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest",
            "Volumes": {
                "/etc/ceph": {},
                "/etc/ganesha": {},
                "/var/lib/ceph": {}
            },
            "WorkingDir": "/",
            "Entrypoint": [
                "/entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "CEPH_POINT_RELEASE": "",
                "GIT_BRANCH": "master",
                "GIT_CLEAN": "True",
                "GIT_COMMIT": "99f689cd2c12f8332924db6a0cc0463bb26631b0",
                "GIT_REPO": "git@github.com:ceph/ceph-container.git",
                "RELEASE": "master",
                "architecture": "x86_64",
                "authoritative-source-url": "registry.access.redhat.com",
                "build-date": "2018-04-18T13:01:58.678631",
                "com.redhat.build-host": "ip-10-29-120-145.ec2.internal",
                "com.redhat.component": "rhceph-rhel7-docker",
                "description": "Red Hat Ceph Storage 3",
                "distribution-scope": "public",
                "install": "/usr/bin/docker run --rm --privileged -v /:/host -e MON_IP=${MON_IP}  -e CEPH_PUBLIC_NETWORK=${CEPH_PUBLIC_NETWORK} -e CEPH_DAEMON=${CEPH_DAEMON} -e MON_NAME=${MON_NAME} -e OSD_DEVICE=${OSD_DEVICE} -e HOST=/host -e IMAGE=${IMAGE} --entrypoint=/install.sh ${IMAGE}",
                "io.k8s.description": "Red Hat Ceph Storage 3",
                "io.k8s.display-name": "Red Hat Ceph Storage 3 on RHEL 7",
                "io.openshift.tags": "rhceph ceph",
                "name": "rhceph",
                "release": "6",
                "run": "/usr/bin/docker run -d --net=host --pid=host -e MON_NAME=${MON_NAME} -e MON_IP=${MON_IP}  -e CEPH_PUBLIC_NETWORK=${CEPH_PUBLIC_NETWORK} -e CEPH_DAEMON=${CEPH_DAEMON} -v /etc/ceph:/etc/ceph -v /var/lib/ceph:/var/lib/ceph ${IMAGE}",
                "summary": "Provides the latest Red Hat Ceph Storage 3 on RHEL 7 in a fully featured and supported base image.",
                "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/rhceph/images/3-6",
                "vcs-ref": "9fe91bb07dc2b866b3bd024bbaf43f09d4eb05e9",
                "vcs-type": "git",
                "vendor": "Red Hat, Inc.",
                "version": "3"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "dc6d0017cc78f7ddfa04bc35545e85f332c01211d26ae09a30be17d9d096ed63",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "6c222e50bc44578677a1cce4b250769a8b57bf3218590f831330117d56e8df49",
                    "EndpointID": "6d2d85251ba0509b150d7713bdfdf3f135a058024bf56670ad9a5f034c8c097c",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": ""
                }
            }
        }
    }
]
Comment 4 Vikhyat Umrao 2018-05-25 10:39:22 EDT 
Is this not same as - https://bugzilla.redhat.com/show_bug.cgi?id=1551637? Or I am missing something.
Comment 5 leseb 2018-06-03 22:30:24 EDT 
Vikhyat, it's different, it's another option.
Comment 6 Vikhyat Umrao 2018-06-04 08:58:03 EDT 
(In reply to leseb from comment #5)
> Vikhyat, it's different, it's another option.

Ahh right. It is with one particular configuration thanks for the feedback, leseb.
Comment 10 leseb 2018-07-31 08:14:46 EDT 
remote: *** Checking commit 6c1f7c9c2fcc5a65e63c0d4e4cb38c1a99780b9d
remote: *** Resolves:
remote: ***   Approved:
remote: ***     rhbz#1582411 (qa_ack?, ceph-3.0?, ceph-3.y?, pm_ack+, devel_ack?)
remote: *** Commit 6c1f7c9c2fcc5a65e63c0d4e4cb38c1a99780b9d allowed
remote: * Publishing information for 1 commits
To ssh://pkgs.devel.redhat.com/containers/rhceph-rhel7
Comment 21 leseb 2018-09-05 04:32:44 EDT 
Hi tomas, you should see it in the ceph.conf, can you share it so I can have a look?
Thanks
Comment 22 Tomas Petr 2018-09-05 04:35:57 EDT 
(In reply to leseb from comment #21)
> Hi tomas, you should see it in the ceph.conf, can you share it so I can have
> a look?
> Thanks

Hi Seb, yes it is in ceph.conf, no issue with that part:
[root@rgws-1 ~]# cat /etc/ceph/ceph.conf 
[client.rgw.rgws-0]
host = rgws-0
keyring = /var/lib/ceph/radosgw/ceph-rgw.rgws-0/keyring
log file = /var/log/ceph/ceph-rgw-rgws-0.log
rgw frontends = civetweb port=10.10.95.187:8080 num_threads=512

[client.rgw.rgws-1]
host = rgws-1
keyring = /var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring
log file = /var/log/ceph/ceph-rgw-rgws-1.log
rgw frontends = civetweb port=10.10.95.186:8080 num_threads=512  <--------
rgw_user_max_buckets = 666
rgw_zone = tpetrzone

[global]
cluster network = 192.168.1.0/28
debug_rgw = 0/1
fsid = 9f76a45a-acc3-4a08-b88b-5faaa0a067ab
log file = /dev/null
mon host = 10.10.95.184,10.10.95.17,10.10.95.185
public network = 10.10.92.0/22


but not in the runtime config:
[root@rgws-1 ~]# docker exec ceph-rgw-rgws-1 ceph daemon /var/run/ceph/ceph-client.rgw.rgws-1.asok config diff
{
    "diff": {
        "current": {
            "admin_socket": "/var/run/ceph/ceph-client.rgw.rgws-1.asok",
            "cluster_network": "192.168.1.0/28",
            "err_to_stderr": "true",
            "fsid": "9f76a45a-acc3-4a08-b88b-5faaa0a067ab",
            "internal_safe_to_start_threads": "true",
            "keyring": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring",
            "log_max_recent": "10000",
            "mds_data": "/var/lib/ceph/mds/ceph-rgw.rgws-1",
            "mgr_data": "/var/lib/ceph/mgr/ceph-rgw.rgws-1",
            "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log",
            "mon_data": "/var/lib/ceph/mon/ceph-rgw.rgws-1",
            "mon_debug_dump_location": "/var/log/ceph/ceph-client.rgw.rgws-1.tdump",
            "mon_host": "10.10.95.184,10.10.95.17,10.10.95.185",
            "osd_data": "/var/lib/ceph/osd/ceph-rgw.rgws-1",
            "osd_journal": "/var/lib/ceph/osd/ceph-rgw.rgws-1/journal",
            "public_network": "10.10.92.0/22",
            "rgw_data": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1",
            "rgw_frontends": "civetweb  port=10.10.95.186:8080", <--------
            "rgw_user_max_buckets": "666",
            "setgroup": "ceph",
            "setuser": "ceph"
Comment 23 leseb 2018-09-05 04:50:59 EDT 
Weird, do we need double quotes?
Comment 24 Tomas Petr 2018-09-05 05:00:49 EDT 
(In reply to leseb from comment #23)
> Weird, do we need double quotes?

This is how the civetweb num_threads is defined in all.yml:

./roles/ceph-defaults/defaults/main.yml:376:radosgw_civetweb_port: 8080
./roles/ceph-defaults/defaults/main.yml:377:radosgw_civetweb_num_threads: 100
./roles/ceph-defaults/defaults/main.yml:378:radosgw_civetweb_options: "num_threads={{ radosgw_civetweb_num_threads }}"

to put it in ceph.conf works, and from there is should be read by rgw process (in non container environment it is and works perfectly):
./roles/ceph-config/templates/ceph.conf.j2:185:rgw frontends = {{ radosgw_frontend_type }} {{ 'port' if radosgw_frontend_type == 'civetweb' else 'endpoint' }}=[{{ hostvars[host]['ansible_all_' + ip_version + '_addresses'] | ipaddr(radosgw_address_block) | first }}]:{{ radosgw_frontend_port }} {{ radosgw_frontend_options }}
./roles/ceph-config/templates/ceph.conf.j2:189:rgw frontends = {{ radosgw_frontend_type }} {{ 'port' if radosgw_frontend_type == 'civetweb' else 'endpoint' }}={{ radosgw_address_block | ipaddr(radosgw_address_block) | first }}:{{ radosgw_frontend_port }} {{ radosgw_frontend_options }}

so checking the same in non-container env would return: 
"rgw_frontends": "civetweb port=10.10.92.91:8080 num_threads=512", <--------

But how to force it into container env, I do not know.
Comment 25 leseb 2018-09-05 05:04:11 EDT 
The container also reads the ceph.conf now and is not forcing any CLI option, if you look at the rgw process in the container you will see we are not passing --rgw-frontend anymore.
Hence my question, can you try putting quotes around maybe?

Thanks.
Comment 26 Tomas Petr 2018-09-05 11:15:15 EDT 
(In reply to leseb from comment #25)
> The container also reads the ceph.conf now and is not forcing any CLI
> option, if you look at the rgw process in the container you will see we are
> not passing --rgw-frontend anymore.
> Hence my question, can you try putting quotes around maybe?
> 
> Thanks.

Hi Seb, it did not helped:
[root@rgws-1 ~]# docker exec ceph-rgw-rgws-1 cat /etc/ceph/ceph.conf |grep rgw
[client.rgw.rgws-0]
host = rgws-0
keyring = /var/lib/ceph/radosgw/ceph-rgw.rgws-0/keyring
log file = /var/log/ceph/ceph-rgw-rgws-0.log
rgw frontends = civetweb port=10.10.95.239:8080 num_threads=100
[client.rgw.rgws-1]
host = rgws-1
keyring = /var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring
log file = /var/log/ceph/ceph-rgw-rgws-1.log
rgw frontends = "civetweb port=10.10.95.238:8080 num_threads=100"
rgw_user_max_buckets = 666
rgw_zone = tpetrzone
debug_rgw = 0/1

[root@rgws-1 ~]# docker exec ceph-rgw-rgws-1 ceph daemon /var/run/ceph/ceph-client.rgw.rgws-1.asok config diff | grep rgw
            "admin_socket": "/var/run/ceph/ceph-client.rgw.rgws-1.asok",
            "keyring": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1/keyring",
            "mds_data": "/var/lib/ceph/mds/ceph-rgw.rgws-1",
            "mgr_data": "/var/lib/ceph/mgr/ceph-rgw.rgws-1",
            "mon_data": "/var/lib/ceph/mon/ceph-rgw.rgws-1",
            "mon_debug_dump_location": "/var/log/ceph/ceph-client.rgw.rgws-1.tdump",
            "osd_data": "/var/lib/ceph/osd/ceph-rgw.rgws-1",
            "osd_journal": "/var/lib/ceph/osd/ceph-rgw.rgws-1/journal",
            "rgw_data": "/var/lib/ceph/radosgw/ceph-rgw.rgws-1",
            "rgw_frontends": "civetweb port=10.10.95.238:8080",
            "rgw_user_max_buckets": "666",
Comment 32 Giulio Fidente 2018-09-26 08:42:37 EDT 
*** Bug 1631086 has been marked as a duplicate of this bug. ***
Comment 34 errata-xmlrpc 2018-09-26 15:16:42 EDT 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2820
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.