Bug 2014500 - beast not working on IPv6 failed to parse endpoint=fd00:fd00:fd00:3000::397:808
Summary: beast not working on IPv6 failed to parse endpoint=fd00:fd00:fd00:3000::397:808
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Cephadm
Version: 5.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 5.1
Assignee: Adam King
QA Contact: Sunil Kumar Nagaraju
Ranjini M N
URL:
Whiteboard:
Depends On:
Blocks: 1820257 2031073
TreeView+ depends on / blocked
 
Reported: 2021-10-15 12:10 UTC by John Fulton
Modified: 2022-04-04 10:22 UTC (History)
15 users (show)

Fixed In Version: ceph-16.2.6-41.el8cp
Doc Type: Enhancement
Doc Text:
.`cephadm` now supports IPv6 networks for Ceph Object Gateway deployment With this release, `cephadm` supports specifying an IPv6 network for Ceph Object Gateway specifications. An example of a service configuration file for deploying Ceph Object Gateway is: .Example ---- service_type: rgw service_id: rgw placement: count: 3 networks: - fd00:fd00:3000::/64 ----
Clone Of:
Environment:
Last Closed: 2022-04-04 10:22:04 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph pull 43602 0 None open mgr/cephadm: Fix RGW ipv6 frontend configuration 2021-10-20 13:42:20 UTC
Red Hat Issue Tracker RHCEPH-2056 0 None None None 2021-10-15 12:16:31 UTC
Red Hat Product Errata RHSA-2022:1174 0 None None None 2022-04-04 10:22:28 UTC

Description John Fulton 2021-10-15 12:10:40 UTC
Description of problem:

Beast as deployed by cephadm is not starting with IPv6:

Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.232+0000 7febbcaf1480  0 starting handler: beast                                                                      
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.234+0000 7febbcaf1480 -1 failed to parse endpoint=fd00:fd00:fd00:3000::397:8080                                       
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.234+0000 7febbcaf1480 -1 ERROR: failed initializing frontend

Similar to BZ 1644840

Version-Release number of selected component (if applicable): 

- ceph-16.2.0-130.el8cp (FIV of BZ 2002639)
- ceph container 5-14

How reproducible: deterministic

Steps to Reproduce:
1. Deploy with cephadm and observe working mon,mgr,osd
2. Apply spec [1] for RGW 
3. Observe in `ceph orch ls` that the RGW service is not running 
4. Try to start RGW w/ systemctl start [2]
5. Observe error in journalctl logs [3]

Actual results:

 [ceph: root@controller-0 /]# ceph orch ls
NAME                     RUNNING  REFRESHED  AGE  PLACEMENT                              
crash                        7/7  7m ago     14h  *                                      
mgr                          3/3  7m ago     14h  controller-0;controller-1;controller-2  
mon                          3/3  7m ago     14h  controller-0;controller-1;controller-2  
osd.default_drive_group      4/8  7m ago     14h  ceph-0;ceph-1;ceph-2;ceph-3            
rgw.rgw                      0/3  7m ago     14h  controller-0;controller-1;controller-2  
[ceph: root@controller-0 /]#

Expected results:
rgw.rgw                      3/3  7m ago     14h  controller-0;controller-1;controller-2  


Additional info:

As per https://tracker.ceph.com/issues/52867 we needed to set an additional setting for OSD to work on IPv6. Is there a similar setting required for RGW? As per 1644840, I'd think this bug is fixed and but maybe I'm missing config option?


Footnotes:

[1] Spec for RGW
"""
---
service_type: rgw
service_id: rgw
service_name: rgw.rgw
placement:
  hosts:
  - controller-0
  - controller-1
  - controller-2
networks:
- fd00:fd00:fd00:3000::/64
spec:
  rgw_frontend_port: 8080
  rgw_realm: default
  rgw_zone: default
"""

[2] [root@controller-0 ceph]# systemctl start ceph-8552069d-fddd-4d77-81e5-9e07cf282f81.controller-0.glbqfs.service

[3] [root@controller-0 ceph]# journalctl -xef -u ceph-8552069d-fddd-4d77-81e5-9e07cf282f81.controller-0.glbqfs.service
...
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.050+0000 7febbcaf1480  0 framework: beast                                                                             
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.050+0000 7febbcaf1480  0 framework conf key: endpoint, val: fd00:fd00:fd00:3000::397:8080                             
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.050+0000 7febbcaf1480  1 radosgw_Main not setting numa affinity                                                       
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.227+0000 7febbcaf1480  0 framework: beast                                                                             
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.227+0000 7febbcaf1480  0 framework conf key: ssl_certificate, val: config://rgw/cert/$realm/$zone.crt                 
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.227+0000 7febbcaf1480  0 framework conf key: ssl_private_key, val: config://rgw/cert/$realm/$zone.key                 
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.227+0000 7feb7eed3700  0 RGWReshardLock::lock failed to acquire lock on reshard.0000000001 ret=-16                    
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.232+0000 7febbcaf1480  0 starting handler: beast                                                                      
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.234+0000 7febbcaf1480 -1 failed to parse endpoint=fd00:fd00:fd00:3000::397:8080                                       
Oct 15 11:47:43 controller-0 conmon[477168]: debug 2021-10-15T11:47:43.234+0000 7febbcaf1480 -1 ERROR: failed initializing frontend                                                          
Oct 15 11:47:43 controller-0 podman[477332]: 2021-10-15 11:47:43.378281038 +0000 UTC m=+0.101282259 container died e5fbe91a7d53a857457556e600b2356aef642442808df45f19ef6d42ff9d7667 (image=und
ercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhceph@sha256:55cb1de88341300daa1ab6d59e4897edc733a3f90162149c21f18abe49ed87c7, name=ceph-8552069d-fddd-4d77-81e5-9e07cf282f81-rgw-rgw-controller
-0-glbqfs)

Comment 1 RHEL Program Management 2021-10-15 12:10:47 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.

Comment 2 Casey Bodley 2021-10-15 13:36:27 UTC
that parser expects v6 addrs to be in square brackets. have you tried [fd00:fd00:fd00:3000::397:8080]?

Comment 4 Sebastian Wagner 2021-10-20 13:42:21 UTC
https://github.com/ceph/ceph/pull/43602

Comment 12 Sebastian Wagner 2021-12-23 09:32:11 UTC
Could you dump the config for the RGW frontend?

Comment 25 errata-xmlrpc 2022-04-04 10:22:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 5.1 Security, Enhancement, and Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1174


Note You need to log in before you can comment on or make changes to this bug.