Bug 1710548

Summary: ceph container logging to respective daemon log file
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Vikhyat Umrao <vumrao>
Component: Ceph-AnsibleAssignee: Guillaume Abrioux <gabrioux>
Status: CLOSED ERRATA QA Contact: Vasishta <vashastr>
Severity: medium Docs Contact: Bara Ancincova <bancinco>
Priority: high    
Version: 3.2CC: aschoen, assingh, ceph-eng-bugs, ceph-qe-bugs, dparkes, dsavinea, edonnell, gabrioux, gmeno, gsitlani, jquinn, kdreyer, kjosy, mamccoma, mmanjuna, mmuench, nthomas, pdhange, roemerso, tchandra, tserlin, vashastr
Target Milestone: rc   
Target Release: 4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ceph-ansible-4.0.0-0.1.rc10.el8cp rhceph:ceph-4.0-rhel-8-containers-candidate-77859-20200121171255 Doc Type: Enhancement
Doc Text:
.Ceph container can now write logs to a respective daemon file The previous way of logging for containerized Ceph environments did not allow limiting the `journalctl` output when looking at log data by using the `sosreport` collection. With this release, logging can be enabled or disabled for a particular Ceph daemon with the following command: [subs='quotes'] ---- ceph config set _daemon_._id_ log_to_file true ---- Where _daemon_ is the type of the daemon and _id_ is its ID. For example, to enable logging for the Monitor daemon with ID `mon0`: ---- ceph config set mon.mon0 log_to_file true ---- This new feature makes debugging easier.
Story Points: ---
Clone Of:
: 1746570 (view as bug list) Environment:
Last Closed: 2020-01-31 12:46:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1730176, 1746570    

Description Vikhyat Umrao 2019-05-15 18:12:31 UTC
Description of problem:
ceph container logging to respective daemon 

Version-Release number of selected component (if applicable):
Red Hat Ceph Storage 3.2.z2

Comment 2 Dimitri Savineau 2019-05-30 18:36:12 UTC
*** Bug 1671484 has been marked as a duplicate of this bug. ***

Comment 11 Giridhar Ramaraju 2019-08-05 13:11:06 UTC
Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 12 Giridhar Ramaraju 2019-08-05 13:12:07 UTC
Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 15 Vasishta 2020-01-02 05:32:31 UTC
Hi Guillaume,

Need help, unable to set log_to_file as mentioned in doc text field.

my env - 
upgraded from ceph-3.3-rhel-7-containers-candidate-54404-20191212220346 to ceph-4.0-rhel-8-containers-candidate-52417-20191217061816 using ceph-ansible-4.0.6-1.el7cp.noarch


$ sudo docker exec ceph-mon-magna109 ceph config daemon mon.magna109 log_to_file true
no valid command found; 10 closest matches:
config show <who> {<key>}
config show-with-defaults <who>
config set <who> <name> <value> {--force}
config rm <who> <name>
config get <who> {<key>}
config dump
config help <key>
config ls
config assimilate-conf
config log {<int>}
Error EINVAL: invalid command

$ sudo docker exec ceph-mon-magna109 ceph config  show log_to_file
Error ENOENT: no config state for daemon log_to_file

$ sudo docker exec ceph-mon-magna109 ceph config get log_to_file
Error EINVAL: unrecognized entity 'log_to_file'


Can you please help me ?

Regards,
Vasishta Shastry
QE, Ceph

Comment 19 Vasishta 2020-01-12 12:27:50 UTC
Hi Guilluame, 

Thanks for the correction, I tried but not able to see log files for corresponding monitor

$ sudo docker exec ceph-mon-magna046 ceph daemon mon.magna046 config set log_to_file true
{
    "success": "log_to_file = 'true' "
}

$ ls /var/log/ceph
ceph-volume.log
$

Please let me know whether I'm missing something in my understanding/expectations.

(It was cluster upgraded from 3.3.x to 4.x latest build as on today)

Regards,
Vasishta Shastry
QE, Ceph

Comment 25 Vasishta 2020-01-17 06:24:44 UTC
Hi Guillaume, 

$ sudo docker exec ceph-mon-magna046 ceph config get mon.magna046 mon_cluster_log_to_file 
true

After your suggestion in Comment 24, I could see ceph.log and ceph.audit.log , but not mon log
$ ls /var/log/ceph
ceph.audit.log  ceph.log  ceph-volume.log

Comment 33 Vasishta 2020-01-22 07:01:14 UTC
# podman exec ceph-mon-magna049 ceph config get mon.magna049 mon_cluster_log_to_file
true
# podman exec ceph-mon-magna049 ceph config get mon.magna049 log_to_file
true
# ls /var/log/ceph
ceph-osd.10.log  ceph-osd.6.log  ceph-osd.9.log  ceph-volume.log

# sudo systemctl -l|grep ceph
ceph-mgr                                                                                                            loaded active running   Ceph Manager                                                                                                                
ceph-mon                                                                                                            loaded active running   Ceph Monitor                                                         


Still not able to see monitor/manager service log files
Moving BZ back to ASSIGNED state.

Comment 43 errata-xmlrpc 2020-01-31 12:46:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0312