Bug 1949149

Summary:	[RFE] Poor man's backtrace
Product:	Red Hat Enterprise Linux 8	Reporter:	Tomas Halman <thalman>
Component:	sssd	Assignee:	Alexey Tikhonov <atikhono>
Status:	CLOSED ERRATA	QA Contact:	Dan Lavu <dlavu>
Severity:	unspecified	Docs Contact:	Josip Vilicic <jvilicic>
Priority:	unspecified
Version:	8.4	CC:	atikhono, dlavu, grajaiya, jhrozek, jvilicic, lslebodn, mzidek, pasik, pbrezina, sgoveas, suwu, tscherf
Target Milestone:	beta	Keywords:	FutureFeature, Triaged
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	sync-to-jira
Fixed In Version:	sssd-2.5.0-1.el8	Doc Type:	Enhancement
Doc Text:	.SSSD now logs backtraces by default With this enhancement, SSSD now stores detailed debug logs in an in-memory buffer and appends them to log files when a failure occurs. By default, the following error levels trigger a backtrace: * Level 0: fatal failures * Level 1: critical failures * Level 2: serious failures You can modify this behavior for each SSSD process by setting the `debug_level` option in the corresponding section of the `sssd.conf` configuration file: * If you set the debugging level to 0, only level 0 events trigger a backtrace. * If you set the debugging level to 1, levels 0 and 1 trigger a backtrace. * If you set the debugging level to 2 or higher, events at level 0 through 2 trigger a backtrace. You can disable this feature per SSSD process by setting the `debug_backtrace_enabled` option to `false` in the corresponding section of `sssd.conf`: ---- [sssd] debug_backtrace_enabled = true debug_level=0 ... [nss] debug_backtrace_enabled = false ... [domain/idm.example.com] debug_backtrace_enabled = true debug_level=2 ... ... ----	Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-11-09 19:47:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tomas Halman 2021-04-13 14:38:32 UTC

Problem statement: when hard-to-reproduce issue occurs, we usually do not have any means to investigate the case, since default log level is rather low and doesn't have much information. And since issue is "hard-to-reproduce" it can take long time to gather logs with high level enabled (or might be impractical at all in high load environments).

Solution: in case SSSD is run with debug_level < full, log messages to a ring buffer in memory and flush the buffer to a log file on error (up to level 2).

Limitation: this will be only supported in case logger == files.

Comment 2 Alexey Tikhonov 2021-05-05 15:19:32 UTC

Pushed PR: https://github.com/SSSD/sssd/pull/5585

* `master`
    * 80963d683a0c7fd146fcfd21770990043b9d8449 - SBUS: changed debug level in sbus_issue_request_done() to avoid backtrace dump in case of 'ERR_MISSING_DP_TARGET'
    * 6fb987b5cf64675a42dfaaccc877ed867867bd73 - SERVER: decrease log level in `orderly_shutdown()` to avoid backtrace in this case.
    * f693078fe44ff0ae2f2c53180afaf532cfb92a1f - CERTMAP: removed "sss_certmap initialized" debug
    * 97f046e72bdec06356a5ed5295f283d0529eb440 - DEBUG: log IMPORTANT_INFO if any bit >= OP_FAILURE is on
    * 0aaf61c66b6f2758f89152f946360e00755a9846 - DEBUG: makes debug backtrace switchable
    * 6b78b7aa802529fc885877f326650fb7e6527607 - CACHE_REQ: fixed REVERSE_INULL warning
    * e3426ebeb52a821495ea5d34fdc408fe23df7416 - PAM: fixes a couple of covscan issues
    * 59ba14e5a70ed0b9253c7a881d664fcd28c337e7 - DEBUG: poor man's backtrace
    * f66b5aedab31f22642a50df8f7f458af8d3d7391 - DEBUG: got rid of most explicit DEBUG_IS_SET checks as a preliminary step for "logs backtrace" feature

Comment 16 errata-xmlrpc 2021-11-09 19:47:29 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sssd bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4435