Bug 1936897

Summary:	[Engine JDK 11] More verbose and configurable GC logging.
Product:	[oVirt] ovirt-engine	Reporter:	Artur Socha <asocha>
Component:	General	Assignee:	Artur Socha <asocha>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Guilherme Santos <gdeolive>
Severity:	low	Docs Contact:
Priority:	low
Version:	4.4.5.6	CC:	bugs, dfodor, mperina
Target Milestone:	ovirt-4.4.6	Keywords:	FutureFeature
Target Release:	4.4.6.1	Flags:	mperina: ovirt-4.4? pm-rhel: planning_ack? mperina: devel_ack+ gdeolive: testing_ack+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-4.6.6.1	Doc Type:	Enhancement
Doc Text:	Feature: More verbose and configurable GC logging. This includes: 1. GC verbose logging disabled by default with INFO verbosity level. 2. Ability to setup desired GC log level via config the property: ENGINE_VERBOSE_GC_LOG_LEVEL=info Available options are: off, error, warning, info, debug, trace 3. Ability to setup desired GC log file rotation via: ENGINE_VERBOSE_GC_LOG_FILE_SIZE=2M #single log file size ENGINE_VERBOSE_GC_LOG_FILES_NUMBER=50 #number of files in rotation Reason: oVirt Engine tends to allocate large chunks of memory (depending on overall setup) that might contribute to potential various slow downs caused by Java Garbage Collector operations. Result: The ability to monitor and discover memory related issues much earlier (before the issue is actually negatively impacting an environment) In order to customize or disable the above default settings it is recommended to put these properties in one of the engine's etc config ie. /etc/ovirt-engine/engine.conf.d/99-setup-gc-logging.conf	Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-05-06 12:14:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1916171

Description Artur Socha 2021-03-09 12:44:05 UTC

Description of problem:
After migration to java 11 some configuration for java GC logging was omitted. Additionally, GC related logs are sent to console.log together with application logs. For clarity they could be logged into dedicated (rotating) file ie. gc.log, plus logging level could configurable so that in various scenarios 'debug' or even 'trace' be available. The detault is 'info' 


How reproducible:
Enable GC verbose logging by setting up ENGINE_VERBOSE_GC=true

Actual results:
Deprecated jvm start options being used -XX:+PrintGCTimeStamps and -XX:+PrintGCDetails

Expected results:
-Xlog framework should be used

Comment 1 Artur Socha 2021-03-09 13:03:45 UTC

Gargabe Collector logging in INFO level enabled by default is not expected to have significant impact on overall performance because these events are aggregated and logged once in a while.

However, verification against performance test suite is highly recommended.

Comment 2 Guilherme Santos 2021-05-05 14:04:02 UTC

Verified on:
ovirt-engine-4.4.6.6-0.10.el8ev.noarch

Steps:
1. echo "ENGINE_VERBOSE_GC=true" >> /etc/ovirt-engine/engine.conf.d/10-setup-java.conf
2. systemctl restart ovirt-engine
3. cat /var/log/ovirt-engine/gc.log

Results:
gc.log file present with verbose GC information following the default parameters (lines number, etc)

Comment 3 Sandro Bonazzola 2021-05-06 12:14:12 UTC

This bugzilla is included in oVirt 4.4.6 release, published on May 4th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.6 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.