Bug 1889711

Summary:	Prometheus metrics on disk take more space compared to OCP 4.5
Product:	OpenShift Container Platform	Reporter:	Simon Pasquier <spasquie>
Component:	Monitoring	Assignee:	Simon Pasquier <spasquie>
Status:	CLOSED ERRATA	QA Contact:	Junqi Zhao <juzhao>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.6	CC:	alegrand, anpicker, erooth, hongyli, juzhao, kakkoyun, lcosic, mloibl, pkrupa, sdodson, surbania, wking
Target Milestone:	---	Keywords:	Upgrades
Target Release:	4.6.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1889710	Environment:
Last Closed:	2020-11-30 16:45:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1889710
Bug Blocks:

Description Simon Pasquier 2020-10-20 12:47:57 UTC

+++ This bug was initially created as a clone of Bug #1889710 +++

Description of problem:
Timers in Go >= 1.14 introduced a degradation with the compression of data on disk (see https://github.com/prometheus/prometheus/pull/7976 for details but users reported that block sizes increased up to 50% compared to previous versions). Prometheus for OCP 4.5 (and before) are built with Go 1.13 so the only affected version is OCP 4.6.

Version-Release number of selected component (if applicable):
4.6

How reproducible:
Always

Steps to Reproduce:
Run Prometheus at least 4 hours and measure the sample compression with:

rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[8h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[8h])

Actual results:
More than 2 bytes on disk per sample.

Expected results:
Between 1 and 2 bytes on disk per sample.

Additional info:
See https://github.com/prometheus/prometheus/issues/7846

--- Additional comment from Simon Pasquier on 2020-10-20 12:45:35 UTC ---

Fixed in https://github.com/openshift/prometheus/pull/61 by bumping Prometheus to v2.22.0.

Comment 6 hongyan li 2020-11-23 03:24:10 UTC

already test with unmerged pr

Comment 9 errata-xmlrpc 2020-11-30 16:45:31 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.6 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5115