Description of problem: OCP Logging runs as a container and the latest currently supported version is based on RHEL 8 UBI and thus collects logs using `journalctl` from 8.6/8.7. systemd-journald in 9.2 cannot be configured to be compatible with journalctl from 8.6/8.7 without completely disabling compression. Version-Release number of selected component (if applicable): systemd-252 How reproducible: 100% Steps to Reproduce: 1. Install RHEL 9.2 2. Run journalctl from within an 8.x container 3. Actual results: systemd-journald 252 cannot be configured to enable compatibility with 8.x journalctl without fully disabling compression which would significantly increase disk utilization and is undesirable. Expected results: systemd-journald 252 can be configured or defaults to LZ4 compression and can be configured such that 8.x journalctl can read the on disk format. We know that this will require the existing environment variables. Environment="SYSTEMD_JOURNAL_COMPACT=0" Environment="SYSTEMD_JOURNAL_KEYED_HASH=0" Additional info: We're requesting this as a transitional state, RHEL 9.3 and later can revert back to ZSTD being the only supported compression. By the time OCP uses 9.3 or later for RHCOS we expect all containers to have moved to UBI9.
I typed this up, it compiles, haven't tested it: diff --git a/src/libsystemd/sd-journal/journal-file.c b/src/libsystemd/sd-journal/journal-file.c index 8da4ca9cad..2fcc9157ae 100644 --- a/src/libsystemd/sd-journal/journal-file.c +++ b/src/libsystemd/sd-journal/journal-file.c @@ -336,6 +336,22 @@ static bool compact_mode_requested(void) { return true; } +static Compression compression_requested(void) { + const char *alg; + Compression ret; + + alg = getenv("SYSTEMD_JOURNAL_COMPRESS"); + if (alg == NULL) + return DEFAULT_COMPRESSION; + + ret = compression_from_string(alg); + if (ret == _COMPRESSION_INVALID) { + log_debug("Failed to parse SYSTEMD_JOURNAL_COMPRESS value, ignoring: %s", alg); + return DEFAULT_COMPRESSION; + } + return ret; +} + static int journal_file_init_header( JournalFile *f, JournalFileFlags file_flags, @@ -1626,7 +1642,7 @@ static Compression maybe_compress_payload(JournalFile *f, uint8_t *dst, const ui #if HAVE_COMPRESSION if (JOURNAL_FILE_COMPRESS(f) && size >= f->compress_threshold_bytes) { - compression = compress_blob(src, size, dst, size - 1, rsize); + compression = compress_blob_explicit(f->compression, src, size, dst, size - 1, rsize); if (compression > 0) log_debug("Compressed data object %"PRIu64" -> %zu using %s", size, *rsize, compression_to_string(compression)); @@ -3785,6 +3801,7 @@ int journal_file_open( .fd = fd, .mode = mode, .open_flags = open_flags, + .compression = compression_requested(), .compress_threshold_bytes = compress_threshold_bytes == UINT64_MAX ? DEFAULT_COMPRESS_THRESHOLD : MAX(MIN_COMPRESS_THRESHOLD, compress_threshold_bytes), diff --git a/src/libsystemd/sd-journal/journal-file.h b/src/libsystemd/sd-journal/journal-file.h index 70d2276ced..a14f4ea51f 100644 --- a/src/libsystemd/sd-journal/journal-file.h +++ b/src/libsystemd/sd-journal/journal-file.h @@ -100,6 +100,7 @@ typedef struct JournalFile { unsigned last_seen_generation; + Compression compression; uint64_t compress_threshold_bytes; #if HAVE_COMPRESSION void *compress_buffer;
(In reply to Colin Walters from comment #3) > I typed this up, it compiles, haven't tested it: This looks like the right way. And it should probably be done upstream, as it looks like an omission. I.e., we have env. vars to turn off other incompatible features, but none to force a particular compression algorithm. (That said, I'm pretty sure the patch in its current form doesn't work. In particular, setting of .incompatible_flags in journal_file_init_header() needs to be updated too.)
> This looks like the right way. And it should probably be done upstream, as it looks like an omission. I.e., we have env. vars to turn off other incompatible features, but none to force a particular compression algorithm. Right, agree. Would someone mind taking this ball and doing a PR upstream? I have to chase a different bug. > (That said, I'm pretty sure the patch in its current form doesn't work. In particular, setting of .incompatible_flags in journal_file_init_header() needs to be updated too.) Ah yes, indeed.
Upstream patch: https://github.com/systemd/systemd/pull/27126
fix merged to github main branch -> https://github.com/redhat-plumbers/systemd-rhel9/pull/159