Bug 1827927 - Upgrade libarchive to 3.3.3
Summary: Upgrade libarchive to 3.3.3
Keywords:
Status: POST
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: libarchive
Version: 8.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: Ondrej Dubaj
QA Contact: RHEL CS Apps Subsystem QE
URL:
Whiteboard:
Depends On:
Blocks: 1816874
TreeView+ depends on / blocked
 
Reported: 2020-04-25 16:13 UTC by Denis Arnaud
Modified: 2020-09-25 13:59 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Enhancement
Target Upstream Version:


Attachments (Terms of Use)

Description Denis Arnaud 2020-04-25 16:13:02 UTC
Description of problem:
libarchive should be upgraded to 3.4.2, as it is a minor upgrade with no SO bump (no ABI issue expected).

Newer versions of libarchive are required for instance for (newer versions of) CMake (e.g., 3.17, like on Fedora Rawhide/33). I indeed intend to package CMake3.17 for EPEL 8. In case you wonder, I started to package libarchive3.4, but figured out that it triggered a conflict on the shared library because the SO version is the same for both versions of libarchive.

On RHEL/CentOS 8, libarchive version is 3.3.2. For both versions of libarchive, the SO version is 13.

Version-Release number of selected component (if applicable):
3.3.2

How reproducible:
Always

Steps to Reproduce:
1. Search for libarchive on RedHat/CentOS 8
2.
3.

Actual results:
The packaged version is 3.3.2. It prevents CMake 3.17 to be built on  RedHat/CentOS 8.

Expected results:
The packaged version should be 3.4.2, so that CMake 3.17 can be built on RedHat/CentOS 8.


Additional info:

Comment 2 Pavel Raiskup 2020-04-25 17:44:21 UTC
> Newer versions of libarchive are required for instance for (newer versions of) CMake

Can you point us to reasons why such new libarchive is needed for CMmake?

Comment 3 Denis Arnaud 2020-04-25 17:59:16 UTC
(In reply to Pavel Raiskup from comment #2)
> > Newer versions of libarchive are required for instance for (newer versions of) CMake
> 
> Can you point us to reasons why such new libarchive is needed for CMmake?

Fedora 30+ (30, 31, 32 and Rawhide) comes with CMake 3.17.1: https://src.fedoraproject.org/rpms/cmake
When trying to build that version on RedHat/CentOS 8, it fails, only because the version of libarchive is too low.
Following is an attempt to build CMake on RedHat/CentOS 8: https://koji.fedoraproject.org/koji/taskinfo?taskID=43778148
See the build log: https://koji.fedoraproject.org/koji/getfile?taskID=43778192&volume=DEFAULT&name=build.log&offset=-4000

Version 3.3.3 of libarchive could probably be good enough (and, by the way, it is a security update: https://github.com/libarchive/libarchive/releases/tag/v3.3.3). But since 3.4.2 provides the same SO version, it makes sense to upgrade to that latest version of libarchive.

And, if you ask why CMake 3.17, then it is another question. In short, CMake 3.17 provides much better support for Python 3 (among other things). As you may know, Python 2 has reached end of life; making sure existing EPEL 8 packages properly support Python 3 is therefore required. And CMake 3.17 is needed for that. CMake 3.11 (current version on RedHat/CentOS 8) is outdated.

Comment 4 Pavel Raiskup 2020-04-25 18:21:01 UTC
Red Hat tracks security issues, and updates - when appropriate.

o if cmake depends on 3.3.3 only because of security problems (not
features) - perhaps it is enough to down-bump the requirement to 3.3.2
from 3.3.3 in CMakeLists.txt.

Comment 5 Denis Arnaud 2020-04-25 19:06:31 UTC
(In reply to Pavel Raiskup from comment #4)
> Red Hat tracks security issues, and updates - when appropriate.
> 
> o if cmake depends on 3.3.3 only because of security problems (not
> features) - perhaps it is enough to down-bump the requirement to 3.3.2
> from 3.3.3 in CMakeLists.txt.

Well, the issue is to get newer versions of CMake than 3.11 (as shipped on RHEL/CentOS 8 currently). And these newer versions, in turn, require libarchive 3.3.3+, which happens to be a security update too.
So, it should be straightforward to upgrade libarchive on RHEL/CentOS 8 at leat to 3.3.3, and even to 3.4.2, as it doesn't break ABI compatibility.
So, win-win-win: upgrade of libarchive on RHEL/CentOS 8 - better security compliance - allowing to build newer versions of CMake

Comment 7 Pavel Raiskup 2020-04-27 12:53:39 UTC
The problem here is that new CMake requires zstd support in libarchive,
that was added in (not only security release) 3.3.3 [1].

[1] https://github.com/libarchive/libarchive/releases/tag/v3.3.3

Comment 9 Denis Arnaud 2020-04-27 19:51:41 UTC
CMake3 3.17.1 seems to be available on (at least) EPEL 7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-b6a455afdf
Wondering how they worked around the issue with zstd support

Comment 10 Tom Stellard 2020-04-27 19:56:39 UTC
It looks like they bundled libarchive: https://src.fedoraproject.org/rpms/cmake3/c/8a3ce1f213d6b856c3621b456ca34216267f5caf?branch=epel7

Comment 11 Ondrej Dubaj 2020-04-30 09:42:06 UTC
Pasting ABI diff of versions 3.3.3 and 3.4.2. We need to investigate the changes, but from first sight it seems 3.3.2 and 3.4.2 are not 100% compatible, as there are offset changes in structure.


abipkgdiff --d1 libarchive-debuginfo-3.3.2-9.el8.x86_64.rpm --d2 libarchive-debuginfo-3.4.2-1.el8.x86_64.rpm libarchive-3.3.2-9.el8.x86_64.rpm libarchive-3.4.2-1.el8.x86_64.rpm 
================ changes of 'libarchive.so.13.3.2'===============
  Functions changes summary: 2 Removed, 4 Changed (186 filtered out), 32 Added functions
  Variables changes summary: 0 Removed, 1 Changed, 2 Added variables

  2 Removed functions:

    'function int __archive_write_close_filter(archive_write_filter*)'    {__archive_write_close_filter}
    'function int __archive_write_open_filter(archive_write_filter*)'    {__archive_write_open_filter}

  32 Added functions:

    'function Bool Ppmd8_Alloc(CPpmd8*, UInt32)'    {Ppmd8_Alloc}
    'function void Ppmd8_Construct(CPpmd8*)'    {Ppmd8_Construct}
    'function int Ppmd8_DecodeSymbol(CPpmd8*)'    {Ppmd8_DecodeSymbol}
    'function void Ppmd8_Free(CPpmd8*)'    {Ppmd8_Free}
    'function void Ppmd8_Init(CPpmd8*, unsigned int, unsigned int)'    {Ppmd8_Init}
    'function CPpmd_See* Ppmd8_MakeEscFreq(CPpmd8*, unsigned int, UInt32*)'    {Ppmd8_MakeEscFreq}
    'function Bool Ppmd8_RangeDec_Init(CPpmd8*)'    {Ppmd8_RangeDec_Init}
    'function void Ppmd8_Update1(CPpmd8*)'    {Ppmd8_Update1}
    'function void Ppmd8_Update1_0(CPpmd8*)'    {Ppmd8_Update1_0}
    'function void Ppmd8_Update2(CPpmd8*)'    {Ppmd8_Update2}
    'function void Ppmd8_UpdateBin(CPpmd8*)'    {Ppmd8_UpdateBin}
    'function int __archive_mkstemp(char*)'    {__archive_mkstemp}
    'function int __archive_read_header(archive_read*, archive_entry*)'    {__archive_read_header}
    'function void __archive_write_entry_filetype_unsupported(archive*, archive_entry*, const char*)'    {__archive_write_entry_filetype_unsupported}
    'function void archive_entry_set_symlink_type(archive_entry*, int)'    {archive_entry_set_symlink_type}
    'function int archive_entry_symlink_type(archive_entry*)'    {archive_entry_symlink_type}
    'function const char* archive_libzstd_version()'    {archive_libzstd_version}
    'function int archive_match_set_inclusion_recursion(archive*, int)'    {archive_match_set_inclusion_recursion}
    'function int archive_read_support_filter_zstd(archive*)'    {archive_read_support_filter_zstd}
    'function int archive_read_support_format_rar5(archive*)'    {archive_read_support_format_rar5}
    'function int archive_write_add_filter_zstd(archive*)'    {archive_write_add_filter_zstd}
    'function int blake2s(void*, size_t, void*, size_t, void*, size_t)'    {blake2s}
    'function int blake2s_final(blake2s_state*, void*, size_t)'    {blake2s_final}
    'function int blake2s_init(blake2s_state*, size_t)'    {blake2s_init}
    'function int blake2s_init_key(blake2s_state*, size_t, void*, size_t)'    {blake2s_init_key}
    'function int blake2s_init_param(blake2s_state*, const blake2s_param*)'    {blake2s_init_param}
    'function int blake2s_update(blake2s_state*, void*, size_t)'    {blake2s_update}
    'function int blake2sp(void*, size_t, void*, size_t, void*, size_t)'    {blake2sp}
    'function int blake2sp_final(blake2sp_state*, void*, size_t)'    {blake2sp_final}
    'function int blake2sp_init(blake2sp_state*, size_t)'    {blake2sp_init}
    'function int blake2sp_init_key(blake2sp_state*, size_t, void*, size_t)'    {blake2sp_init_key}
    'function int blake2sp_update(blake2sp_state*, void*, size_t)'    {blake2sp_update}

  4 functions with some indirect sub-type change:

    [C]'function void* __archive_read_ahead(archive_read*, size_t, ssize_t*)' at archive_read.c:1315:1 has some indirect sub-type changes:
      parameter 1 of type 'archive_read*' has sub-type changes:
        in pointed to type 'struct archive_read' at archive_read_private.h:161:1:
          type size hasn't changed
          1 data member changes (4 filtered):
           type of 'archive_read_filter_bidder archive_read::bidders[16]' changed:
             array element type 'struct archive_read_filter_bidder' changed: 
               type size hasn't changed
               1 data member changes (3 filtered):
                type of 'int (archive_read_filter_bidder*, archive_read_filter*)* archive_read_filter_bidder::bid' changed:
                  in pointed to type 'function type int (archive_read_filter_bidder*, archive_read_filter*)':
                    parameter 2 of type 'archive_read_filter*' has sub-type changes:
                      in pointed to type 'struct archive_read_filter' at archive_read_private.h:82:1:
                        type size changed from 1408 to 1472 (in bits)
                        1 data member insertion:
                          'int (archive_read_filter*, archive_entry*)* archive_read_filter::read_header', at offset 640 (in bits) at archive_read_private.h:102:1
                        14 data member changes (9 filtered):
                         'void* archive_read_filter::data' offset changed from 640 to 704 (in bits) (by +64 bits)
                         'const char* archive_read_filter::name' offset changed from 704 to 768 (in bits) (by +64 bits)
                         'int archive_read_filter::code' offset changed from 768 to 832 (in bits) (by +64 bits)
                         'char* archive_read_filter::buffer' offset changed from 832 to 896 (in bits) (by +64 bits)
                         'size_t archive_read_filter::buffer_size' offset changed from 896 to 960 (in bits) (by +64 bits)
                         'char* archive_read_filter::next' offset changed from 960 to 1024 (in bits) (by +64 bits)
                         'size_t archive_read_filter::avail' offset changed from 1024 to 1088 (in bits) (by +64 bits)
                         'void* archive_read_filter::client_buff' offset changed from 1088 to 1152 (in bits) (by +64 bits)
                         'size_t archive_read_filter::client_total' offset changed from 1152 to 1216 (in bits) (by +64 bits)
                         'const char* archive_read_filter::client_next' offset changed from 1216 to 1280 (in bits) (by +64 bits)
                         'size_t archive_read_filter::client_avail' offset changed from 1280 to 1344 (in bits) (by +64 bits)
                         'char archive_read_filter::end_of_file' offset changed from 1344 to 1408 (in bits) (by +64 bits)
                         'char archive_read_filter::closed' offset changed from 1352 to 1416 (in bits) (by +64 bits)
                         'char archive_read_filter::fatal' offset changed from 1360 to 1424 (in bits) (by +64 bits)

             type size hasn't changed


    [C]'function archive_write_filter* __archive_write_allocate_filter(archive*)' at archive_write.c:208:1 has some indirect sub-type changes:
      return type changed:
        in pointed to type 'struct archive_write_filter' at archive_write_private.h:48:1:
          type size hasn't changed
          1 data member insertion:
            'int archive_write_filter::state', at offset 736 (in bits) at archive_write_private.h:63:1
          no data member changes (6 filtered);

    [C]'function int _archive_entry_acl_from_text_l(archive_entry*, const char*, int, archive_string_conv*)' at archive_entry.c:1550:1 has some indirect sub-type changes:
      parameter 1 of type 'archive_entry*' has sub-type changes:
        in pointed to type 'struct archive_entry' at archive_entry_private.h:73:1:
          type size hasn't changed
          1 data member insertion:
            'int archive_entry::ae_symlink_type', at offset 8608 (in bits) at archive_entry_private.h:181:1

    [C]'function const stat* archive_entry_stat(archive_entry*)' at archive_entry_stat.c:40:1 has some indirect sub-type changes:
      parameter 1 of type 'archive_entry*' has sub-type changes:
        in pointed to type 'struct archive_entry' at archive_entry_private.h:73:1:
          type size hasn't changed
          1 data member insertion:
            'int archive_entry::ae_symlink_type', at offset 8608 (in bits) at archive_entry_private.h:181:1



  2 Added variables:

    'const Byte[16] const PPMD8_kExpEscape'    {PPMD8_kExpEscape}
    'const IPpmd8 __archive_ppmd8_functions'    {__archive_ppmd8_functions}

  1 Changed variable:

    [C]'const IPpmd7 __archive_ppmd7_functions' was changed at archive_ppmd7_private.h:118:1:
      type of variable changed:
       in unqualified underlying type 'typedef IPpmd7' at archive_ppmd7_private.h:116:1:
         underlying type 'struct {void (CPpmd7*)* Ppmd7_Construct; typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)* Ppmd7_Alloc; void (CPpmd7*, ISzAlloc*)* Ppmd7_Free; void (CPpmd7*, unsigned int)* Ppmd7_Init; void (CPpmd7z_RangeDec*)* Ppmd7z_RangeDec_CreateVTable; void (CPpmd7z_RangeDec*)* PpmdRAR_RangeDec_CreateVTable; typedef Bool (CPpmd7z_RangeDec*)* Ppmd7z_RangeDec_Init; typedef Bool (CPpmd7z_RangeDec*)* PpmdRAR_RangeDec_Init; int (CPpmd7*, IPpmd7_RangeDec*)* Ppmd7_DecodeSymbol; void (CPpmd7z_RangeEnc*)* Ppmd7z_RangeEnc_Init; void (CPpmd7z_RangeEnc*)* Ppmd7z_RangeEnc_FlushData; void (CPpmd7*, CPpmd7z_RangeEnc*, int)* Ppmd7_EncodeSymbol;}' at archive_ppmd7_private.h:94:1 changed:
           type size hasn't changed
           2 data member changes (4 filtered):
            type of 'typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)* Ppmd7_Alloc' changed:
              in pointed to type 'function type typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)':
                parameter 3 of type 'ISzAlloc*' was removed


            type of 'void (CPpmd7*, ISzAlloc*)* Ppmd7_Free' changed:
              in pointed to type 'function type void (CPpmd7*, ISzAlloc*)':
                parameter 2 of type 'ISzAlloc*' was removed




================ end of changes of 'libarchive.so.13.3.2'===============

Comment 12 Ondrej Dubaj 2020-04-30 10:16:04 UTC
Sorry for mistake, it is abidiff between versions 3.3.2 and 3.4.2

Comment 13 Ondrej Dubaj 2020-04-30 10:17:37 UTC
Pasting ABI diff of versions 3.3.2 and 3.3.2. There are some changes, but it seems this would be a better solutions than rebasing to 3.4.2. We have to investigate this more also according to CMake-3.17

abipkgdiff --d1 libarchive-debuginfo-3.3.2-9.el8.x86_64.rpm --d2 libarchive-debuginfo-3.3.3-1.el8.x86_64.rpm libarchive-3.3.2-9.el8.x86_64.rpm libarchive-3.3.3-1.el8.x86_64.rpm 
================ changes of 'libarchive.so.13.3.2'===============
  Functions changes summary: 0 Removed, 0 Changed (23 filtered out), 3 Added functions
  Variables changes summary: 0 Removed, 1 Changed, 0 Added variable

  3 Added functions:

    'function const char* archive_libzstd_version()'    {archive_libzstd_version}
    'function int archive_read_support_filter_zstd(archive*)'    {archive_read_support_filter_zstd}
    'function int archive_write_add_filter_zstd(archive*)'    {archive_write_add_filter_zstd}

  1 Changed variable:

    [C]'const IPpmd7 __archive_ppmd7_functions' was changed at archive_ppmd7_private.h:118:1:
      type of variable changed:
       in unqualified underlying type 'typedef IPpmd7' at archive_ppmd7_private.h:116:1:
         underlying type 'struct {void (CPpmd7*)* Ppmd7_Construct; typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)* Ppmd7_Alloc; void (CPpmd7*, ISzAlloc*)* Ppmd7_Free; void (CPpmd7*, unsigned int)* Ppmd7_Init; void (CPpmd7z_RangeDec*)* Ppmd7z_RangeDec_CreateVTable; void (CPpmd7z_RangeDec*)* PpmdRAR_RangeDec_CreateVTable; typedef Bool (CPpmd7z_RangeDec*)* Ppmd7z_RangeDec_Init; typedef Bool (CPpmd7z_RangeDec*)* PpmdRAR_RangeDec_Init; int (CPpmd7*, IPpmd7_RangeDec*)* Ppmd7_DecodeSymbol; void (CPpmd7z_RangeEnc*)* Ppmd7z_RangeEnc_Init; void (CPpmd7z_RangeEnc*)* Ppmd7z_RangeEnc_FlushData; void (CPpmd7*, CPpmd7z_RangeEnc*, int)* Ppmd7_EncodeSymbol;}' at archive_ppmd7_private.h:94:1 changed:
           type size hasn't changed
           2 data member changes:
            type of 'typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)* Ppmd7_Alloc' changed:
              in pointed to type 'function type typedef Bool (CPpmd7*, typedef UInt32, ISzAlloc*)':
                parameter 3 of type 'ISzAlloc*' was removed


            type of 'void (CPpmd7*, ISzAlloc*)* Ppmd7_Free' changed:
              in pointed to type 'function type void (CPpmd7*, ISzAlloc*)':
                parameter 2 of type 'ISzAlloc*' was removed




================ end of changes of 'libarchive.so.13.3.2'===============

Comment 14 Ondrej Dubaj 2020-04-30 10:18:15 UTC
Edit: 3.3.2 and 3.3.3

Comment 15 Denis Arnaud 2020-05-01 14:52:24 UTC
CMake 3.17 is now also available on EPEL 8 (with a bundled version of libarchive, if I'm correct): https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-ad02b27ee3

So, up to you to upgrade libarchive on RHEL 8, at least to 3.3.3, so that CMake3 can use it (rather than bundling it).

In the meantime, we have a working solution for CMake3 on EPEL 8.

Comment 16 Denis Arnaud 2020-05-01 15:03:28 UTC
CMake 3.17 on EPEL8: https://bugzilla.redhat.com/show_bug.cgi?id=1756974

Comment 22 Honza Horak 2020-09-15 13:52:45 UTC
Changing the summary to be in sync with the actual plan -- to update libarchive to 3.3.3 (3.4.2 has some ABI compatibility issues that are more described in comment #11).


Note You need to log in before you can comment on or make changes to this bug.