Bug 1726638

Summary:	glibc: Implement SVE Vector PCS on AArch64
Product:	Red Hat Enterprise Linux 8	Reporter:	Nick Clifton <nickc>
Component:	glibc	Assignee:	Florian Weimer <fweimer>
Status:	CLOSED ERRATA	QA Contact:	qe-baseos-tools-bugs
Severity:	medium	Docs Contact:
Priority:	high
Version:	8.2	CC:	ashankar, codonell, dj, fweimer, jfeeney, mnewsome, pfrankli, skolosov, tgummels, woodard
Target Milestone:	rc	Keywords:	FutureFeature, Patch, Triaged
Target Release:	8.2	Flags:	mnewsome: mirror+
Hardware:	aarch64
OS:	Linux
Whiteboard:
Fixed In Version:	glibc-2.28-86.el8	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-04-28 16:50:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1382435
Bug Blocks:	1543699, 1618515, 1683831, 1687244, 1746605, 1746913, 1755139, 1776957

Description Nick Clifton 2019-07-03 10:41:52 UTC

Clone of #1724130, created for the glibc component.

Description of problem:

Arm has introduced a vector PCS for SVE and Advanced SIMD. This showed a problem with lazy binding on SVE vector PCS while testing the qemu implementation of SVE primarily to do with saving and restoring the additional callee saved register state in the new vector PCS when lazily resolving SVE functions.

The draft text for fixing the ABI for this is at
https://sourceware.org/ml/libc-alpha/2019-05/msg00501.html

It requires the marking of vector functions in the symbol table so the dynamic linker can bind them early. For this the symbol marking has to be propagated from the compiler to the final dynamic linked binary.

Running an executable with vector pcs turned on for SVE. Note that such binaries could be generated by other commercial toolchains as well.

This has been fixed by a series of patches for gcc
binutils and glibc. These need to be backported on systems
that run on SVE hardware to support binaries with extern
SVE calls and to support compiling code with SVE or AdvSIMD
vector calls.

Without the glibc backport, the dynamic linker clobbers
SVE call argument registers in the lazy binding entry code
when the function is first called. So binaries with extern
SVE calls will misbehave at runtime.

Without the binutils backport, the bfd linker silently
drops symbol markings at link time, creating broken
binaries that misbehave even with an updated glibc.
Without the binutils backport, the assembler fails to
assemble code with SVE or AdvSIMD calls because it does
not understand the new .variant_pcs directive.

Without the gcc backport, gcc-9 emits AdvSIMD vector calls
without marking them with .variant_pcs and thus they won't
follow the new ABI. This is unlikely to cause runtime
problems now as AdvSIMD registers are unlikely to be
corrupted by the dynamic linker, but the marking is an ABI
requirement and will be necessary to support LD_AUDIT or
arbitrary ifunc resolvers for AdvSIMD vector calls among
other features.

We plan to backport the patches below to gcc-9, binutils-2.32
and several glibc release branches.

patches:

commit 55f82d328d2dd1c7c13c1992f4b9bf9c95b57551
Commit: Szabolcs Nagy <szabolcs.nagy>
CommitDate: 2019-06-13 09:44:44 +0100

aarch64: add STO_AARCH64_VARIANT_PCS and DT_AARCH64_VARIANT_PCS

commit 82bc69c012838a381c4167c156a06f4598f34227
Commit: Szabolcs Nagy <szabolcs.nagy>
CommitDate: 2019-06-13 09:45:00 +0100

aarch64: handle STO_AARCH64_VARIANT_PCS

Comment 1 Florian Weimer 2019-08-29 09:55:11 UTC

*** Bug 1383683 has been marked as a duplicate of this bug. ***

Comment 2 Carlos O'Donell 2019-08-29 13:34:08 UTC

commit 30ba0375464f34e4bf8129f3d3dc14d0c09add17
Author: Szabolcs Nagy <szabolcs.nagy>
Date:   Tue Jul 9 12:11:39 2019 +0100

    aarch64: simplify the DT_AARCH64_VARIANT_PCS handling code
    
    Remove unnecessary variant_pcs field: the dynamic tag can be checked
    directly.
    
            * sysdeps/aarch64/dl-machine.h (elf_machine_runtime_setup): Remove the
            DT_AARCH64_VARIANT_PCS check.
            (elf_machine_lazy_rel): Use l_info[DT_AARCH64 (VARIANT_PCS)].
            * sysdeps/aarch64/linkmap.h (struct link_map_machine): Remove
            variant_pcs.

commit 82bc69c012838a381c4167c156a06f4598f34227
Author: Szabolcs Nagy <szabolcs.nagy>
Date:   Thu Apr 25 15:35:35 2019 +0100

    aarch64: handle STO_AARCH64_VARIANT_PCS
    
    Avoid lazy binding of symbols that may follow a variant PCS with different
    register usage convention from the base PCS.
    
    Currently the lazy binding entry code does not preserve all the registers
    required for AdvSIMD and SVE vector calls.  Saving and restoring all
    registers unconditionally may break existing binaries, even if they never
    use vector calls, because of the larger stack requirement for lazy
    resolution, which can be significant on an SVE system.
    
    The solution is to mark all symbols in the symbol table that may follow
    a variant PCS so the dynamic linker can handle them specially.  In this
    patch such symbols are always resolved at load time, not lazily.
    
    So currently LD_AUDIT for variant PCS symbols are not supported, for that
    the _dl_runtime_profile entry needs to be changed e.g. to unconditionally
    save/restore all registers (but pass down arg and retval registers to
    pltentry/exit callbacks according to the base PCS).
    
    This patch also removes a __builtin_expect from the modified code because
    the branch prediction hint did not seem useful.
    
            * sysdeps/aarch64/dl-dtprocnum.h: New file.
            * sysdeps/aarch64/dl-machine.h (DT_AARCH64): Define.
            (elf_machine_runtime_setup): Handle DT_AARCH64_VARIANT_PCS.
            (elf_machine_lazy_rel): Check STO_AARCH64_VARIANT_PCS and bind such
            symbols at load time.
            * sysdeps/aarch64/linkmap.h (struct link_map_machine): Add variant_pcs.

commit 55f82d328d2dd1c7c13c1992f4b9bf9c95b57551
Author: Szabolcs Nagy <szabolcs.nagy>
Date:   Thu Apr 25 15:35:35 2019 +0100

    aarch64: add STO_AARCH64_VARIANT_PCS and DT_AARCH64_VARIANT_PCS
    
    STO_AARCH64_VARIANT_PCS is a non-visibility st_other flag for marking
    symbols that reference functions that may follow a variant PCS with
    different register usage convention from the base PCS.
    
    DT_AARCH64_VARIANT_PCS is a dynamic tag that marks ELF modules that
    have R_*_JUMP_SLOT relocations for symbols marked with
    STO_AARCH64_VARIANT_PCS (i.e. have variant PCS calls via a PLT).
    
            * elf/elf.h (STO_AARCH64_VARIANT_PCS): Define.
            (DT_AARCH64_VARIANT_PCS): Define.

Comment 12 Sergey Kolosov 2020-02-04 11:53:46 UTC

Verified by checking /usr/include/elf.h, it contains DT_AARCH64_VARIANT_PCS and STO_AARCH64_VARIANT_PCS definitions.

Comment 14 errata-xmlrpc 2020-04-28 16:50:14 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1828