generic-library/isa-l
1
0
Fork 0
mirror of https://github.com/intel/isa-l.git synced 2025-10-12 22:36:52 +02:00
Code Issues Projects Releases Wiki Activity

aarch64: Use NEON when SVE width is 128 bits

Browse Source 
On AArch64 systems with SVE support, 128-bit SVE implementations can
perform significantly worse than equivalent NEON code due to the
different optimization strategies used in each implementation. The NEON
version is unrolled 4 times, providing excellent performance at the
fixed 128-bit width. The SVE version can achieve similar or better
performance through its variable-width operations on systems with
256-bit or 512-bit SVE, but on 128-bit SVE systems, the NEON unrolled
implementation is faster due to reduced overhead.

This change adds runtime detection of SVE vector length and falls back
to the optimized NEON implementation when SVE is operating at 128-bit
width, ensuring optimal performance across all AArch64 configurations.

This implementation checks the vector length with an intrinsic if the
compiler supports it (which works on Apple as well) and falls back to
using prctl otherwise.

This optimization ensures that systems benefit from:
- 4x unrolled NEON code on 128-bit SVE systems
- Variable-width SVE optimizations on wider SVE implementations
- Maintained compatibility across different AArch64 configurations

Performance improvement on systems with 128-bit SVE:
- Encode: 7509.80 MB/s → 8995.59 MB/s (+19.8% improvement)
- Decode: 9383.67 MB/s → 12272.38 MB/s (+30.8% improvement)

Signed-off-by: Jonathan Swinney <jswinney@amazon.com>
This commit is contained in:
Jonathan Swinney
2025-07-11 16:38:19 +00:00
committed by Pablo de Lara
parent 09cec64707
commit aedcd375ba
1 changed files with 43 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
erasure_code/aarch64/ec_aarch64_dispatcher.c
View File

@@ -30,6 +30,40 @@
#include "erasure_code.h"
#include "gf_vect_mul.h"
#ifdef __ARM_FEATURE_SVE
// If the compiler defines SVE intrinsics, include that header
#include <arm_sve.h>
#elif defined(__linux__)
// Otherwise include these headers and define these constants as a fallback for Linux only
#include <stddef.h>
#include <sys/auxv.h>
#include <sys/prctl.h>
#ifndef PR_SVE_GET_VL
#define PR_SVE_GET_VL 51
#endif
#ifndef PR_SVE_VL_LEN_MASK
#define PR_SVE_VL_LEN_MASK 0xffff
#endif
#endif
static inline size_t
get_sve_vector_length_bytes(void)
{
#ifdef __ARM_FEATURE_SVE
// Use intrinsic if available at compile time
return svcntb();
#elif defined(__linux__)
// Fall back to prctl on Linux
long sve_vl = prctl(PR_SVE_GET_VL);
if (sve_vl != -1) {
return sve_vl & PR_SVE_VL_LEN_MASK;
}
#endif
return 0; // Unknown or unavailable
}
extern void
gf_vect_dot_prod_sve(int, int, unsigned char *, unsigned char **, unsigned char *);
extern void
@@ -94,8 +128,16 @@ DEFINE_INTERFACE_DISPATCHER(ec_encode_data)
#if defined(__linux__)
unsigned long auxval = getauxval(AT_HWCAP);
if (auxval & HWCAP_SVE)
if (auxval & HWCAP_SVE) {
size_t vector_length = get_sve_vector_length_bytes();
// If 128-bit SVE (16 bytes), use NEON instead
if (vector_length == 16 && (auxval & HWCAP_ASIMD)) {
return ec_encode_data_neon;
}
return ec_encode_data_sve;
}
if (auxval & HWCAP_ASIMD)
return ec_encode_data_neon;
#elif defined(__APPLE__)