Traits methods:

• Support for .NET Standard 2.1 new vector methods: ConvertToDouble, ConvertToInt32, ConvertToInt64, ConvertToSingle, ConvertToUInt32, ConvertToUInt64, Narrow, Widen .
• Support for .NET 5.0 new vector methods: Ceiling, Floor .
• Support for .NET 6.0 new vector methods: Sum .
• Support for .NET 7.0 new vector methods: ExtractMostSignificantBits, Shuffle, ShiftLeft, ShiftRightArithmetic, ShiftRightLogical .
• Support for .NET 8.0 new vector methods: WidenLower, WidenUpper.
• Provides the vector methods of narrow saturate: YNarrowSaturate, YNarrowSaturateUnsigned .
• Provides the vector methods of round: YRoundToEven, YRoundToZero .
• Provides the vector methods of shuffle: YShuffleInsert, YShuffleKernel, YShuffleG2, YShuffleG4, YShuffleG4X2 . Also provides ShuffleControlG2/ShuffleControlG4 enum.
• Provides vector methods for de-interleave: YGroup2Unzip, YGroup2UnzipEven, YGroup2UnzipOdd, YGroup3Unzip, YGroup3UnzipX2, YGroup4Unzip, YGroup6Unzip_Bit128.
• Provides vector methods for interleave: YGroup2Zip, YGroup2ZipHigh, YGroup2ZipLow, YGroup3Zip, YGroup3ZipX2, YGroup4Unzip, YGroup6Zip_Bit128.
• ...
• Full list: TraitsMethodList

Supported instruction set:

• x86 (Need .NET Core 3.0+)
  • 128-bit vector: Sse, Sse2, Sse3, Ssse3, Sse41, Sse42. And 128-bit instructions from Avx family.
  • 256-bit vector: Avx, Avx2. And 256-bit instructions from Avx512VL.
  • 512-bit vector: Avx512BW, Avx512DQ, Avx512F, Avx512Vbmi.
• Arm (Need .NET 5.0+)
  • 128-bit vector: AdvSimd.
• Wasm (Need .NET 8.0+)
  • 128-bit vector: PackedSimd.

Purpose

The SIMD instruction set is known to accelerate multimedia processing (graphics, images, audio, video, ...) , artificial intelligence, scientific computing, etc. However, traditional SIMD programming suffers from the following pain points.

• Difficult to cross-platform. Because different CPU systems provide different SIMD instruction sets, for example, there are many differences between the SIMD instruction sets of X86 and Arm platforms. If you want to port the program to another platform, you need to find the SIMD instruction set manual of that platform and develop it again.
• Bit widths are difficult to upgrade. Even for the same platform, as it evolves, instruction sets with wider bit widths are gradually added. For example, the X86 platform, in addition to the obsolete 64-bit MMX series instructions, provides a 128-bit SSE instruction set, a 256-bit AVX instruction set, and some high-end processors are starting to support the 512-bit AVX-512 instruction set. Algorithms previously written with 128-bit SSE series instructions need to be redeveloped to take full advantage of the wider SIMD instruction set if they are to be ported to the 256-bit AVX instruction set.
• Poor code readability and high development threshold. Many modern C compilers map Intrinsic Functions for SIMD instructions, which is much easier and more readable than writing assembly code. However, due to the use of some obscure abbreviations for function names, and the fact that C does not support function name overloading, as well as the complexity of the C language itself, there is still a high threshold for code readability and development difficulty.

NET Core 1.0 in 2016 added vector types such as Vector<T>, which largely solves the above pain points.

• Easy cross-platform. NET platform is run by JIT (Just-In-Time Compiler). Only one set of algorithms based on vector methods is written and compiled into only one set of programs. When that program is subsequently run on a different platform, the vector method is compiled by JIT into a platform-specific SIMD instruction set, thus taking full advantage of hardware acceleration.
• Bitwidth can be upgraded automatically. For the Vector<T> type, its length is not fixed, but is the same as the longest vector register for that processor. Specifically, if the CPU supports the AVX instruction set (strictly AVX2 and above), the Vector<T> type is 256 bits; if the CPU only supports the SSE instruction set (strictly SSE2 and above), the Vector<T> type is 128 bits. Simply put, you can write your program using only the Vector<T> type, and when the program runs, JIT will automatically use the widest SIMD instruction set.
• The code is more readable and lowers the development threshold. .NET platform, the method names of vector types are composed of complete English words, and make full use of C# syntax features such as function name overloading, so that these method names are both concise and clear. The readability of the code has been greatly improved.

The vector type Vector<T> although well designed, it lacks many important vector functions such as Ceiling, Sum, Shift, Shuffle, etc. This led to many algorithms that were difficult to implement with vector types. When .NET platform versions are upgraded, sometimes several vector methods are added. .NET 7.0 released in 2022, for example, added ShiftRightArithmetic, Shuffle and other methods. However, there are still few vector methods, such as the lack of saturation processing. To address the lack of vector methods, .NET Core 3.0 starts to support intrinsic functions. This allows developers to use the SIMD instruction set directly, but again, this faces problems such as difficulty in cross-platform and bit-width upgrades. As the .NET platform is upgraded, more intrinsic functions will be added. For example, .NET 5.0 adds intrinsic functions for the Arm platform. For developing libraries, you can't just support .NET 7.0, but you need to support multiple .NET versions. So you will face tedious version checking and conditional processing. And the highest version of the .NET Standard class library (2.1) still does not support vector methods like Ceiling, which makes version checking even more tedious.

This library is dedicated to solve the above troubles, so that you can write cross-platform SIMD algorithms more easily. Feature:

• Support for low versions of .NET programs (.NET Standard 1.1, .NET Core 1.0, .NET Framework 4.5, ...). Enables low version of .NET programs to use the latest vector functions. For example, ShiftRightArithmetic, Shuffle, etc. are new in .NET 7.0.
• Powerful functions . In addition to referencing vector methods from higher versions of .NET, this library also provides many useful vector methods by referring to intrinsic functions. e.g. ShiftLeft_Fast, YNarrowSaturate ...
• High performance. This library can take full advantage of the X86 and Arm architecture's intrinsic functions for hardware acceleration of vector type computations, and can enjoy inline compilation optimization. This library solves the problem that some of BCL's vector methods (e.g. Multiply, Shuffle, etc.) are not hardware-accelerated on some platforms, because it supplements the hardware-accelerated algorithms.
• Software algorithms are also fast. If you find a method of vector type does not support hardware acceleration, .NET Bcl will switch to software algorithm, but many of its software algorithms contain branching statements, so the performance is poor. The software algorithm of this library is a highly optimized branchless algorithm.
• Easy to use. This library supports not only Vector<T>, but also Vector128<T>/Vector256<T> and other vector types. The class name of the tool class is easy to remember (Vectors/Vector64s/Vector128s/Vector256s) and provides many common vector constants through a generic class of the same name.
• For each traits method, some properties are added to obtain information. e.g. _AcceleratedTypes, _FullAcceleratedTypes .

Tip: The Disassembly window in Visual Studio allows you to view the assembly code at runtime. For example, when running on a machine that supports the Avx instruction set, Vectors.ShiftLeft_Const will be compiled inline and optimized to use the vpsllw instruction. And for constant value(1), it will be compiled as the immediate number of the instruction.

Example 2: Using Vectors.ShiftLeft_Args and Vectors.ShiftLeft_Core, you can move some of the operations outside the loop to be processed earlier. For example, when running on a machine that supports the Avx instruction set, xmm1 is set outside the loop, and then used it in the vpsllw instruction of the inner loop. And here it is shown: the inline compilation optimization eliminates redundant xmm/ymm conversions.

Getting started

1) Install via NuGet

Either open the 'Package Management Console' and enter the following or use the built-in GUI

NuGet: PM> Install-Package VectorTraits

2) Usage examples

The static class Vectors provides some methods. e.g. CreateRotate, ShiftLeft, Shuffle. The generic structure 'Vectors' provides fields for commonly used constants.

The example code is in the samples/VectorTraits.Sample folder. The source code is as follows.

3) Example results

.NET8.0 on X86

Program: VectorTraits.Sample

Note: The text before Vectors.BaseInstance is the environment information output by TraitsOutput.OutputEnvironment. OutputEnvironment. The text starting from srcis the main code of the example. Since the CPU supports the X86 Avx2 instruction set,Vector.Countis 32(256bit), andVectors.InstanceisVectorTraits256Avx2`.

.NET8.0 on Arm

Program: VectorTraits.Sample

The result is the same as the X86 one, only the environment information is different. Since the CPU supports Arm's AdvSimd instruction set, Vector<byte>.Count is 16(128bit) and Vectors.Instance is VectorTraits128AdvSimdB64.

.NET Framework 4.5 on X86

Program: VectorTraits.Sample.NetFw.

ShiftLeft/Shuffle of Vectors works fine. Since the CPU supports the X86 Avx2 instruction set, Vector<byte>.Count is 32 (256bit). Vectors.InstanceisVectorTraits256Base. It's not VectorTraits256Avx2because the intrinsic function wasn't supported until.NET Core 3.0`. The value of ShiftLeft_AcceleratedTypes contains types such as "Int16", which means that ShiftLeft is hardware-accelerated when using these types. The library makes clever use of vector algorithms to try to achieve hardware acceleration even without intrinsic functions.

Results of benchmark

Unit of data: Million operations per second. The larger the number, the better the performance.

ShiftLeft

ShiftLeft: Shifts each element of a vector left by the specified amount. It is a new vector method in .NET 7.0.

ShiftLeft - X86 - AMD Ryzen 7 7840H

Description.

• SumSLLScalar: Use scalar algorithm.
• SumSLLNetBcl: Use the BCL method (Vector.ShiftLeft) with variable arguments. Note that this method is only available in .NET 7.0.
• SumSLLNetBcl_Const: Use the BCL method (Vector.ShiftLeft) with constant arguments. Note that this method is only available in .NET 7.0.
• SumSLLTraits: Use this library's normal method (Vectors.ShiftLeft) with variable arguments.
• SumSLLTraits_Core: Use this library's Core suffixed methods (Vectors.ShiftLeft_Args, Vectors.ShiftLeft_Core) with variable arguments.
• SumSLLConstTraits: Use this library's Const suffixed method (Vectors.ShiftLeft_Const) with constants arguments.
• SumSLLConstTraits_Core: Use this library's ConstCore suffixed methods (Vectors.ShiftLeft_Args, Vectors.ShiftLeft_ConstCore) with constant arguments.

BCL's method (Vector.ShiftLeft) runs on X86 platform, only Int16/Int32/Int64 are hardware accelerated, while Byte is not hardware accelerated. This is probably because the Avx2 instruction set only has 16-64 bit left shift instructions, and does not provide other types of instructions, so the BCL is converted to a software algorithm. For these types of numbers, this library will replace them with efficient algorithms realized by combinations of other instructions. For example, for Byte type, SumSLLConstTraits_Core in . NET 7.0 has the value of 73873.979, which is 73873.979/1693.330≈43.6264 times the performance of scalar algorithm, and 73873.979/1164.382≈63.4448 times the performance of BCL method. 32872.874/1137.564≈28.8976times. Because X86 intrinsic functions have only been available since.NET Core 3.0. Therefore, for Int64 types, hardware acceleration is not available until after .NET Core 3.0`.

For ShiftLeft, when shiftAmount is a constant, the performance is generally better than when it is a variable. This is true for both BCL and this library methods. Using this library's Core suffix optimizes performance by moving some operations out of the loop to be processed earlier. When the CPU provides instructions with constant parameters (the technical term is "immediate parameters"), the performance of the instructions is generally higher. So the library also provides a ConstCore suffix method, which selects the fastest instruction for that platform. Sometimes the performance fluctuates due to "CPU Turbo Boost", "other processes taking CPU resources", etc. But rest assured, after checking the assembly instructions of the Release's program runtime, it is already running on the best hardware instructions. An example of this is the following figure.

ShiftLeft - Arm - AWS Arm t4g.small

Description.

• SumSLLScalar: Use scalar algorithm.
• SumSLLNetBcl: Use the BCL method (Vector.ShiftLeft) with variable arguments. Note that this method is only available in .NET 7.0.
• SumSLLNetBcl_Const: Use the BCL method (Vector.ShiftLeft) with constant arguments. Note that this method is only available in .NET 7.0.
• SumSLLTraits: Use this library's normal method (Vectors.ShiftLeft) with variable arguments.
• SumSLLTraits_Core: Use this library's Core suffixed methods (Vectors.ShiftLeft_Args, Vectors.ShiftLeft_Core) with variable arguments.
• SumSLLConstTraits: Use this library's Const suffixed method (Vectors.ShiftLeft_Const) with constants arguments.
• SumSLLConstTraits_Core: Use this library's ConstCore suffixed methods (Vectors.ShiftLeft_Args, Vectors.ShiftLeft_ConstCore) with constant arguments.

The BCL method (Vector.ShiftLeft) runs on the Arm platform with hardware acceleration for integer types. The AdvSimd instruction set provides special instructions for left shifting of 8 to 64 bit integers. This library uses the same instructions when running on the Arm platform. The performance is close. Because Arm's intrinsic functions have only been available since .NET 5.0. The hardware acceleration for Int64 types is not available until after `.NET 5.0'.

ShiftRightArithmetic

ShiftRightArithmetic: Shifts (signed) each element of a vector right by the specified amount. It is a new vector method in .NET 7.0.

ShiftRightArithmetic - X86 - AMD Ryzen 7 7840H

Description.

• SumSRAScalar: Use scalar algorithm.
• SumSRANetBcl: Use the BCL method (Vector.ShiftRight) with variable arguments. Note that this method is only available in .NET 7.0.
• SumSRANetBcl_Const: Use the BCL method (Vector.ShiftRight) with constant arguments. Note that this method is only available in .NET 7.0.
• SumSRATraits: Use this library's normal method (Vectors.ShiftRight) with variable arguments.
• SumSRATraits_Core: Use this library's Core suffixed methods (Vectors.ShiftRight_Args, Vectors.ShiftRight_Core) with variable arguments.
• SumSRAConstTraits: Use this library's Const suffixed method (Vectors.ShiftRight_Const) with constants arguments.
• SumSRAConstTraits_Core: Use this library's ConstCore suffixed methods (Vectors.ShiftRight_Args, Vectors.ShiftRight_ConstCore) with constant arguments.

The BCL method (Vector.ShiftRightArithmetic) runs on X86 platforms with hardware acceleration only for Int16/Int32, but not for SByte/Int64. This is probably because the Avx2 instruction set only has 16-32 bit arithmetic right shift instructions. The Avx512 instruction set has added a 64 bit arithmetic right shift instruction. For these types of numbers, this library replaces them with efficient algorithms that are implemented by a combination of other instructions. As of .NET Core 3.0, hardware acceleration is available.

ShiftRightArithmetic - Arm - AWS Arm t4g.small

Description.

• SumSRAScalar: Use scalar algorithm.
• SumSRANetBcl: Use the BCL method (Vector.ShiftRight) with variable arguments. Note that this method is only available in .NET 7.0.
• SumSRANetBcl_Const: Use the BCL method (Vector.ShiftRight) with constant arguments. Note that this method is only available in .NET 7.0.
• SumSRATraits: Use this library's normal method (Vectors.ShiftRight) with variable arguments.
• SumSRATraits_Core: Use this library's Core suffixed methods (Vectors.ShiftRight_Args, Vectors.ShiftRight_Core) with variable arguments.
• SumSRAConstTraits: Use this library's Const suffixed method (Vectors.ShiftRight_Const) with constants arguments.
• SumSRAConstTraits_Core: Use this library's ConstCore suffixed methods (Vectors.ShiftRight_Args, Vectors.ShiftRight_ConstCore) with constant arguments.

BCL methods (Vector.ShiftRightArithmetic) are hardware accelerated for integer types when running on Arm platforms. The AdvSimd instruction set provides special instructions for arithmetic right shifting of 8 to 64 bit integers. This library uses the same instructions when running on the Arm platform. The performance is similar. As of .NET 5.0, hardware acceleration is available.

Shuffle

Shuffle: Shuffle and clear. Creates a new vector by selecting values from an input vector using a set of indices. It is a new vector method in .NET 7.0. Since .NET 7.0, the Shuffle method has been provided in Vector128/Vector256, but the Shuffle method has not yet been provided in Vector.

Shuffle allows an index to exceed the valid range, and then sets the corresponding element to 0. This feature slows down performance a bit, so this library also provides the YShuffleKernel method (Only shuffle). If you want to make sure that the index is always within the valid range, it is faster to use YShuffleKernel.

Shuffle - X86 - AMD Ryzen 7 7840H

Description.

• SumScalar: Use the scalar algorithm.
• Sum256_Bcl: Use BCL's 256-bit vector methods (Vector256.Shuffle).
• Sum512_Bcl: Use BCL's 512-bit vector methods (Vector512.Shuffle).
• SumTraits: Use the normal methods of this library (Vectors.Shuffle).
• SumTraits_Args0: Use this library's Core suffixed methods (Vectors.Shuffle_Args, Vectors.Shuffle_Core), without ValueTuple, use the "out" keyword to Returns multiple values.
• SumTraits_Args: Use this library's Core suffixed methods (Vectors.Shuffle_Args, Vectors.Shuffle_Core), using ValueTuple.
• SumKernelTraits: Use the normal methods of this library's YShuffleKernel (Vectors.YShuffleKernel).
• SumKernelTraits_Args0: Use the Core suffixed methods of this library's YShuffleKernel (Vectors.YShuffleKernel_Args, Vectors.YShuffleKernel_Core), without ValueTuple, use the "out" keyword to return multiple values.
• SumKernelTraits_Args: Use the Core suffixed methods of this library's YShuffleKernel (Vectors.YShuffleKernel_Args, Vectors.YShuffleKernel_Core), using ValueTuple.

BCL's method (Vector.Shuffle) runs on X86 platforms without hardware acceleration for all number types. This library replaces these types with efficient algorithms implemented by combinations of other instructions. As of .NET Core 3.0, hardware acceleration is available. Methods using this library's Core suffix optimize performance by moving some operations out of the loop to be processed earlier. This is especially true for the Shuffle method. YShuffleKernel can be used instead of Shuffle if you can ensure that the index is always in the valid range. It is faster. For Args suffixed methods, in addition to returning multiple values with the "out" keyword, ValueTuple can be used to receive multiple values, simplifying the code. However, be aware that ValueTuple can sometimes slow down performance.

Shuffle - Arm - AWS Arm t4g.small

Description.

• SumScalar: Use the scalar algorithm.
• Sum128_Bcl: Use BCL methods (Vector128.Shuffle).
• SumTraits: Use the normal methods of this library (Vectors.Shuffle).
• SumTraits_Args0: Use this library's Core suffixed methods (Vectors.Shuffle_Args, Vectors.Shuffle_Core), without ValueTuple, use the "out" keyword to Returns multiple values.
• SumTraits_Args: Use this library's Core suffixed methods (Vectors.Shuffle_Args, Vectors.Shuffle_Core), using ValueTuple.
• SumKernelTraits: Use the normal methods of this library's YShuffleKernel (Vectors.YShuffleKernel).
• SumKernelTraits_Args0: Use the Core suffixed methods of this library's YShuffleKernel (Vectors.YShuffleKernel_Args, Vectors.YShuffleKernel_Core), without ValueTuple, use the "out" keyword to return multiple values.
• SumKernelTraits_Args: Use the Core suffixed methods of this library's YShuffleKernel (Vectors.YShuffleKernel_Args, Vectors.YShuffleKernel_Core), using ValueTuple.

BCL's method (Vector.Shuffle) runs on the Arm platform without hardware acceleration for all number types. This library replaces these types with efficient algorithms implemented by combinations of other instructions. As of .NET 5.0, hardware acceleration is available. Note that prior to .NET 7.0, SumTraits_Args sometimes had a large performance difference from SumTraits_Args0, due to the large performance loss of ValueTuple under Arm.

YNarrowSaturate

YNarrowSaturate: Saturate narrows two Vector instances into one Vector .

YNarrowSaturate - X86 - AMD Ryzen 7 7840H

Description.

• SumNarrow_If: Use scalar algorithm based on if statements.
• SumNarrow_MinMax: Use scalar algorithm based on the Min/Max methods of the Math class.
• SumNarrowVectorBase: Use this library's base method (VectorTraitsBase.Statics.YNarrowSaturate). It is implemented by combining vector methods using BCL, and can take advantage of hardware acceleration.
• SumNarrowVectorTraits: Use this library's traits method (Vectors.YNarrowSaturate). It is implemented as an intrinsic function, allowing for better hardware acceleration.

For 16-32 bit integers, SumNarrowVectorTraits are much better than SumNarrowVectorBase after .NET Core 3.1. This is because X86 provides specialized instructions. For 64-bit integers (Int64/UInt64), X86 does not provide an equivalent instruction. However, the SumNarrowVectorTraits version of the code uses a better intrinsic function algorithm, so it still outperforms SumNarrowVectorBase in many cases.

YNarrowSaturate - Arm - AWS Arm t4g.small

Description.

• SumNarrow_If: Use scalar algorithm based on if statements.
• SumNarrow_MinMax: Use scalar algorithm based on the Min/Max methods of the Math class.
• SumNarrowVectorBase: Use this library's base method (VectorTraitsBase.Statics.YNarrowSaturate). It is implemented by combining vector methods using BCL, and can take advantage of hardware acceleration.
• SumNarrowVectorTraits: Use this library's traits method (Vectors.YNarrowSaturate). It is implemented as an intrinsic function, allowing for better hardware acceleration.

Since .NET 5.0, the Arm intrinsic function is provided. Therefore, starting from NET 5.0, SumNarrowVectorTraits are much more powerful than SumNarrowVectorBase.

YGroup3Unzip

YGroup3Unzip: De-Interleave 3-element groups into 3 vectors. It converts the 3-element groups AoS to SoA. It can also deinterleave packed RGB pixel data into R,G,B planar data.

YGroup3UnzipX2: De-Interleave 3-element groups into 3 vectors and process 2x data.

YGroup3Unzip - X86 - AMD Ryzen 7 7840H

Description.

• SumBase_Basic: Use scalar algorithm.
• SumNarrowVectorBase: Use this library's base method (VectorTraitsBase.Statics.YGroup3Unzip). It is implemented by combining vector methods using BCL, and can take advantage of hardware acceleration.
• SumNarrowVectorTraits: Use this library's traits method (Vectors.YGroup3Unzip). It is implemented as an intrinsic function, allowing for better hardware acceleration.
• SumX2Base: Use VectorTraitsBase.Statics.YGroup3UnzipX2. For 8~16 bit integers, YGroup3UnzipX2 is generally faster than YGroup3Unzip, more so under earlier versions of .NET.
• SumX2Traits: Use Vectors.YGroup3UnzipX2.

YGroup3Unzip - Arm - AWS Arm t4g.small

More results

See: BenchmarkResults

Documentation

• Traits method list: TraitsMethodList
• Online document: https://zyl910.github.io/VectorTraits_doc/
• DocFX: Run docfx_serve.bat. Then browse http://localhost:8080/ .
• Doxygen: Run the Doxywizard and click File ->Open on the menu bar. Select the Doxyfile file and click "OK". Click on the "Run" tab and click on the "Run doxygen" button. It will generate documents in the "doc_gen" folder.

ChangeLog

Full list: ChangeLog