Zero allocation LINQ with LINQ to Span, LINQ to SIMD, and LINQ to Tree (FileSystem, JSON, GameObject, etc.) for all .NET platforms(netstandard2.0, 2.1, net8, net9) and Unity, Godot.
Unlike regular LINQ, ZLinq doesn't increase allocations when adding more method chains, and it also has higher basic performance. You can check various benchmark patterns at GitHub Actions/Benchmark. ZLinq shows high performance in almost all patterns, with some benchmarks showing overwhelming differences.
As a bonus, LINQ operators and optimizations equivalent to .NET 10 can be used in .NET Framework 4.8 (netstandard2.0) and Unity (netstandard2.1).
dotnet add package ZLinqusing ZLinq;
var seq = source
.AsValueEnumerable() // only add this line
.Where(x => x % 2 == 0)
.Select(x => x * 3);
foreach (var item in seq) { }
99% compatibility with .NET 10's LINQ (including new Shuffle, RightJoin, LeftJoin operators)
Zero allocation for method chains through struct-based Enumerable via ValueEnumerable
LINQ to Span to full support LINQ operations on Span<T> using .NET 9/C# 13's allows ref struct
LINQ to Tree to extend tree-structured objects (built-in support for FileSystem, JSON, GameObject)
LINQ to SIMD to automatic application of SIMD where possible and customizable arbitrary operations
Optional Drop-in replacement Source Generator to automatically accelerate all LINQ methods
In ZLinq, we have proven high compatibility and performance by running dotnet/runtime's System.Linq.Tests as a drop-in replacement, passing 9000 tests.
Previously, value type-based LINQ implementations were often experimental, but ZLinq fully implements all methods to completely replace standard LINQ in production use, delivering high performance suitable even for demanding applications like games. The performance aspects are based on my experience with previous LINQ implementations (linq.js, SimdLinq, UniRx, R3), zero-allocation implementations (ZString, ZLogger), and high-performance serializers (MessagePack-CSharp, MemoryPack).
ZLinq achieves zero-allocation LINQ implementation using the following structs and interfaces.
public readonly ref struct ValueEnumerable<TEnumerator, T>(TEnumerator enumerator)
where TEnumerator : struct, IValueEnumerator<T>, allows ref struct
{
public readonly TEnumerator Enumerator = enumerator;
}
public interface IValueEnumerator<T> : IDisposable
{
bool TryGetNext(out T current); // as MoveNext + Current
// Optimization helper
bool TryGetNonEnumeratedCount(out int count);
bool TryGetSpan(out ReadOnlySpan<T> span);
bool TryCopyTo(Span<T> destination, Index offset);
}
Besides changing to a struct-based approach, we've integrated MoveNext and Current to reduce the number of iterator calls. Also, some operators don't need to hold Current, which allows minimizing the struct size. Additionally, being struct-based, we efficiently separate internal state by copying the Enumerator instead of using GetEnumerator. With .NET 9/C# 13 or later, allows ref struct enables natural integration of Span<T> into LINQ.
Operators have this method signature. C# cannot infer types from generic constraints(dotnet/csharplang#6930). Therefore, the traditional Struct LINQ approach required implementing all operator combinations as instance methods, resulting in 100,000+ methods and massive assembly sizes. However, in ZLinq, we've successfully avoided all the boilerplate method implementations by devising an approach that properly conveys types to C# compiler.
Additionally, TryGetNonEnumeratedCount(out int count), TryGetSpan(out ReadOnlySpan<T> span), and TryCopyTo(Span<T> destination, Index offset) defined in the interface itself enable flexible optimizations. To minimize assembly size, we've designed the library to achieve maximum optimization with minimal method additions. For example, TryCopyTo works efficiently with methods like ToArray when combined with TryGetNonEnumeratedCount. However, it also allows copying to smaller-sized destinations. By combining this with Index, we can optimize First, Last, and ElementAt using just TryCopyTo by passing a single-element Span along with an Index.
Use using ZLinq; and call AsValueEnumerable() on any iterable type to use ZLinq's zero-allocation LINQ.
using ZLinq;
var source = new int[] { 1, 2, 3, 4, 5 };
// Call AsValueEnumerable to apply ZLinq
var seq1 = source.AsValueEnumerable().Where(x => x % 2 == 0);
// Can also be applied to Span (only in .NET 9/C# 13 environments that support allows ref struct)
Span<int> span = stackalloc int[5] { 1, 2, 3, 4, 5 };
var seq2 = span.AsValueEnumerable().Select(x => x * x);
Even if it's netstandard 2.0 or below .NET 10, all operators up to .NET 10 are available.
You can method chain and foreach like regular LINQ, but there are some limitations. Please see Difference and Limitation for details. ZLinq has drop-in replacements that apply ZLinq without needing to call AsValueEnumerable(). For more information, see Drop-in replacement. Detailed information about LINQ to Tree for LINQ-ifying tree structures (FileSystems and JSON) and LINQ to SIMD for expanding SIMD application range can be found in their respective sections.
Additional Operators
In ZLinq, we prioritize compatibility, so we try to minimize adding custom operators. However, the following methods have been added to enable efficient processing with zero allocation:
AsValueEnumerable()
Converts existing collections to a type that can be chained with ZLinq. Any IEnumerable<T> can be converted, but for the following types, conversion is done with zero allocation without IEnumerable<T>.GetEnumerator() allocation. Standard supported types are T[], List<T>, ArraySegment<T>, Memory<T>, ReadOnlyMemory<T>, ReadOnlySequence<T>, Dictionary<TKey, TValue>, Queue<T>, Stack<T>, LinkedList<T>, HashSet<T>, ImmutableArray<T>, Span<T>, ReadOnlySpan<T>. However, conversion from ImmutableArray<T> requires .NET 8 or higher, and conversion from Span<T>, ReadOnlySpan<T> requires .NET 9 or higher.
ValueEnumerable.Range operates more efficiently when handling with ZLinq than Enumerable.Range().AsValueEnumerable(). The same applies to Repeat and Empty.
Average() : where INumber<T>, Sum() : where INumber<T>
System.Linq's Average and Sum are limited to certain primitive types, but ZLinq extends them to all INumber<T> types. In .NET 8 or higher, where constraints are included, but for others (netstandard2.0, 2.1), runtime errors will occur when called with non-primitive target types.
SumUnchecked()
Sum is checked, but checking for overflow during SIMD execution creates performance overhead. SumUnchecked skips overflow checking to achieve maximum SIMD aggregation performance. Note that this requires .NET 8 or higher, and SIMD-supported types are sbyte, short, int, long, byte, ushort, uint, ulong, double, and the source must be able to get a Span (TryGetSpan returns true).
int CopyTo(Span<T> destination), void CopyTo(List<T> list)
CopyTo can be used to avoid allocation of the return collection unlike ToArray or ToList. int CopyTo(Span<T> destination) allows the destination to be smaller than the source, returning the number of elements copied. void CopyTo(List<T> list) clears the list and then fills it with elements from the source, so the destination size is list.Count.
(TSource[] Array, int Size) ToArrayPool()
The returned array is rented from ArrayPool<TSource>.Shared. When using it, you need to create a slice of elements using Array.AsSpan(0, Size) or similar. Also, after use, you must return it with ArrayPool<TSource>.Shared.Rent(Array). If TSource is a reference type, set clearArray: true to prevent memory leaks.
JoinToString(char|string seperator)
Since ZLinq is not IEnumerable<T>, it cannot be passed to String.Join. JoinToString provides the same functionality as String.Join, returning a string joined with the separator.
Difference and Limitation
For .NET 9 and above, ValueEnumerable<T> is a ref struct and cannot be converted to IEnumerable<T>. To ensure compatibility when upgrading, AsEnumerable is not provided by default even for versions prior to .NET 9.
Since ValueEnumerable<T> is not an IEnumerable<T>, it cannot be passed to methods that require IEnumerable<T>. It's also difficult to pass it to other methods due to the complex type signatures required by generics (implementation is explained in the Custom Extensions section). Using ToArray() is one solution, but this can cause unnecessary allocations in some cases.
String.Join has overloads for both IEnumerable<string> and params object[]. Passing ValueEnumerable<T> directly will select the object[] overload, which may not give the desired result. In this case, use the JoinToString operator instead.
ValueEnumerable<T> is a struct, and its size increases slightly with each method chain. With many chained methods, copy costs can become significant. When iterating over small collections, these copy costs can outweigh the benefits, causing performance to be worse than standard LINQ. However, this is only an issue with extremely long method chains and small iteration counts, so it's rarely a practical concern.
Each chain operation returns a different type, so you cannot reassign to the same variable. For example, code that continuously reassigns Concat in a for loop cannot be implemented.
In .NET 8 and above, the Sum and Average methods for double use SIMD processing, which performs parallel processing based on SIMD width. This results in calculation errors that differ from normal ones due to the different order of addition.
Drop-in replacement
When introducing ZLinq.DropInGenerator, you can automatically use ZLinq for all LINQ methods without calling AsValueEnumerable().
dotnet add package ZLinq.DropInGenerator
It works by using a Source Generator to add extension methods for each type that take priority, making ZLinq methods be selected instead of System.Linq when the same name and arguments are used.
After installing the package, you need to configure it with an assembly attribute.
generateNamespace is the namespace for the generated code, and DropInGenerateTypes selects the target types.
DropInGenerateTypes allows you to choose from Array, Span (Span/ReadOnlySpan), Memory (Memory/ReadOnlyMemory), List, and Enumerable (IEnumerable).
These are Flags, so you can combine them, such as DropInGenerateTypes.Array | DropInGenerateTypes.Span.
There are also predefined combinations: Collection = Array | Span | Memory | List and Everything = Array | Span | Memory | List | Enumerable.
You can enable it for all files by global using the generated namespace:
global using ZLinq.DropIn;
When using DropInGenerateTypes.Enumerable, which generates extension methods for IEnumerable<T>, you need to make generateNamespace global as a namespace priority.
For example:
This is the most aggressive configuration, causing all LINQ methods to be processed by ZLinq, and making it impossible to use normal LINQ methods (if Enumerable is not included, you can call AsEnumerable() to execute with System.Linq).
It's better to use application's default namespace rather than globally, as this allows you to switch between normal LINQ using namespaces. This approach is recommended when you need to target Enumerable.
using ZLinq;
[assembly: ZLinqDropInAttribute("MyApp", DropInGenerateTypes.Everything)]
// namespace under MyApp
namespace MyApp.Foo
{
public class Bar
{
public static void Foo(IEnumerable<int> source)
{
// ZLinq ValueEnumerable<T>
var seq = source.Select(x => x * 2).Shuffle();
using var e = seq.Enumerator;
while (e.TryGetNext(out var current))
{
Console.WriteLine(current);
}
}
}
}
// not under MyApp namespace
namespace NotMyApp
{
public class Baz
{
public static void Foo(IEnumerable<int> source)
{
// IEnumerable<T>
var seq = source.Select(x => x * 2); // .Shuffle();
using var e = seq.GetEnumerator();
while (e.MoveNext())
{
Console.WriteLine(e.Current);
}
}
}
}
ZLinq is powerful and in many cases it performs better than regular LINQ, but it also has its limitations. For more information, please refer to Difference and Limitation. When you are not familiar with it, we recommend that you use DropInGenerateTypes.Collection instead of DropInGenerateTypes.Everything.
Other options for ZLinqDropInAttribute include GenerateAsPublic, ConditionalCompilationSymbols, and DisableEmitSource.
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = false, Inherited = false)]
public sealed class ZLinqDropInAttribute : Attribute
{
/// <summary>
/// Gets the namespace where the generated LINQ implementations will be placed.
/// If empty, the implementations will be generated in the global namespace.
/// </summary>
public string GenerateNamespace { get; }
/// <summary>
/// Gets the types of collections for which LINQ implementations should be generated.
/// </summary>
public DropInGenerateTypes DropInGenerateTypes { get; }
/// <summary>
/// Gets whether the generated LINQ implementations should be public.
/// When true, the implementations will be generated with public visibility.
/// When false (default), the implementations will be generated with internal visibility.
/// </summary>
public bool GenerateAsPublic { get; set; }
/// <summary>
/// Gets or sets the conditional compilation symbols to wrap the generated code with #if directives.
/// If specified, the generated code will be wrapped in #if/#endif directives using these symbols.
/// </summary>
public string? ConditionalCompilationSymbols { get; set; }
/// <summary>
/// Gets or sets whether to disable source generation in emitted code.
/// When true, the source code comments will not be included in the generated code.
/// When false (default), source code comments will be included in the generated code.
/// </summary>
public bool DisableEmitSource { get; set; }
/// <summary>
/// Initializes a new instance of the <see cref="ZLinqDropInAttribute"/> class.
/// </summary>
/// <param name="generateNamespace">The namespace where the generated LINQ implementations will be placed. If empty, place to global.</param>
/// <param name="dropInGenerateTypes">The types of collections for which LINQ implementations should be generated.</param>
public ZLinqDropInAttribute(string generateNamespace, DropInGenerateTypes dropInGenerateTypes)
{
GenerateNamespace = generateNamespace;
DropInGenerateTypes = dropInGenerateTypes;
}
}
LINQ to Tree
LINQ to XML introduced the concept of querying around axes to C#. Even if you don't use XML, similar APIs are incorporated into Roslyn and effectively used for exploring SyntaxTrees. ZLinq extends this concept to make it applicable to anything that can be considered a Tree, allowing Ancestors, Children, Descendants, BeforeSelf, and AfterSelf to be applied.
Specifically, by defining a struct that implements the following interface, it becomes iterable:
public interface ITraverser<TTraverser, T> : IDisposable
where TTraverser : struct, ITraverser<TTraverser, T> // self
{
T Origin { get; }
TTraverser ConvertToTraverser(T next); // for Descendants
bool TryGetHasChild(out bool hasChild); // optional: optimize use for Descendants
bool TryGetChildCount(out int count); // optional: optimize use for Children
bool TryGetParent(out T parent); // for Ancestors
bool TryGetNextChild(out T child); // for Children | Descendants
bool TryGetNextSibling(out T next); // for AfterSelf
bool TryGetPreviousSibling(out T previous); // BeforeSelf
}
Standard packages are available for FileSystemInfo and JsonNode. For Unity, it's applicable to GameObject and Transform.
FileSystem
dotnet add package ZLinq.FileSystem
using ZLinq;
var root = new DirectoryInfo("C:\\Program Files (x86)\\Steam");
// FileSystemInfo(FileInfo/DirectoryInfo) can call `Ancestors`, `Children`, `Descendants`, `BeforeSelf`, `AfterSelf`
var allDlls = root
.Descendants()
.OfType<FileInfo>()
.Where(x => x.Extension == ".dll");
var grouped = allDlls
.GroupBy(x => x.Name)
.Select(x => new { FileName = x.Key, Count = x.Count() })
.OrderByDescending(x => x.Count);
foreach (var item in grouped)
{
Console.WriteLine(item);
}
In .NET 8 and above, there are operators that apply SIMD when ValueEnumerable<T>.TryGetSpan returns true. The scope of application is wider than in regular System.Linq.
Range to ToArray/ToList/CopyTo/etc...
Repeat for unmanaged struct and size is power of 2 to ToArray/ToList/CopyTo/etc...
Sum for sbyte, short, int, long, byte, ushort, uint, ulong, double
Sum performs calculations as checked, but if you don't need to worry about overflow, using SumUnchecked is faster.
Method
N
Mean
Allocated
ForLoop
16384
25,198.556 ns
-
SystemLinqSum
16384
1,402.259 ns
-
ZLinqSum
16384
1,351.449 ns
-
ZLinqSumUnchecked
16384
721.832 ns
-
By using ZLinq.Simd in your using statements, you can call .AsVectorizable() on T[] or Span<T> or ReadOnlySpan<T>, which allows you to use Sum, SumUnchecked, Average, Max, Min, Contains, and SequenceEqual. This explicitly indicates execution with SIMD regardless of the LINQ chain state (though type checking is ambiguous so processing might occur in a normal loop, and if Vector.IsHardwareAccelerated && Vector<T>.IsSupported is false, normal loop processing will be used).
From int[] or Span<int>, you can call VectorizedFillRange. This is equivalent to ValueEunmerable.Range().CopyTo() and allows you to quickly generate sequential numbers through SIMD processing.
Method
Mean
Allocated
Range
540.0 ns
-
For
6,228.9 ns
-
VectorizedUpdate
In ZLinq, you can perform relatively flexible vectorized loop processing using Func. With T[] and Span<T>, you can use the VectorizedUpdate method. By writing two lambda expressions - Func<Vector<T>, Vector<T>> vectorFunc for vector operations and Func<T, T> func for handling remainder elements - you can perform loop update processing at SIMD width.
using ZLinq.Simd; // needs using
int[] source = Enumerable.Range(0, 10000).ToArray();
[Benchmark]
public void For()
{
for (int i = 0; i < source.Length; i++)
{
source[i] = source[i] * 10;
}
}
[Benchmark]
public void VectorizedUpdate()
{
// arg1: Vector<int> => Vector<int>
// arg2: int => int
source.VectorizedUpdate(static x => x * 10, static x => x * 10);
}
Method
N
Mean
Error
StdDev
Allocated
For
10000
4,560.5 ns
67.24 ns
3.69 ns
-
VectorizedUpdate
10000
558.9 ns
6.42 ns
0.35 ns
-
There is delegate overhead when compared to writing everything inline, but processing can be faster than using for-loops. However, this varies case by case, so please take measurements in advance based on your data volume and method content. Of course, if you're seeking the best possible performance, you should write code inline.
Vectorizable Methods
You can convert from T[] or Span<T> or ReadOnlySpan<T> to Vectorizable<T> using AsVectorizable(), which allows you to use Aggregate, All, Any, Count, Select, and Zip methods that accept a Func as an argument.
source.AsVectorizable().All(x => Vector.GreaterThanAll(x, new(5000)), x => x > 5000);
Any
source.AsVectorizable().Any(x => Vector.LessThanAll(x, new(5000)), x => x < 5000);
Count
source.AsVectorizable().Count(x => Vector.GreaterThan(x, new(5000)), x => x > 5000);
Method
Mean
Error
StdDev
Allocated
VectorizableCount
1,048.4 ns
39.39 ns
2.16 ns
-
LinqCount
10,909.3 ns
54.79 ns
3.00 ns
-
Select -> ToArray or CopyTo
source.AsVectorizable().Select(x => x * 3, x => x * 3).ToArray();
source.AsVectorizable().Select(x => x * 3, x => x * 3).CopyTo(destination);
Zip -> ToArray or CopyTo
// Zip2
array1.AsVectorizable().Zip(array2, (x, y) => x + y, (x, y) => x + y).CopyTo(destination);
array1.AsVectorizable().Zip(array2, (x, y) => x + y, (x, y) => x + y).ToArray();
// Zip3
array1.AsVectorizable().Zip(array2, array3, (x, y, z) => x + y + z, (x, y, z) => x + y + z).CopyTo(destination);
array1.AsVectorizable().Zip(array2, array3, (x, y, z) => x + y + z, (x, y, z) => x + y + z).ToArray();
Method
Mean
ZLinqVectorizableZipCopyTo
24.17 μs
ZLinqVectorizableZip3CopyTo
29.26 μs
ZLinqZipCopyTo
329.43 μs
ZLinqZip3CopyTo
584.69 μs
Unity
There are two installation steps required to use it in Unity.
Install ZLinq from NuGet using NuGetForUnity
Open Window from NuGet -> Manage NuGet Packages, Search "ZLinq" and Press Install.
Install the ZLinq.Unity package by referencing the git URL
With the help of the Unity package, in addition to the standard ZLinq, LINQ to GameObject functionality becomes available for exploring GameObject/Transform.
using ZLinq;
public class SampleScript : MonoBehaviour
{
public Transform Origin;
void Start()
{
Debug.Log("Ancestors--------------"); // Container, Root
foreach (var item in Origin.Ancestors()) Debug.Log(item.name);
Debug.Log("Children--------------"); // Sphere_A, Sphere_B, Group, Sphere_A, Sphere_B
foreach (var item in Origin.Children()) Debug.Log(item.name);
Debug.Log("Descendants--------------"); // Sphere_A, Sphere_B, Group, P1, Group, Sphere_B, P2, Sphere_A, Sphere_B
foreach (var item in Origin.Descendants()) Debug.Log(item.name);
Debug.Log("BeforeSelf--------------"); // C1, C2
foreach (var item in Origin.BeforeSelf()) Debug.Log(item.name);
Debug.Log("AfterSelf--------------"); // C3, C4
foreach (var item in Origin.AfterSelf()) Debug.Log(item.name);
}
}
You can chain query(LINQ to Objects). Also, you can filter by component using the OfComponent<T> helper.
// all filtered(tag == "foobar") objects
var foobars = root.Descendants().Where(x => x.tag == "foobar");
// get FooScript under self childer objects and self
var fooScripts = root.ChildrenAndSelf().OfComponent<FooScript>();
NOTE: In Unity, since .NET Standard 2.1 is referenced, SIMD is not utilized.
In .NET 9, ValueEnumerable is a ref struct, so it cannot be converted to IEnumerable<T>. However, in Unity it's a regular struct, making it possible to convert to IEnumerable<T>. You can improve interoperability by preparing an extension method like this:
public static class ZLinqExtensions
{
public static IEnumerable<T> AsEnumerable<TEnumerator, T>(this ValueEnumerable<TEnumerator, T> valueEnumerable)
where TEnumerator : struct, IValueEnumerator<T>
{
using (var e = valueEnumerable.Enumerator)
{
while (e.TryGetNext(out var current))
{
yield return current;
}
}
}
}
In Unity, you can convert NativeArray, NativeSlice using AsEnumerable() to write queries with ZLinq. If Unity Collections(com.unity.collections) package version is 2.1.1 or above, NativeQueue, NativeHashSet, NativeText, FixedList32Bytes, FixedList64Bytes, FixedList128Bytes, FixedList512Bytes, FixedList4096Bytes, FixedString32Bytes, FixedString64Bytes, FixedString128Bytes, FixedString512Bytes, and FixedString4096Bytes support AsValueEnumerable().
You can also use drop-in replacement. Add ZLinq.DropInGenerator from NuGetForUnity. If you want to use DropInGenerator, the minimum supported Unity version will be 2022.3.12f1, as it is necessary to support C# Incremental Source Generator(Compiler Version, 4.3.0).
Assembly attributes need to be set for each asmdef. For example, place a .cs file like the following in each asmdef. The DropInGenerator is defined in the assembly attributes.
// AssemblyAttributes.cs
using ZLinq;
[assembly: ZLinqDropIn("MyApp", DropInGenerateTypes.Array | DropInGenerateTypes.List)]
For more details about DropInGenerator, please refer to the Drop-in replacement section.
Godot
The minimum supported Godot version will be 4.0.0.
You can install ZLinq.Godot package via NuGet.
dotnet add package ZLinq.Godot
In addition to the standard ZLinq, LINQ to Node functionality is available.
using Godot;
using ZLinq;
public partial class SampleScript : Node2D
{
public override void _Ready()
{
var origin = GetNode<Node2D>("Container/Origin");
GD.Print("Ancestors--------------"); // Container, Root, root (Root Window)
foreach (var item in origin.Ancestors()) GD.Print(item.Name);
GD.Print("Children--------------"); // Sphere_A, Sphere_B, Group, Sphere_A, Sphere_B
foreach (var item in origin.Children()) GD.Print(item.Name);
GD.Print("Descendants--------------"); // Sphere_A, Sphere_B, Group, P1, Group, Sphere_B, P2, Sphere_A, Sphere_B
foreach (var item in origin.Descendants()) GD.Print(item.Name);
GD.Print("BeforeSelf--------------"); // C1, C2
foreach (var item in origin.BeforeSelf()) GD.Print(item.Name);
GD.Print("AfterSelf--------------"); // C3, C4
foreach (var item in origin.AfterSelf()) GD.Print(item.Name);
}
}
You can chain query(LINQ to Objects). Also, you can filter by node type using the OfType().
// get ancestors under a Window
var ancestors = root.Ancestors().TakeWhile(x => x is not Window);
// get FooScript under self childer objects and self
var fooScripts = root.ChildrenAndSelf().OfType<FooScript>();
Custom Extensions
Implementing extension methods for IEnumerable<T> is common. There are two types of operators: consuming operators like Count and Sum, and chainable operators like Select and Where. This section explains how to implement them.
Consume Operator
The method signature is slightly more complex compared to IEnumerable<T>, requiring constraints on TEnumerator. For .NET 9 or later, allows ref struct is also needed.
public static class MyExtensions
{
public static void Consume<TEnumerator, TSource>(this ValueEnumerable<TEnumerator, TSource> source)
where TEnumerator : struct, IValueEnumerator<TSource>
#if NET9_0_OR_GREATER
, allows ref struct
#endif
{
using var e = source.Enumerator; // using Enumerator
while (e.TryGetNext(out var current)) // MoveNext + Current
{
}
}
}
Instead of GetEnumerator(), use Enumerator, and instead of MoveNext + Current, use TryGetNext(out) to consume the iterator. The Enumerator must be used with using.
Consumers can call the Enumerator's optimization methods: TryGetNonEnumeratedCount, TryGetSpan, and TryCopyTo. For example, getting a Span like this is faster than normal iteration with TryGetNext:
public static class MyExtensions
{
public static void ForEach<TEnumerator, TSource>(this ValueEnumerable<TEnumerator, TSource> source, Action<TSource> action)
where TEnumerator : struct, IValueEnumerator<TSource>
#if NET9_0_OR_GREATER
, allows ref struct
#endif
{
using var e = source.Enumerator;
if (e.TryGetSpan(out var span))
{
// faster iteration
foreach (var item in span)
{
action(item);
}
}
else
{
while (e.TryGetNext(out var item))
{
action(item);
}
}
}
public static ImmutableArray<T> ToImmutableArray<TEnumerator, T>(this ValueEnumerable<TEnumerator, T> source)
where TEnumerator : struct, IValueEnumerator<T>
#if NET9_0_OR_GREATER
, allows ref struct
#endif
{
using var e = source.Enumerator;
if (e.TryGetSpan(out var span))
{
return ImmutableArray.Create(span);
}
else
{
// set capacity if can
var builder = e.TryGetNonEnumeratedCount(out var count)
? ImmutableArray.CreateBuilder<T>(count)
: ImmutableArray.CreateBuilder<T>();
while (e.TryGetNext(out var current))
{
builder.Add(current);
}
return builder.ToImmutable();
}
}
}
Since the enumerator's state changes, you cannot call other methods after calling TryGetNext. Also, you cannot call TryGetNext after TryCopyTo or TryGetSpan returns true.
Custom Operator
Unlike IEnumerable<T>, you can't use yield return, so everything must be implemented by hand, making it more difficult than Consume operators. A simple Select implementation looks like this. For .NET 9 or later, IValueEnumerator<T> must be implemented as a ref struct. Also, the accessibility must be public or internal.
public static class MyExtensions
{
public static ValueEnumerable<SimpleSelect<TEnumerator, TSource, TResult>, TResult> SimpleSelect<TEnumerator, TSource, TResult>(this ValueEnumerable<TEnumerator, TSource> source, Func<TSource, TResult> selector)
where TEnumerator : struct, IValueEnumerator<TSource>
#if NET9_0_OR_GREATER
, allows ref struct
#endif
{
// ValueEnumerable is only a wrapper so unwrapping to enumerator immediately is ok.
// `new(new())` is `new ValueEnumerable(new SimpleSelect())`, wrap enumerator to ValueEnumerable.
return new(new(source.Enumerator, selector));
}
}
#if NET9_0_OR_GREATER
public ref struct
#else
public struct
#endif
SimpleSelect<TEnumerator, TSource, TResult>(TEnumerator source, Func<TSource, TResult> selector) : IValueEnumerator<TResult>
where TEnumerator : struct, IValueEnumerator<TSource>
#if NET9_0_OR_GREATER
, allows ref struct
#endif
{
TEnumerator source = source; // need to store source enumerator in field explicitly (ref struct limitation)
public bool TryGetNonEnumeratedCount(out int count)
{
// If source count is not changed, return count.
// Select count is same as source.
return source.TryGetNonEnumeratedCount(out count);
}
public bool TryGetSpan(out ReadOnlySpan<TResult> span)
{
// For example, Take or Skip could return a Slice
span = default;
return false;
}
public bool TryCopyTo(scoped Span<TResult> destination, Index offset)
{
// TryCopyTo implementation needs to consider Index calculations, so it's quite complex.
// Also, destination can be smaller than the source size.
// Helper methods for calculations are available in ZLinq.Internal.EnumeratorHelper,
// such as TryGetSliceRange, TryGetSlice, TryGetSliceRange, TryConsumeGetAt, etc.
return false;
}
// This is the main body of the normal processing
public bool TryGetNext(out TResult current)
{
while (source.TryGetNext(out var value))
{
current = selector(value);
return true;
}
current = default!;
return false;
}
public void Dispose()
{
// Always dispose the source
source.Dispose();
}
}
For TryGetNonEnumeratedCount, TryGetSpan, and TryCopyTo, it's fine to return false if implementation is difficult. If state is needed (for example, Take needs to keep track of the number of calls), place it in a field, but note that you should not initialize reference types or structs containing reference types in the constructor. This is because in method chains, Enumerators are passed by copy, so reference types would share references. If you need to hold reference types, they must be initialized when TryGetNext is first called.
Acknowledgement
Since the preview version release, we have received multiple ideas for fundamental interface revisions leading to performance improvements from @Akeit0, and test and benchmark infrastructure from @filzrev. We are grateful for their many contributions.
