Bar Arnon

Evolution of An Async LINQ operator

Mon, 12 Jul 2021 00:00:00 +0000

LINQ (Language-Integrated Query) has been around for quite a while in the world of .NET (since .NET Framework 3.5 and C# 3.0) but recently async streams (i.e. IAsyncEnumerable<T>) were added to .NET and with them came async LINQ. It allows using the richness of LINQ while reaping the benefits of async-await by asynchronously accessing databases, microservices and remote APIs.

System.Linq.Async provides the standard LINQ operators we’ve all come to expect like Where, Select, GroupBy, etc. as extensions for IAsyncEnumerable but it’s also common for developers to add their own custom ones. Doing that has some pitfalls in “old-school” (synchronous) LINQ but even more so when you throw async-await into the mix.

For example, if we take Where, which receives an enumerable and a predicate to filter the items with, a very naive implementation may be something like (note the new await foreach):

static async Task<List<T>> Where<T>(this IAsyncEnumerable<T> source, Func<T, bool> predicate)
{
    // Create a list to hold the filtered items
    var filteredItems = new List<T>();
    await foreach (var item in source)
    {
        if (predicate(item))
        {
            // Add the filtered item to the list
            filteredItems.Add(item);
        }
    }

    // Return the list containing the filtered items
    return filteredItems;
}

However, that implementation breaks one of the foundational properties of LINQ: Deferred Execution.

Deferred Execution

Deferring the execution means the operation isn’t executed at the point where the method is invoked, it only executes when the enumerable gets enumerated (e.g. with await foreach). The easiest way to fix that would be to utilize C# 8.0’s async iterators:

static async IAsyncEnumerable<T> Where<T>(this IAsyncEnumerable<T> source, Func<T, bool> predicate)
{
    await foreach (var item in source)
    {
        if (predicate(item))
        {
            // Return the filtered item and pause until the next MoveNextAsync call
            yield return item;
        }
    }
}

An async iterator method is a method that:

Returns an IAsyncEnumerable<T>.
Is declared with the async modifier.
Contains yield return (or yield break) statements.

The compiler, behind the scenes, turns this kind of methods into state machines. They allow to start returning items only when the consumer asks for them, which is done by enumerating the returned async enumerable.

So now when Where is called no execution has started yet, it will only start when the await foreach is reached:

IAsyncEnumerable<Hamster> hamsters = // ..
IAsyncEnumerable<Hamster> filteredHamsters = hamsters.Where(hamster => hamster.Name == "Bar");
// No execution has happened yet
await foreach (Hamster hamster in filteredHamsters)
{
    Console.WriteLine(hamster.Age);
}

Using an async iterator also achieves another foundational property of LINQ: Streaming Execution.

Streaming Execution

When the execution is streaming the items are returned one at a time as the consumer asks for them. The iterator doesn’t need to consume the entire source enumerable in order to return the filtered items, it can return them one by one (in fact, the source enumerable can even be infinite).

Now, while we want the main execution of our method to be deferred there is a part we want to run eagerly right when the method is called: argument validation. If something is wrong with one of the arguments passed to our operator method we don’t want to wait until the result is enumerated in order to notify the consumer about it. We want to “fail fast” and throw an exception quickly. Since an async iterator only starts running when the result is enumerated we’ll have to break our method into two.

In order to do that we can utilize another relatively new C# feature: local methods. That way the local method can be an async iterator while the outer method can eagerly validate the arguments:

static IAsyncEnumerable<T> Where<T>(this IAsyncEnumerable<T> source, Func<T, bool> predicate)
{
    // Argument validation
    if (source is null) throw new ArgumentNullException(nameof(source));
    if (predicate is null) throw new ArgumentNullException(nameof(predicate));

    return Core();

    // Local async iterator method
    async IAsyncEnumerable<T> Core()
    {
        await foreach (var item in source)
        {
            if (predicate(item))
            {
                yield return item;
            }
        }
    }
}

Everything up to here is relevant for both async and synchronous LINQ, but implementing an async LINQ operator has some extra things you need to get right, starting with cancellation.

WithCancellation

Since the underlying operations being performed when enumerating async streams are potentially remote asynchronous operations that may take a while to complete it’s beneficial to be able to cancel them when needed (imagine a long DB query for a web request that already timed out). In order to support cancellation the IAsyncEnumerable.GetEnumerator method accepts a CancellationToken, but since people don’t usually call GetEnumerator by themselves (they call it implicitly via await foreach) IAsyncEnumerable has an extension method to “bake in” the CancellationToken into the enumerable:

CancellationToken cancellationToken = //..
IAsyncEnumerable<Hamster> filteredHamsters = // ..
await foreach (var hamster in filteredHamsters.WithCancellation(cancellationToken))
{
    Console.WriteLine(hamster.Age);
}

Cancellation in .NET is cooperative, which means that the consumer may pass a CancellationToken but the code being executed needs to look at that token to observe the cancellation and act upon it. But how can our Where method, which was already invoked and returned an enumerable, access this token being passed afterwards while enumerating the same enumerable? With the C# compiler’s help.

Async iterator methods can accept a CancellationToken annotated with the EnumeratorCancellation attribute. That tells the compiler to create a CancellationToken that gets cancelled whether the consumer passed a token using WithCancellation in the await foreach or whether they passed a token directly when invoking the async iterator method.

All we need it to do is add that CancellationToken parameter to our async iterator method and use it when enumerating the source enumerable:

static IAsyncEnumerable<T> Where<T>(this IAsyncEnumerable<T> source, Func<T, bool> predicate)
{
    if (source is null) throw new ArgumentNullException(nameof(source));
    if (predicate is null) throw new ArgumentNullException(nameof(predicate));

    // Using the CancellationToken parameter's default value
    return Core();

    async IAsyncEnumerable<T> Core([EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await foreach (var item in source.WithCancellation(cancellationToken))
        {
            if (predicate(item))
            {
                yield return item;
            }
        }
    }
}

We can only add that token to the local Core method since the outer method isn’t an async iterator (no async keyword). However, we wouldn’t want to do that even if we could. Imagine calling multiple LINQ operators one after another and passing the same token again and again (e.g. Where(.., token).Select(.., token).OrderBy(.., token)). We want to allow the consumer to pass the token once when enumerating, and it’s on us to make sure the token “moves through the pipe” of all the operators and reach the actual asynchronous operation we want to cancel (e.g. the DB query).

ConfigureAwait(false)

Another uniquely async-await thing it’s important to get right is when to call ConfigureAwait(false). When awaiting inside async methods, unless there’s a good reason not to, you usually want to ignore the SynchronizationContext if there is one. The SynchronizationContext allows to return back to some context (usually the UI thread) after the asynchronous operation being awaited resumes. Using it in places where it’s not needed slightly hurts performance and may lead to deadlocks (more on it here).

So you can use ConfigureAwait(false) when enumerating an IAsyncEnumerable just like you can when awaiting a task:

static IAsyncEnumerable<T> Where<T>(this IAsyncEnumerable<T> source, Func<T, bool> predicate)
{
    if (source is null) throw new ArgumentNullException(nameof(source));
    if (predicate is null) throw new ArgumentNullException(nameof(predicate));

    return Core();

    async IAsyncEnumerable<T> Core([EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await foreach (var item in source.WithCancellation(cancellationToken).ConfigureAwait(false))
        {
            if (predicate(item))
            {
                yield return item;
            }
        }
    }
}

It’s interesting to note that await foreach is pattern-based just like foreach, await and other capabilities in C#. Both the WithCancellation and ConfigureAwait methods return the ConfiguredCancelableAsyncEnumerable struct which doesn’t implement the IAsyncEnumerable interface. But it does have a GetAsyncEnumerator method that returns an enumerator that has MoveNextAsync, Current & DisposeAsync. If follows the required pattern and therefore can be used in await foreach (I’ve explained more about these patterns in C# in an earlier post).

Where, WhereAwait & WhereAwaitWithCancellation

Our Where implementation is now complete, but that method still has some limitations. It allows to filter an async enumerable using a predicate, but that predicate delegate is synchronous (it returns a bool, not a Task<bool>). What if for example, you can only filter the items by calling another microservice? Async LINQ covers these cases as well.

The convention in async LINQ is to have three kinds of “overloads” for each operator:

An implementation that accepts a synchronous delegate (e.g. our Where method).
An implementation that accepts an async delegate (a delegate with an async return type).
- That method should have an Await suffix to signify that the delegates are being awaited (WhereAwait in our case).
An implementation that accepts a cancellable async delegate (an async delegate that accepts a CancellationToken on top of its other parameters).
- That method should have an AwaitWithCancellation suffix (WhereAwaitWithCancellation).

Naming the versions differently helps the consumer to distinguish between the versions, but it also makes overload resolution easier for the compiler and helps it to target type the lambda expressions passed as arguments.

WhereAwait

The signature for WhereAwait is similar to Where, only instead of accepting Func<T, bool> as the predicate it’s now Func<T, ValueTask<bool>>. ValueTask<T> is an awaitable like Task<T> that allows to avoid Task<T> allocations in cases where the operations ran synchronously (I’ve posted about it when it was introduced). Using ValueTask<T> instead of Task<T> helps the consumer to generally avoid some unnecessary allocations which reduces GC pressure.

The implementation is also similar to Where. We just need to await the ValueTask<bool> returned from invoking the predicate, and to ignore the SynchronizationContext with ConfigureAwait(false):

static IAsyncEnumerable<T> WhereAwait<T>(this IAsyncEnumerable<T> source, Func<T, ValueTask<bool>> predicate)
{
    if (source is null) throw new ArgumentNullException(nameof(source));
    if (predicate is null) throw new ArgumentNullException(nameof(predicate));

    return Core();

    async IAsyncEnumerable<T> Core([EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await foreach (var item in source.WithCancellation(cancellationToken).ConfigureAwait(false))
        {
            // Await the ValueTask<bool> returned from the predicate
            if (await predicate(item).ConfigureAwait(false))
            {
                yield return item;
            }
        }
    }
}

IAsyncEnumerable<Hamster> hamsters = // ..
IAsyncEnumerable<Hamster> filteredHamsters = hamsters.WhereAwait(async hamster => await FindHamsterTypeAsync(hamster) == HamsterType.Dwarf);

WhereAwaitWithCancellation

WhereAwaitWithCancellation allows the consumer to not only perform an asynchronous operation inside the predicate, but also to cancel that operation when the enumeration is cancelled. All we need for that to work is to accept the correct delegate and pass our CancellationToken when invoking it:

static IAsyncEnumerable<T> WhereAwaitWithCancellation<T>(this IAsyncEnumerable<T> source, Func<T, CancellationToken, ValueTask<bool>> predicate)
{
    if (source is null) throw new ArgumentNullException(nameof(source));
    if (predicate is null) throw new ArgumentNullException(nameof(predicate));

    return Core();

    async IAsyncEnumerable<T> Core([EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await foreach (var item in source.WithCancellation(cancellationToken).ConfigureAwait(false))
        {
            // Pass the CancellationToken to the predicate itself, on top of the enumerable
            if (await predicate(item, cancellationToken).ConfigureAwait(false))
            {
                yield return item;
            }
        }
    }
}

IAsyncEnumerable<Hamster> hamsters = // ..
IAsyncEnumerable<Hamster> filteredHamsters = hamsters.WhereAwaitWithCancellation(async (hamster, token) => await FindHamsterTypeAsync(hamster, token) == HamsterType.Dwarf);

Summary

It has been common, in my experience, for many projects to add their own custom extensions to LINQ and I expect the same would gradually happen with async LINQ as its adoption grows. With async LINQ (just as with async-await as a whole) it’s a bit easier to shoot yourself in the foot. To avoid that you need to make sure the execution is deferred and streaming, cancellation is observed, ConfigureAwait(false) is used appropriately and the right “overloads” are implemented.

Note: This isn’t how most operators are actually implemented in async LINQ. In order to utilize some optimizations the async iterators are custom implemented, instead of relying on the compiler generated ones. You can see the Where versions’ implementations here.

Introducing AsyncUtilities

Mon, 05 Feb 2018 00:00:00 +0000

I’ve recently (~6 months ago) started collecting various slightly useful utility classes and extension methods related to asynchronous programming into a library: AsyncUtilities. Since there are already many useful tools in the BCL and in existing libraries out there these are quite on the fringe:

Utilities:

ValueTask
AsyncLock
Striped Lock
TaskEnumerableAwaiter
CancelableTaskCompletionSource

Extension Methods:

ContinueWithSynchronously
TryCompleteFromCompletedTask
ToCancellationTokenSource

Utilities:

`ValueTask`

When writing async methods that usually complete synchronously ValueTask<TResult> can be used to avoid allocating a Task<TResult> instance in the synchronous case (I’ve written about it here). There isn’t a non-generic version of ValueTask<TResult> in the BCL. An explanation can be found in the comments for ValueTask<TResult> in the corefx repository:

There is no non-generic version of ValueTask<TResult> as the Task.CompletedTask property may be used to hand back a successfully completed singleton in the case where a Task-returning method completes synchronously and successfully.

However, when you have a tree of async methods without results and they in turn only call methods that return ValueTask<TResult> you need to either allocate Task instances, even if the operations complete synchronously, or you need to remove the async keyword and custom-implement the async mechanism with Task.ContinueWith.

Because I dislike both of these options I made a non-generic ValueTask. Here’s how it can be used:

async ValueTask DrawAsync(string name)
{
    var pen = await GetItemAsync<Pen>("pen");
    var apple = await GetItemAsync<Apple>("apple");

    var applePen = pen.JamIn(apple);
    applePen.Draw();
}

async ValueTask<T> GetItemAsync<T>(string name)
{
    var item = GetFromCache<T>(name);
    if (item != null)
    {
        return item;
    }

    return await GetFromDbAsync<T>(name);
}

`AsyncLock`

When awaiting async operations, there’s no thread-affinity (by default). It’s common to start the operation in a certain thread, and resume in another. For that reason, all synchronization constructs that are thread-affine (i.e. match the lock with a specific thread) can’t be used in an async context. One of these is the simple lock statement (which actually uses Monitor.Enter, Monitor.Exit). For such a case I included AsyncLock which is a simple wrapper over a SemaphoreSlim of 1 (for now, at least):

AsyncLock _lock = new AsyncLock();

async Task ReplaceAsync<T>(string id, T newItem)
{
    using (var dbContext = new DBContext())
    {
        using (await _lock.LockAsync())
        {
            await dbContext.DeleteAsync(id);
            await dbContext.InsertAsync(newItem);
        }
    }
}

Striped Lock

Lock striping is a technique used to reduce contention on a lock by splitting it up to multiple lock instances (stripes) with higher granularity where each key is associated with a certain lock/strip (this, for example, is how locks are used inside ConcurrentDictionary to enable higher concurrency). Using a striped lock is similar in practice to using a Dictionary<TKey, TLock> however that forces the number of locks to match the number of keys, while using a striped lock allows to set the concurrency level independently: A higher degree means more granularity but higher memory consumption and vice versa.

`Striped`

The Striped generic class allows using any kind of lock (with any kind of key). The only necessary things are:

The number of stripes
The lock must have a parameterless constructor, or a delegate for creating it is supplied.
The key must implement GetHashCode and Equals correctly, or an IEqualityComparer<TKey> be supplied.

ThreadSafeCache<string> _cache = new ThreadSafeCache<string>();
Striped<string, object> _lock = Striped.Create<string, object>(Environment.ProcessorCount);

string GetContent(string filePath)
{
    var content = _cache.Get(filePath);
    if (content != null)
    {
        return content;
    }

    lock (_lock[filePath])
    {
        // Double-checked lock
        content = _cache.Get(filePath);
        if (content != null)
        {
            return content;
        }

        content = File.ReadAllText(filePath);
        _cache.Set(content);
        return content;
    }
}

`StripedAsyncLock`

Since this library is mainly for asynchronous programming, and you can’t use a simple synchronous lock for async methods, there’s also a concrete encapsulation for an async lock. It returns an IDisposable so it can be used in a using scope:

ThreadSafeCache<string> _cache = new ThreadSafeCache<string>();
StripedAsyncLock<string> _lock = new StripedAsyncLock<string>(stripes: 100);

async Task<string> GetContentAsync(string filePath)
{
    var content = _cache.Get(filePath);
    if (content != null)
    {
        return content;
    }

    using (await _lock.LockAsync(filePath))
    {
        // Double-checked lock
        content = _cache.Get(filePath);
        if (content != null)
        {
            return content;
        }

        using (var reader = File.OpenText(filePath))
        {
            content = await reader.ReadToEndAsync();
        }

        _cache.Set(content);
        return content;
    }
}

`TaskEnumerableAwaiter`

TaskEnumerableAwaiter is an awaiter for a collection of tasks. It makes the C# compiler support awaiting a collection of tasks directly instead of calling Task.WhenAll first:

HttpClient _httpClient = new HttpClient();

static async Task DownloadAllAsync()
{
    string[] urls = new[]
    {
        "http://www.google.com",
        "http://www.github.com",
        "http://www.twitter.com"
    };

    string[] strings = await urls.Select(url => _httpClient.GetStringAsync(url));
    foreach (var content in strings)
    {
        Console.WriteLine(content);
    }
}

It supports both IEnumerable<Task> & IEnumerable<Task<TResult>> and using ConfigureAwait(false) to avoid context capturing. I’ve written about it more extensively here.

`CancelableTaskCompletionSource`

When you’re implementing asynchronous operations yourself you’re usually dealing with TaskCompletionSource which allows returning an uncompleted task and completing it in the future with a result, exception or cancellation. CancelableTaskCompletionSource joins together a CancellationToken and a TaskCompletionSource by cancelling the CancelableTaskCompletionSource.Task when the CancellationToken is cancelled. This can be useful when wrapping pre async/await custom asynchronous implementations, for example:

Task<string> OperationAsync(CancellationToken cancellationToken)
{
    var taskCompletionSource = new CancelableTaskCompletionSource<string>(cancellationToken);

    StartAsynchronousOperation(result => taskCompletionSource.SetResult(result));

    return taskCompletionSource.Task;
}

void StartAsynchronousOperation(Action<string> callback)
{
    // ...
}

Extension Methods:

`TaskExtensions.ContinueWithSynchronously`

When implementing low-level async constructs, it’s common to add a small continuation using Task.ContinueWith instead of using an async method (which adds the state machine overhead). To do that efficiently and safely you need the TaskContinuationOptions.ExecuteSynchronously and make sure it runs on the ThreadPool:

Task.Delay(1000).ContinueWith(
    _ => Console.WriteLine("Done"),
    CancellationToken.None,
    TaskContinuationOptions.ExecuteSynchronously,
    TaskScheduler.Default);

TaskExtensions.ContinueWithSynchronously encapsulates that for you (with all the possible overloads):

Task.Delay(1000).ContinueWithSynchronously(_ => Console.WriteLine("Done"));

`TaskCompletionSourceExtensions.TryCompleteFromCompletedTask`

When working with TaskCompletionSource it’s common to copy the result (or exception/cancellation) of another task, usually returned from an async method. TryCompleteFromCompletedTask checks the task’s state and complete the TaskCompletionSource accordingly. This can be used for example when there’s a single worker executing the actual operations one at a time and completing TaskCompletionSource instances that consumers are awaiting:

BlockingCollection<string> _urls = new BlockingCollection<string>();
Queue<TaskCompletionSource<string>> _waiters = new Queue<TaskCompletionSource<string>>();
HttpClient _httpClient = new HttpClient();

async Task DownloadAllAsync()
{
    while (true)
    {
        await DownloadAsync(_urls.Take());
    }
}

async Task DownloadAsync(string url)
{
    Task<string> downloadTask = _httpClient.GetStringAsync(url);
    await downloadTask;

    TaskCompletionSource<string> taskCompletionSource = _waiters.Dequeue();
    taskCompletionSource.TryCompleteFromCompletedTask(downloadTask);
}

`TaskExtensions.ToCancellationTokenSource`

It can sometimes be useful to treat an existing task as a signaling mechanism for cancellation, especially when that task doesn’t represent a specific operation but an ongoing state. ToCancellationTokenSource creates a CancellationTokenSource that gets cancelled when the task completes.

For example, TPL Dataflow blocks have a Completion property which is a task to enable the block’s consumer to await its completion. If we want to show a loading animation to represent the block’s operation, we need to cancel it when the block completes and we know that happened when the Completion task completes:

HttpClient _httpClient = new HttpClient();

async Task<IEnumerable<string>> DownloadAllAsync(IEnumerable<string> urls)
{
    var results = new ConcurrentBag<string>();
    var block = new ActionBlock<string>(async url =>
    {
        var result = await _httpClient.GetStringAsync(url);
        results.Add(result);
    });

    var cancellationToken = block.Completion.ToCancellationTokenSource().Token;
    var loadingAnimation = new LoadingAnimation(cancellationToken);
    loadingAnimation.Show();

    foreach (var url in urls)
    {
        block.Post(url);
    }

    await block.Completion;
    return results;
}

ConcurrentDictionary Is Not Always Thread-Safe

Tue, 16 Jan 2018 00:00:00 +0000

ConcurrentDictionary is a thread-safe dictionary implementation but surprisingly (at least to me) not all of its members can be safely used by multiple threads concurrently. The Thread Safety section on the ConcurrentDictionary MSDN article has this to say:

All public and protected members of ConcurrentDictionary<TKey, TValue> are thread-safe and may be used concurrently from multiple threads. However, members accessed through one of the interfaces the ConcurrentDictionary<TKey, TValue> implements, including extension methods, are not guaranteed to be thread safe and may need to be synchronized by the caller.

It is guaranteed that invoking the ConcurrentDictionary methods and properties is thread safe, but since it implements several interfaces (e.g. IDictionary, ICollection) using one of their members (or extension methods for them) isn’t. One of these examples is LINQ’s Enumerable.ToList, which is an extension method for IEnumerable. Using it on a ConcurrentDictionary instance while it’s being mutated by other threads may throw an ArgumentException. Here’s a simple repro:

var dictionary = new ConcurrentDictionary<int, int>();
Task.Run(() =>
{
    var random = new Random();
    while (true)
    {
        var value = random.Next(10000);
        dictionary[value] = value;
    }
});

while (true)
{
    dictionary.ToList();
}

This happens because the extension method calls the List constructor that accepts an IEnumerable and that constructor has an optimization for ICollection (which ConcurrentDictionary implements);

public List(IEnumerable<T> collection) {
    ICollection<T> c = collection as ICollection<T>;
    if( c != null) {
        int count = c.Count;
        if (count == 0)
        {
            _items = _emptyArray;
        }
        else {
            _items = new T[count]; // Gets the dictionary size
            c.CopyTo(_items, 0); // Copy from the dictionary to the array
            _size = count;
        }
    }
    else {
        // ...
    }
}

It first gets the size of the dictionary by invoking Count, then initializes an array in that size and finally calls CopyTo to copy over all the KeyValuePair items from the dictionary to that array. Since the dictionary is mutated by multiple threads, the size can increase (or decrease) after Count is invoked but before CopyTo is. That will result in an ArgumentException when the ConcurrentDictionary tries to access the array outside its bounds.

If you just need a separate collection with the dictionary’s items this exception can be avoided by calling the ConcurrentDictionary.ToArray method instead, which operates in a similar way, but does so after acquiring all the dictionary’s internal locks:

public KeyValuePair<TKey, TValue>[] ToArray()
{
    int locksAcquired = 0;
    try
    {
        AcquireAllLocks(ref locksAcquired);
        int count = 0;
        checked
        {
            for (int i = 0; i < m_tables.m_locks.Length; i++)
            {
                count += m_tables.m_countPerLock[i];
            }
        }

        KeyValuePair<TKey, TValue>[] array = new KeyValuePair<TKey, TValue>[count];

        CopyToPairs(array, 0);
        return array;
    }
    finally
    {
        ReleaseLocks(0, locksAcquired);
    }
}

This method, however, shouldn’t be confused with LINQ’s Enumerable.ToArray extension method, that may throw an ArgumentException just like Enumerable.ToList. ConcurrentDictionary.ToArray wins the overload resolution over Enumerable.ToArray since the latter is an extension method but if the dictionary is used through one of its interfaces (that inherits from IEnumerable) the Enumerable.ToArray method will be called and the exception may be thrown. So the following piece of code doesn’t throw:

var dictionary = new ConcurrentDictionary<int, int>();
Task.Run(() =>
{
    var random = new Random();
    while (true)
    {
        var value = random.Next(10000);
        dictionary[value] = value;
    }
});

while (true)
{
    dictionary.ToArray();
}

But explicitly holding the dictionary in an IDictionary variable does:

IDictionary<int, int> dictionary = new ConcurrentDictionary<int, int>();
Task.Run(() =>
{
    var random = new Random();
    while (true)
    {
        var value = random.Next(10000);
        dictionary[value] = value;
    }
});

while (true)
{
    dictionary.ToArray();
}

Duck Typing And Async/Await

Tue, 02 Jan 2018 00:00:00 +0000

As some of you may know, many of the features in C# light up through “duck typing”. Duck typing is where you accept an object that behaves in a certain way (i.e. has certain methods, properties, etc.) instead of a specific type, or as is usually explained: “If it walks like a duck, and talks like a duck, it’s a duck”.

The foreach statement for example, doesn’t look for types that implement IEnumerable/IEnumerable<T> but instead expects a GetEnumerator method that returns some enumerator type (can be either a class or a struct) that has MoveNext and Current. So while the following code clearly breaks in runtime the compiler has no issues with it:

void Foo()
{
    foreach (var item in new FakeEnumerable())
    {
        Console.WriteLine(item);
    }
}

class FakeEnumerable
{
    public FakeEnumerator GetEnumerator() =>
        new FakeEnumerator();
}

struct FakeEnumerator
{
    public bool MoveNext() =>
        throw new NotImplementedException();

    public object Current =>
        throw new NotImplementedException();
}

One of these features is async/await (or the Task-based Asynchronous Pattern). The compiler doesn’t expect Task or Task<T> specifically, it looks for a GetAwaiter method that returns an awaiter that in turn implements INotifyCompletion and has:

IsCompleted: Enables optimizations when the operation completes synchronously.
OnCompleted: Accepts the callback to invoke when the asynchronous operation completes.
GetResult: Returns the result of the operation (if there is one) and rethrows the exception if one occurred.

GetAwaiter can also be an extension method so it’s possible to turn existing types to awaitables with the right custom awaiter.

I recently made such an awaiter for awaiting a collection of tasks together by calling Task.WhenAll and wrapping the returned Task’s awaiter:

struct TaskEnumerableAwaiter : INotifyCompletion
{
    private TaskAwaiter _awaiter;

    public bool IsCompleted => _awaiter.IsCompleted;

    internal TaskEnumerableAwaiter(IEnumerable<Task> tasks) =>
        _awaiter = Task.WhenAll(tasks).GetAwaiter();

    public void OnCompleted(Action continuation) =>
        _awaiter.OnCompleted(continuation);

    public void GetResult() =>
        _awaiter.GetResult();
}

public static class EnumerableExtensions
{
    public static TaskEnumerableAwaiter GetAwaiter(this IEnumerable<Task> tasks) =>
        new TaskEnumerableAwaiter(tasks);
}

The GetAwaiter extension method for IEnumerable<Task> returning this TaskEnumerableAwaiter means the compiler can treat a collection of tasks as an awaitable and await it just like any other task:

HttpClient _httpClient = new HttpClient();

async Task DownloadAllAsync()
{
    var urls = new []
    {
        "http://www.google.com",
        "http://www.github.com",
        "http://www.twitter.com"
    };

    await urls.Select(DownloadAsync);
}

async Task DownloadAsync(string url)
{
    var content = await _httpClient.GetStringAsync(url);
    Console.WriteLine(content);
}

This may be confusing for developers who aren’t familiar enough with async/await so I can’t recommend this as a best-practice, but if you’re still interested I added this support to my AsyncUtilities library.

It also has support for awaiting an IEnumerable<Task<TResult>>, disabling SynchronizationContext capturing with ConfigureAwait and it utilizes C# 7.2 readonly struct. The complete code can be found on github.

The Issue With Scoped Async Synchronization Constructs

Tue, 21 Feb 2017 00:00:00 +0000

With async/await becoming more and more prevalent in modern code so has the need for async synchronization constructs. Unlike their synchronous counterparts (e.g. Monitor, Mutex, ReaderWriterLock, etc.) .NET doesn’t offer almost any built-in asynchronous synchronization constructs, but it does contain the basic building blocks to build them on your own (mainly Task, TaskCompletionSource<T> and SemaphoreSlim.WaitAsync). Stephen Toub published a series of posts (over 5 years ago) on the Parallel Framework team’s blog demonstrating that by building AsyncSemaphore, AsyncLock, AsyncReaderWriterLock and more.

However, there’s an issue with most implementations of the scoped async synchronization constructs: they usually return a task.

Let’s take AsyncLock built using a SemaphoreSlim as an example. Since a semaphore enables concurrent access up to a certain limit a semaphore with the limit of 1 is essentially a lock. Here’s a simple implementation (a more efficient one can be found here):

class AsyncLock
{
    private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1, 1);

    public async Task<IDisposable> LockAsync()
    {
        await _semaphore.WaitAsync();
        return new Releaser(_semaphore);
    }

    private class Releaser : IDisposable
    {
        private readonly SemaphoreSlim _semaphore;

        public Releaser(SemaphoreSlim semaphore)
        {
            _semaphore = semaphore;
        }

        public void Dispose()
        {
            _semaphore.Release();
        }
    }
}

While we could expose an API of AsyncLock.AcquireAsync and AsyncLock.Release to be used with a try-finally block:

await _lock.AcquireAsync();
try
{
    // critical section...
}
finally
{
    _lock.Release();
}

It’s much simpler to use (as most implementation do) a scoped API that returns a Task<IDisposable>. This allows using a using block to automatically release the lock at the end of the critical section (even when an exception is thrown inside it):

using (await _lock.LockAsync())
{
    // critical section...
}

The subtle issue with this approach is that most developers forget that tasks are by themselves IDisposable. That makes the following code compile perfectly well:

using (_lock.LockAsync())
{
    // critical section...
}

The missing await means the code doesn’t wait for the lock to be acquired and proceeds regardless so the lock doesn’t actually protect anything and the critical section can be run concurrently. Usually when you don’t await a Task inside an async method the compiler will warn you, but not in this case as the returned Task isn’t abandoned and is being used by the using block.

It’s trivially easy to include this bug in your code (I know I have) but quite difficult to realize that once you did, especially since this will only blow up in runtime if you dispose of the task at the exact time when the lock is contended.

My simple solution is not to return Task<IDisposable> from async synchronization constructs, but a new awaitable (TaskWrapper) that doesn’t implement IDisposable. All TaskWrapper needs to do to be an awaitable is to return a valid awaiter, and it does so by returning the TaskAwaiter for the original underlying task:

struct TaskWrapper<T>
{
    private readonly Task<T> _task;

    public TaskWrapper(Task<T> task)
    {
        _task = task;
    }

    public TaskAwaiter<T> GetAwaiter() => _task.GetAwaiter();
}

Since you can’t return something other than Task/Task<T>/void from async methods (at least until C# 7.0 is released supporting arbitrary async returns) it’s simpler to split LockAsync into 2 separate methods, an internal async one that actually implements the locking logic and another that wraps the returned Task<IDisposable> with a TaskWrapper<T>:

public TaskWrapper<IDisposable> LockAsync() =>
    new TaskWrapper<IDisposable>(LockInternalAsync());

private async Task<IDisposable> LockInternalAsync()
{
    await _semaphore.WaitAsync();

    return new Releaser(_semaphore);
}

Doing this prevents our offensive code from compiling as TaskWrapper<IDisposable> doesn’t implement IDisposable and so can’t be used in a using statement, but awaiting it results in an IDisposable (not to be confused with Task<IDisposable>) that can and should be used in that using statement. That way you don’t need to worry whether you’ve missed an await somewhere since the compiler will do that for you.

Next you can find a more efficient (and more complicated) implementation of AsyncLock using TaskWrapper:

class AsyncLock
{
    private readonly SemaphoreSlim _semaphore;
    private readonly IDisposable _releaser;
    private readonly Task<IDisposable> _releaserTask;

    public AsyncLock()
    {
        _semaphore = new SemaphoreSlim(1, 1);
        _releaser = new Releaser(_semaphore);
        _releaserTask = Task.FromResult(_releaser);
    }

    public TaskWrapper<IDisposable> LockAsync()
    {
        var waitTask = _semaphore.WaitAsync();
        return new TaskWrapper<IDisposable>(
            waitTask.IsCompleted
                ? _releaserTask
                : waitTask.ContinueWith(
                    (_, releaser) => (IDisposable)releaser,
                    _releaser,
                    CancellationToken.None,
                    TaskContinuationOptions.ExecuteSynchronously,
                    TaskScheduler.Default));
    }

    private class Releaser : IDisposable
    {
        private readonly SemaphoreSlim _semaphore;

        public Releaser(SemaphoreSlim semaphore)
        {
            _semaphore = semaphore;
        }

        public void Dispose()
        {
            _semaphore.Release();
        }
    }
}

Return Any (Task-Like) Type From An Async Method

Mon, 25 Jul 2016 00:00:00 +0000

Since async/await was added to C# 5.0 you could await any custom awaitable type as long as it follows a specific pattern: has a GetAwaiter method that returns an awaiter that in turn has IsCompleted, OnCompleted and GetResult (more on it here). But the language is stricter when it comes to the return type of an async method. You can only return 3 types from an async method: void, Task and Task<T>.

Task is for async methods that don’t have a result (i.e. procedures) while Task<T> is for async methods that return a result (i.e. functions). That leaves async void methods which mainly exist for backwards-compatibility with event handlers (the void Button_Click(object sender, EventArgs e) kind) and should be avoided elsewhere as unhandled exceptions inside them will crash the entire process.

These compiler rules were recently expanded to also allow returning any custom type from an async method as long as it follows a specific pattern. For example if you have a custom HardTask<T> type for these extra-hard tasks, in order to return it from an async method you would need:

A method builder that enables the compiler to actually build the HardTask<T>. It’s very similar to AsyncTaskMethodBuilder<T> but the main difference is that the Task property returns the custom HardTask<T> instead of Task<T>:

struct HardTaskMethodBuilder<TResult>
{
    public void Start<TStateMachine>(ref TStateMachine stateMachine)
        where TStateMachine : IAsyncStateMachine { ... }
    public void SetStateMachine(IAsyncStateMachine stateMachine) { ... }
    public void SetResult(TResult result) { ... }
    public void SetException(Exception exception) { ... }
    // Returns HardTask<TResult>, not Task<TResult>
    public HardTask<TResult> Task { get; }
    public void AwaitOnCompleted<TAwaiter, TStateMachine>(
        ref TAwaiter awaiter, 
        ref TStateMachine stateMachine)
        where TAwaiter : INotifyCompletion
        where TStateMachine : IAsyncStateMachine { ... }
    public void AwaitUnsafeOnCompleted<TAwaiter, TStateMachine>(
        ref TAwaiter awaiter,
        ref TStateMachine stateMachine) 
        where TAwaiter : ICriticalNotifyCompletion 
        where TStateMachine : IAsyncStateMachine { ... }
}

An AsyncMethodBuilderAttribute on the custom type specifying the method builder to be used with it:

[AsyncMethodBuilder(typeof(HardTaskMethodBuilder<>))]
class HardTask<TResult>
{
    ...
}

The main driver behind this feature is performance as it will enable easier use of ValueTask<T> (which already supports this feature). ValueTask<T> is a discriminated union of T for synchronous cases and Task<T> for asynchronous ones. By being a struct it keeps the synchronous case allocation-free and reduces GC pressure. This is useful for operations that usually complete synchronously but occasionally run asynchronously like async streams or async collections but the best example in my opinion are caches.

When using a cache most results are returned synchronously from the cache but for the few cache-misses you usually need to do some I/O asynchronously. Before ValueTask<T> you could either create a new Task<T> instance in the synchronous case with Task.FromResult (which the GC now needs to collect) or complicate the code and cache the Task objects themselves. By returning a ValueTask<T> you can wrap a T result if it’s available or a Task<T> if you need to execute an asynchronous operation to get it. Implementing this behavior is much easier with the help of the async/await “compiler magic”. For example this HamsterProvider that holds a cache of Hamster objects returned from MongoDB:

class HamsterProvider
{
    ConcurrentDictionary<string, Hamster> _dictionary; // ...
    IMongoCollection<Hamster> _hamsters; // ...

    public ValueTask<Hamster> GetHamsterAsync(string name)
    {
        Hamster hamster;
        if (_dictionary.TryGetValue(name, out hamster))
        {
            // Return synchronously from cache 
            return new ValueTask<Hamster>(hamster);
        }

        // Kick off the asynchronous query
        Task<Hamster> task = _hamsters.Find(_ => _.Name == name).SingleOrDefaultAsync();
        // Add a continuation to cache the result after the query completed
        Task<Hamster> continuation = task.ContinueWith(completedTask => 
        {
            Hamster result = completedTask.Result;
            if (result == null)
            {
                throw new Exception($"Hamster named {result.Name} does not exist.");
            }
        
            _dictionary.TryAdd(result.Name, result);
            return result;
        });

        // Return the not-yet-completed continuation for the caller to await
        return new ValueTask<Hamster>(continuation);
    }
}

Can be refactored into this much more readable code using async/await instead of Task.ContinueWith without adding any allocations or degrading performance:

class HamsterProvider
{
    ConcurrentDictionary<string, Hamster> _dictionary; // ...
    IMongoCollection<Hamster> _hamsters; // ...

    public async ValueTask<Hamster> GetHamsterAsync(string name)
    {
        Hamster hamster;
        if (_dictionary.TryGetValue(name, out hamster))
        {
             // Return synchronously from cache 
            return hamster;
        }

        // Kick off and await the asynchronous query
        hamster = await _hamsters.Find(_ => _.Name == name).SingleOrDefaultAsync();
        if (hamster == null)
        {
            throw new Exception($"Hamster named {hamster.Name} does not exist.");
        }

        // Cache the result
        _dictionary.TryAdd(hamster.Name, hamster);
        // Return the result
        return hamster;
    }
}

While this feature is geared towards ValueTask<T> it’s not limited to it. You can use it with any type you like as long as you implement the builder correctly. For example you can create a task that adds a log before each await, support WinRT’s IAsyncAction (when C# adds extension static methods) or even implement the Maybe monad as Chad Kimes did with NullableTaskLike<T>.

This feature was merged into the dev15-preview-4 branch so I assume this will show up in the next Visual Studio 15 preview (not to be confused with VS 2015) and end up in C# 7.0 when it’s released. If you want to go deeper you can take a look at Lucian Wischik’s feature proposal and Charles Stoner’s implementation PR.

Not All Beginnings Must Have An End

Mon, 13 Jun 2016 00:00:00 +0000

Those of you who are familiar with the Asynchronous Programming Model (APM) know that you need to match every BeginXXX with an EndXXX. In this post I’ll try my best to convince you that if a class implements IDisposable correctly you can forgo calling EndXXX if Dispose was already invoked. I’ve tried doing that before on Stack Overflow without much success and was faced with a lot of pushback.

Asynchronous Programming Model

First, a short explanation on what APM even is. APM is a pattern for writing callback-based asynchronous code. Almost all asynchronous code before .NET 4.5 and async/await was based on APM. It’s a pattern where for each asynchronous operation there’s a BeginXXX method and an EndXXX method (e.g. BeginSend and EndSend). BeginXXX accepts the operation’s parameters and a callback to be invoked when the operation completes and returns an IAsyncResult representing the asynchronous operation. EndXXX (which is usually called inside the callback) accepts the same IAsyncResult returned from BeginXXX and returns the operation’s result (if the operation hadn’t completed yet the call will block until it does). Here’s how it looks in code (I’ll be using UdpClient for examples but it’s the same for TcpClient, Socket, etc.):

byte[] data = // ...
UdpClient udpClient = // ...
udpClient.BeginSend(
    data,
    data.Length,
    asyncResult =>
    {
        try
        {
            udpClient.EndSend(asyncResult); // Observe exceptions
            Console.WriteLine("Sent successfully");
        }
        catch (SocketException exception)
        {
            // Handle...
        }
    },
    null);

It’s important to call EndXXX even when the operation doesn’t return a result to observe exceptions.

Nowadays asynchronous code is usually written with async/await and .NET offers task-returning methods next to the APM method for most BCL types. However, under the hood most of these methods are still APM methods covered up with a task-returning overload using Task.Factory.FromAsync. You can see, for example, in UdpClient.ReceiveAsync the calls to BeginReceive and EndReceive:

public Task<UdpReceiveResult> ReceiveAsync()
{
    return Task<UdpReceiveResult>.Factory.FromAsync(
        (callback, state) => BeginReceive(callback, state),
        (ar) =>
        {
            IPEndPoint remoteEP = null;
            Byte[] buffer = EndReceive(ar, ref remoteEP);
            return new UdpReceiveResult(buffer, remoteEP);
        },
        null);
}

Cancellation

These async methods (which are actually APM methods) usually don’t accept a CancellationToken because it was only added in .NET 4.0 (if they do it may only have an effect before the operation started) and rarely accept a timeout so they are non-cancellable. What you usually do if a task gets “stuck” is just pretend as if they were cancellable by attaching a task to it that you can cancel and abandoning the original task on cancellation, for example:

static Task<UdpReceiveResult> ReceiveAsync(
    this UdpClient udpClient,
    CancellationToken cancellationToken)
{
    // Start the original operation
    Task<UdpReceiveResult> receiveTask = udpClient.ReceiveAsync();

    // Add support for cancellation
    return receiveTask.WithCancellation(cancellationToken);
}

static async Task<T> WithCancellation<T>(
    this Task<T> task,
    CancellationToken cancellationToken)
{
    // Create a self-cancelling TaskCompletionSource 
    var tcs = new TaskCompletionSourceWithCancellation<T>(cancellationToken);

    // Wait for completion or cancellation
    Task<T> completedTask = await Task.WhenAny(task, tcs.Task);
    return await completedTask;
}

WithCancellation uses TaskCompletionSourceWithCancellation which is a useful utility for having a self cancelling TaskCompletionSource. It accepts a CancellationToken and adds a registration to complete the TaskCompletionSource when the token is cancelled.

class TaskCompletionSourceWithCancellation<TResult> : TaskCompletionSource<TResult>
{
    public TaskCompletionSourceWithCancellation(CancellationToken cancellationToken)
    {
        CancellationTokenRegistration registration =
            cancellationToken.Register(() => TrySetResult(default(TResult)));
        
        // Remove the registration after the task completes
        Task.ContinueWith(_ => registration.Dispose()); 
    }
}

That’s all well and good, but that original task (the one returned from ReceiveAsync in this case) is still out there holding on to some state and waiting to be completed. The WithCancellation extension only allows you as the consumer to move forward on cancellation but it can’t actually abort the original task. The only way to get rid of it is to dispose of the instance itself (if it’s IDisposable). Calling Dispose on the instance will go over all the pending operations and complete them by invoking all the callbacks. Since these callbacks almost always call EndXXX (or EndReceive in this case) and the instance was already disposed these tasks will end up with an ObjectDisposedException (which will eventually lead to an UnobservedTaskException).

So, for each of these “stuck” tasks you can either let it linger forever (AKA memory leak) or clean it up with Dispose that results in an exception. Avoiding a memory leak is preferable but too much of these exceptions (>50 per second) could easily hurt your performance.

IDisposable Trumps APM

Now, since calling Dispose on an IDisposable instance cleans all resources up and invoking EndXXX on that instance after Dispose has no effect as it throws an ObjectDisposedException immediately there’s no reason to call EndXXX after cancellation to begin with, especially if these exceptions become a performance issue. You can’t control the implementation of the task-returning wrapper, but you can create one of your own that does accept a CancellationToken but doesn’t call EndXXX if the token is cancelled. That way you can abandon the non-cancellable “stuck” operation, dispose of the instance to clear resources and avoid the unnecessary ObjectDisposedException:

static Task<UdpReceiveResult> UnsafeReceiveAsync(
    this UdpClient udpClient,
    CancellationToken cancellationToken)
{
    return UnsafeFromAsync(
        udpClient.BeginReceive,
        asyncResult =>
        {
            IPEndPoint remoteEP = null;
            byte[] buffer = udpClient.EndReceive(asyncResult, ref remoteEP);
            return new UdpReceiveResult(buffer, remoteEP);
        },
        cancellationToken);
}

This also requires having your own version of Task.Factory.FromAsync because this method is the one that actually invokes EndXXX

static Task<TResult> UnsafeFromAsync<TResult>(
    Func<AsyncCallback, object, IAsyncResult> beginMethod,
    Func<IAsyncResult, TResult> endMethod,
    CancellationToken cancellationToken)
{
    // Create a self-cancelling TaskCompletionSource 
    var taskCompletionSource =
        new TaskCompletionSourceWithCancellation<TResult>(cancellationToken);

    beginMethod(
        asyncResult =>
        {
            try
            {
                // Don't call endMethod if cancellation was requested
                if (!cancellationToken.IsCancellationRequested)
                {
                    taskCompletionSource.TrySetResult(endMethod(asyncResult));
                }
            }
            catch (OperationCanceledException operationCanceledException)
            {
                taskCompletionSource.TrySetCanceled(
                    operationCanceledException.CancellationToken);
            }
            catch (Exception exception)
            {
                taskCompletionSource.TrySetException(exception);
            }
        },
        null);

    return taskCompletionSource.Task;
}

Keep in mind that this pattern is only appropriate in extreme cases where the exception rate has been shown to be too high (hence the Unsafe prefix).

In Summary

Some asynchronous operations can “block” indefinitely (imagine waiting for a response that will never come) and the only way to really cancel them is to dispose of their underlying instance. In that case invoking EndXXX is pointless as it only throws an exception without doing anything. These exceptions are usually not an issue, but when they becomes one, it’s perfectly safe to just avoid calling EndXXX and to let Dispose clear all resources held by the instance.

TPL Dataflow Is The Best Library You're Not Using

Mon, 23 May 2016 00:00:00 +0000

TPL Dataflow is an in-process actor library on top of the Task Parallel Library enabling more robust concurrent programming. It was first introduced in the async-ctp (the preview for async/await) but was eventually released as a standalone nuget package. It abstracts away most of the hard work needed when building asynchronous and/or parallel processing code but I feel most people who might benefit from it aren’t aware of it.

A basic building block in the Dataflow library is the ActionBlock. You simply create it, tell it what to do, start posting items into it and wait untill it’s done. It will buffer the items, start a task to process them sequentially (by default) and end the task when the buffer is empty. Here’s a simple example:

// Create a block with a synchronous action
var block = new ActionBlock<Hamster>(_ => _.Feed());

// Add items to the block and start processing
foreach (Hamster hamster in hamsters)
{
    block.Post(hamster);
}

block.Complete(); // Tell the block to complete and stop accepting new items
await block.Completion; // Asynchronously wait until all items completed processing

However, production-ready code would need to support async/await, concurrency, cancellation and capping the internal buffer to avoid uncontrolled memory growth. ActionBlock (and the rest of TPL Dataflow) support all that right out of the box with an options property-bag:

// Create a block with an asynchronous action
var block = new ActionBlock<Hamster>(
    async _ => await _.FeedAsync(),
    new ExecutionDataflowBlockOptions
    {
        BoundedCapacity = 10000, // Cap the item count
        CancellationToken = cancellationToken, // Enable cancellation
        MaxDegreeOfParallelism = Environment.ProcessorCount, // Parallelize on all cores
    });

// Add items to the block and asynchronously wait if BoundedCapacity is reached
foreach (Hamster hamster in hamsters)
{
    await block.SendAsync(hamster);
}

block.Complete();
await block.Completion;

Notice that we now use SendAsync as it allows to asynchronously wait when there’s no more room in the block for new items as opposed to Post which tries to add an item synchronously and returns false if the block is full. This allows throttling producers when the consumers can’t handle the load. Implementing all these features yourself would take many lines of code, too much time and probably result in a few bugs.

Now, while ActionBlock is the most useful block and alone cover the vast majority of use-cases, there are other blocks like BufferBlock, TransformBlock, BatchBlock, etc. that can be joined together like LEGO to create processing pipelines (i.e. data flows):

// Create a block that receives IngredientBatches and generates Pizzas
var pizzaMakerBlock = new TransformBlock<IngredientBatch, Pizza>(
    _ => Pizza.Make(_.Dough, _.Sauce, _.Toppings));

// Create a block to concurrently and asynchronously deliver the Pizzas
var deliveryBlock = new ActionBlock<Pizza>(
    async _ => await _.DeliverAsync(),
    new ExecutionDataflowBlockOptions
    {
        MaxDegreeOfParallelism = 20, // delivery guys
    });

// Link the blocks together to build a pipeline
pizzaMakerBlock.LinkTo(
    deliveryBlock, 
    new DataflowLinkOptions { PropagateCompletion = true });

// Add items to the first block and let them flow through the pipeline
foreach (IngredientBatch batch in ingredientBatches)
{
    pizzaMakerBlock.Post(batch);
}

pizzaMakerBlock.Complete(); // Completion will propagate to the next block in the pipe
await deliveryBlock.Completion; // It's only necessary to wait for the last block

Dataflow’s simplicity and robustness however don’t seem to lead to popularity and usage. It only has 269 questions on Stack Overflow (>20% of them I answered) and it’s shadowed by Reactive Extensions (which can cover some of the same use-cases). My assumption is that being one of the first parts of .NET to be released as a separate nuget package limited the exposure it would have otherwise received (at least it led the way for present-day .NET Core where all libraries are separate nuget packages). But in my experience when people discover Dataflow and see how it solves their issues they are happy to embrace it.

So, this is me acting as a self-appointed TPL Dataflow evangelist. My entire product (Microsoft ATA) runs inside various ActionBlocks’ delegates and I encourage you to consider how it fits your code and architecture as well.

If you already use TPL Dataflow I would be happy to hear about how and in what codebase in the comments.

Async LINQ To Objects Over MongoDB

Wed, 16 Dec 2015 00:00:00 +0000

I’ve been working with MongoDB for the past few years and lately the guys behind the C# driver are working on some interesting things, especially around async/await support.

This year they released a complete rewrite of the driver with an async-only API. Now since IEnumerator doesn’t support asynchronous enumeration (because MoveNext is synchronous) they used their own enumerator interface called IAsyncCursor which looks like this:

public interface IAsyncCursor<out TDocument> : IDisposable
{
    IEnumerable<TDocument> Current { get; }
    Task<bool> MoveNextAsync(
        CancellationToken cancellationToken = default(CancellationToken));
}

To enable asynchronous iteration without having to implement it yourself they added a subset of the LINQ operators, but only those that materialize the query (e.g. ForEachAsync, ToListAsync, etc.)

In the latest version of the driver (v2.1) they added full “LINQ to MongoDB” support (i.e. IQueryable) that you can use to create your mongo queries. Just like with LINQ to SQL you can use Where, Select and so forth to build a complex expression-based query and have it sent to the server only when you start enumerating the IQueryable.

The problem arises when you need an expression to be executed on the client-side and not be sent to the server. This is mainly relevant for expressions that can’t be translated into a mongo query. For example the following code comparing two of the queried document’s fields (which is unsupported in MongoDB) will throw an exception saying: “Unsupported filter: ([FirstName] == [LastName]).”

async Task LinqToMongo()
{
    IMongoCollection<Hamster> collection = GetCollection();
    IMongoQueryable<Hamster> queryable = collection.AsQueryable();
    queryable = queryable.Where(_ => _.FirstName == _.LastName); // unsupported filter
    Console.WriteLine(await queryable.CountAsync());
}

Normally you would just cast the IQueryable into an IEnumerable (with AsEnumerable) and use LINQ to Objects which also supports deferred execution. However, since IEnumerable is synchronous doing that defeats the whole purpose of using async/await to begin with. You could also materialize the entire collection into memory and then use client-side filters but that can take too much memory and time.

A friend of mine (Tsach Vayness) suggested finding an existing library with async LINQ to Objects support and plugging it into the MongoDB C# driver. That enables using all the LINQ to Objects operators over MongoDB. There are a few of these libraries and the best, in my opinion, is Reactive Extensions’ Interactive Extensions (Ix-Async on nuget.org).

Edit (2021-12):

Hey, it’s me from the future. In C# 8.0 async streams (i.e. IAsyncEnumerables) were added to support truly asynchronous enumeration. That makes the desire to treat MongoDB’s IAsynCursor as a “regular” async stream you can use with async LINQ and await foreach. So I made a small library allowing just that: MongoAsyncEnumerableAdapter. It’s much more robust and safe than the implementation below and I encourage you to use it instead. The custom implementation deep dive below is still interesting so I also encourage you to keep reading.

All that’s needed is an adapter from mongo’s IAsyncCursorSource, IAsyncCursor to Interactive Extensions’ IAsyncEnumerable, IAsyncEnumerator (which are already pretty similar) and then you can use all of Ix’s operators on the MongoDB cursors. Here’s the previous example comparing two of the queried document’s fields fixed by moving the filter to the client-side:

async Task LinqToObjects()
{
    IMongoCollection<Hamster> collection = GetCollection();
    IMongoQueryable<Hamster> queryable = collection.AsQueryable();
    IAsyncEnumerable<Hamster> asyncEnumerable = queryable.ToAsyncEnumerable();
    asyncEnumerable = asyncEnumerable.Where(_ => _.FirstName == _.LastName);
    Console.WriteLine(await asyncEnumerable.Count());
}

Most of the “magic” enabling this happens in AsyncEnumeratorAdapter.MoveNext. First, you create an IAsyncCursor out of the IAsyncCursorSource in an async fashion with ToCursorAsync (which is possible because MoveNext returns a Task). Then you call (and await) MoveNext on the created _asyncCursor. If it returned true then _asyncCursor.Current contains a batch of items you can enumerate and call _asyncCursor.MoveNext again when the batch is completed. You repeat that continually until the underlying MoveNext returns false meaning there are no more items to enumerate.

public async Task<bool> MoveNext(CancellationToken cancellationToken)
{
    if (_asyncCursor == null)
    {
        _asyncCursor = await _asyncCursorSource.ToCursorAsync(cancellationToken);
    }
    if (_batchEnumerator != null && _batchEnumerator.MoveNext())
    {
        return true;
    }
    if (_asyncCursor != null && await _asyncCursor.MoveNextAsync(cancellationToken))
    {
        _batchEnumerator?.Dispose();
        _batchEnumerator = _asyncCursor.Current.GetEnumerator();
        return _batchEnumerator.MoveNext();
    }
    return false;
}

Here is the full code for the adapters:

public static class AsyncCursorSourceExtensions
{
    public static IAsyncEnumerable<T> ToAsyncEnumerable<T>(
        this IAsyncCursorSource<T> asyncCursorSource) => 
        new AsyncEnumerableAdapter<T>(asyncCursorSource);

    private class AsyncEnumerableAdapter<T> : IAsyncEnumerable<T>
    {
        private readonly IAsyncCursorSource<T> _asyncCursorSource;

        public AsyncEnumerableAdapter(IAsyncCursorSource<T> asyncCursorSource)
        {
            _asyncCursorSource = asyncCursorSource;
        }

        public IAsyncEnumerator<T> GetEnumerator() => 
            new AsyncEnumeratorAdapter<T>(_asyncCursorSource);
    }

    private class AsyncEnumeratorAdapter<T> : IAsyncEnumerator<T>
    {
        private readonly IAsyncCursorSource<T> _asyncCursorSource;
        private IAsyncCursor<T> _asyncCursor;
        private IEnumerator<T> _batchEnumerator;

        public T Current => _batchEnumerator.Current;

        public AsyncEnumeratorAdapter(IAsyncCursorSource<T> asyncCursorSource)
        {
            _asyncCursorSource = asyncCursorSource;
        }

        public async Task<bool> MoveNext(CancellationToken cancellationToken)
        {
            if (_asyncCursor == null)
            {
                _asyncCursor = await _asyncCursorSource.ToCursorAsync(cancellationToken);
            }
            
            if (_batchEnumerator != null &&
                _batchEnumerator.MoveNext())
            {
                return true;
            }
            
            if (_asyncCursor != null &&
                await _asyncCursor.MoveNextAsync(cancellationToken))
            {
                _batchEnumerator?.Dispose();
                _batchEnumerator = _asyncCursor.Current.GetEnumerator();
                return _batchEnumerator.MoveNext();
            }
            
            return false;
        }

        public void Dispose()
        {
            _asyncCursor?.Dispose();
            _asyncCursor = null;
        }
    }
}

On The Efficiency Of ValueTask

Mon, 30 Nov 2015 00:00:00 +0000

The corefxlab repository contains library suggestions for corefx which itself is a repository containing the .NET Core foundational libraries. One of the gems hidden among these libraries is ValueTask<T> that was added by Stephen Toub as part of the System.Threading.Tasks.Channels library but may be extremely useful on its own. The full implementation of ValueTask<T> can be found here, but this is an interesting subset of the API:

public struct ValueTask<TResult>
{
    public ValueTask(TResult result);
    public ValueTask(Task<TResult> task);
    public static implicit operator ValueTask<TResult>(Task<TResult> task);
    public static implicit operator ValueTask<TResult>(TResult result);
    public Task<TResult> AsTask();
    public bool IsCompletedSuccessfully { get; }
    public TResult Result { get; }
    public ValueTaskAwaiter GetAwaiter();
    public ValueTaskAwaiter ConfigureAwait(bool continueOnCapturedContext);
    // ...
}

I first noticed ValueTask<T> in the API documentation when reviewing the channels PR made to corefxlab. I suggested adding a short explanation which Stephen quickly provided:

“ValueTask<T> is a discriminated union of a T and a Task<T>, making it allocation-free for ReadAsync<T> to synchronously return a T value it has available (in contrast to using Task.FromResult<T>, which needs to allocate a Task<T> instance). ValueTask<T> is awaitable, so most consumption of instances will be indistinguishable from with a Task<T>.”

ValueTask, being a struct, enables writing async methods that do not allocate memory when they run synchronously without compromising API consistency. Imagine having an interface with a Task returning method. Each class implementing this interface must return a Task even if they happen to execute synchronously (hopefully using Task.FromResult). You can of course have 2 different methods on the interface, a synchronous one and an async one but this requires 2 different implementations to avoid “sync over async” and “async over sync”.

ValueTask<T> has implicit casts from both T and Task<T> (EDIT: The implicit casts were removed to prepare for arbitrary async returns) and can be awaited by itself which makes it extremely simple to use. Consider this possibly async provider API returning a ValueTask<T>:

interface IHamsterProvider
{
    ValueTask<Hamster> GetHamsterAsync(string name);
}

This provider interface can be implemented synchronously (for in-memory hamsters) by performing a lookup in a Dictionary and returning the Hamster instance (which is implicitly converted into a ValueTask<Hamster> without any additional allocations):

class LocalHamsterProvider : IHamsterProvider
{
    ConcurrentDictionary<string, Hamster> _dictionary; // ...

    public ValueTask<Hamster> GetHamsterAsync(string name)
    {
        Hamster hamster = _dictionary[name];
        return hamster;
    }
}

Or asynchronously (for hamsters stored in MongoDB) by performing an asynchronous query and returning the Task<Hamster> instance (which is implicitly converted into ValueTask<Hamster> as well):

class MongoHamsterProvider : IHamsterProvider
{
    IMongoCollection<Hamster> _collection; // ...

    public ValueTask<Hamster> GetHamsterAsync(string name)
    {
        Task<Hamster> task = _collection.Find(_ => _.Name == name).SingleAsync();
        return task;
    }
}

The consumer of this API can await the ValueTask<Hamster> as if it was a Task<Hamster> without knowing whether it was performed asynchronously or not with the benefit of no added allocations:

Hamster hamster = await Locator.Get<IHamsterProvider>().GetHamsterAsync("bar");

While this example shows 2 different implementations, one synchronous and the other asynchronous, they could easily be combined. Imagine a provider using a local cache for hamsters and falling back to the DB when needed. The few truly asynchronous calls to GetHamsterAsync would indeed require allocating a Task<Hamster> (which will be implicitly converted to ValueTask<Hamster>) but the rest would complete synchronously and allocation-free:

class HamsterProvider : IHamsterProvider
{
    ConcurrentDictionary<string, Hamster> _dictionary; // ...
    IMongoCollection<Hamster> _collection; // ...

    public ValueTask<Hamster> GetHamsterAsync(string name)
    {
        Hamster hamster;
        if (_dictionary.TryGetValue(name, out hamster))
        {
            return hamster;
        }
        Task<Hamster> task = _collection.Find(_ => _.Name == name).SingleAsync();
        task.ContinueWith(_ => _dictionary.TryAdd(_.Result.Name, _.Result));
        return task;
    }
}

This kind of “hybrid” use of async-await is very common, since these outbound operations to a data store, remote API, etc. can usually benefit from some kind of caching.

It’s hard to quantify how much of an improvement widespread usage of ValueTask can bring as it greatly depends on the actual usage but avoiding allocations not only saves the allocation cost but also greatly reduces the garbage collection overhead. That’s why it was requested to be added to .NET Core regardless of the channels library.

Protobuf-net Is Broken Around DateTime

Sat, 03 Oct 2015 00:00:00 +0000

Protocol Buffers by Google are a great mechanism for serializing (and deserializing) structured data in a very fast and efficient way. Protobuf-net is Marc Gravell’s port of Protocol Buffers for the .NET ecosystem.

While being very efficient, protobuf-net has a big issue when deserializing .NET’s DateTimes. Behind the scenes DateTimes are converted into Unix-Time which is a count (of ticks in this case) starting from the Unix Epoch (1970/01/01 UTC). When deserializing back to .NET protobuf-net adds that count to a DateTime representing the Epoch-Time resulting in the correct DateTime value. The issue with this process is that it loses the DateTime’s original DateTimeKind.

DateTimeKind is an enum telling whether the DateTime’s value represents a local time, UTC time or unspecified. That value isn’t serialized by protobuf-net so all DateTimes, be they local time or UTC, are deserialized as DateTimeKind.Unspecified.

DateTimeKind.Unspecified values have a behavior that I initially found surprising but later realized is the best possible option. Let’s assume you’re in Hawaii (because where else would you want to be?) and your time zone is UTC-10:00. If you have a DateTime value with DateTimeKind.Unspecified and you call ToLocalTime the method assumes the value is in UTC and “corrects” it, so 11:00 becomes 01:00. If however you call ToUniversalTime on that value the method now assumes it’s in local time and “corrects” it so 11:00 becomes 21:00. So the same value is treated as local while adjusting to universal and universal when adjusting to local. Let’s see that in code:

static void Main()
{
    var dolly = new Sheep { DateOfBirth = new DateTime(1966, 07, 05, 11, 0, 0, DateTimeKind.Utc) };
    Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z" (Z means UTC)

    dolly = Serializer.DeepClone(dolly); // Serialize and deserialize using protobuf-net
    Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00" (no Z means unspecified)
    Console.WriteLine(dolly.DateOfBirth.ToLocalTime().ToString("HH:mm:ss K")); // "01:00:00 -10:00" (Hawaii timezone)
    Console.WriteLine(dolly.DateOfBirth.ToUniversalTime().ToString("HH:mm:ss K")); // "21:00:00 Z"
}

[ProtoContract(ImplicitFields = ImplicitFields.AllPublic)]
class Sheep
{
    public DateTime DateOfBirth { get; set; }
}

This can get extremely problematic especially if you, like me, depend upon some library that uses ToUniversalTime or ToLocalTime. For me that library was the .NET’s MongoDB Driver that stores all DateTimes in MongoDB in UTC. Using these 2 libraries together is impossible as DateTime values would keep changing value infinitely.

I have posted on the protobuf-net repository on github explaining this and managed to convince him to fix this problem (which he did with this commit). However, this fix was made 5 months prior to me writing this post and there’s still isn’t a new release including the fix (the latest stable release is from 2013/09/30).

But don’t fear… I do have a workaround you can use for the meantime. Protobuf-net uses a DateTime value representing the Unix Epoch (1970/01/01) to create all the DateTimes by adding the relevant delta to the epoch value. Since creating a new DateTime from an existing DateTime preserves the DateTimeKind, replacing the single epoch value with a UTC one will result with all protobuf-net DateTime values having DateTimeKind.UTC. We can do that by using reflection and replacing the epoch value with a UTC one:

typeof (BclHelpers).
    GetField("EpochOrigin", BindingFlags.NonPublic | BindingFlags.Static).
    SetValue(null, new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc));

var dolly = new Sheep { DateOfBirth = new DateTime(1966, 07, 05, 11, 0, 0, DateTimeKind.Utc) };
Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z" (Z means UTC)

dolly = Serializer.DeepClone(dolly); // Serialize and deserialize using protobuf-net
Console.WriteLine(dolly.DateOfBirth.ToString("HH:mm:ss K")); // "11:00:00 Z"

I admit it’s not pretty, but until Marc releases a new version, it’s preferable to building your own protobuf-net.

Surprising Contention In System.Threading.Timer

Sat, 25 Jul 2015 00:00:00 +0000

While profiling our application’s performance we stumbled upon a surprising contention point inside System.Threading.Timer (I say surprising as System.Threading.Timer is the more appropriate timer for multi-threaded environments out of the available timers)

This can be demonstrated by executing the following piece of code:

static void Main()
{
    for (var i = 0; Environment.ProcessorCount > i; i++)
    {
        Task.Factory.StartNew(() =>
        {
            while (true)
            {
                new Timer(
                    _ => { },
                    null,
                    TimeSpan.FromMilliseconds(100),
                    Timeout.InfiniteTimeSpan);
            }
        }, TaskCreationOptions.LongRunning);
    }

    Console.ReadLine();
}

It starts a thread for each core on the machine that creates timers in a loop (as long as memory allows). If you open Performance Monitor while this runs you would see very high values for the “Contention Rate / sec” performance counter (more than 1000 per second on my 4 core machine).

Looking at the code you can see that when you create a new Timer (or change an existing timer) the internal Timer.Change method takes a lock on a singleton instance of TimerQueue. Since this is a simple lock (i.e. Monitor) on a singleton instance, every Timer you create in your entire AppDomain is contention waiting to happen:

internal bool Change(uint dueTime, uint period)
{
    bool success;

    lock (TimerQueue.Instance) // Global lock.
    {
        if (m_canceled)
            throw new ObjectDisposedException(
                null,
                Environment.GetResourceString("ObjectDisposed_Generic"));

        // prevent ThreadAbort while updating state
        try { }
        finally
        {
            m_period = period;

            if (dueTime == Timeout.UnsignedInfinite)
            {
                TimerQueue.Instance.DeleteTimer(this);
                success = true;
            }
            else
            {
                if (FrameworkEventSource.IsInitialized &&
                    FrameworkEventSource.Log.IsEnabled(
                        EventLevel.Informational,
                        FrameworkEventSource.Keywords.ThreadTransfer))
                    FrameworkEventSource.Log.ThreadTransferSendObj(
                        this,
                        1,
                        string.Empty,
                        true);

                success = TimerQueue.Instance.UpdateTimer(this, dueTime, period);
            }
        }
    }

    return success;
}

The notes for that TimerQueue class show that the fact that timers are created quite often was taken into consideration. Especially in this part:

“Perf assumptions: We assume that timers are created and destroyed frequently, but rarely actually fire.”

By looking at this other part:

“TimerQueue maintains a list of active timers in this AppDomain. We use a single native timer, supplied by the VM to schedule all managed timers in the AppDomain.”

I assume they could have created several of these native timers to reduce the contention (but maybe that’s not the case).

Now, you might say that you don’t really create many of these timers, but a modern .NET server using async/await extensively creates many timers hidden away from sight. For example, every single call to Task.Delay creates a timer internally which is disposed when the task completes. It’s also common to create a CancellationTokenSource that automatically cancels itself after a certain interval using an internal Timer.

For us the main issue was CancellationTokenSources created with a TimeSpan used to timeout our many (>100/s) I/O operations. Since the timeout for these was always the same I made a utility class that uses a single Timer and performs an action on all the items (CancellationTokenSources in this case) that their timeout expired:

public class CollectiveTimer<T>
{
    private readonly ConcurrentQueue<QueueItem> _queue;
    
    public CollectiveTimer(
        Action<T> action,
        TimeSpan timeout,
        CancellationToken cancellationToken)
    {
        _queue = new ConcurrentQueue<QueueItem>();
        Task.Run(async () =>
        {
            while (!cancellationToken.IsCancellationRequested)
            {
                QueueItem queueItem;
                var now = DateTime.UtcNow;
                while (_queue.TryPeek(out queueItem) && now - queueItem.Time >= timeout)
                {
                    _queue.TryDequeue(out queueItem);
                    action(queueItem.Item);
                }

                await Task.Delay(TimeSpan.FromMilliseconds(50));
            }
        });
    }
    
    public void Enqueue(T item) =>
        _queue.Enqueue(new QueueItem(item, DateTime.UtcNow));
    
    private sealed class QueueItem
    {
        public T Item { get; private set; }
        public DateTime Time { get; private set; }
        public QueueItem(T item, DateTime time)
        {
            Item = item;
            Time = time;
        }
    }
}

Usage:

var collectiveTimer = new CollectiveTimer<CancellationTokenSource>(
    cts => cts.Cancel(), 
    TimeSpan.FromMilliseconds(200),
    cancellationToken);

This made our contention completely disappear (at least until the next one)

LongRunning Is Useless For Task.Run With Async/Await

Thu, 02 Jul 2015 00:00:00 +0000

Back in the olden days of .NET 4.0 we didn’t have Task.Run. All we had to start a task was the complicated Task.Factory.StartNew. Among its parameters there’s a TaskCreationOptions often used to specify TaskCreationOptions.LongRunning. That flag gives TPL a hint that the task you’re about to execute will be longer than usual.

Nowadays with .NET 4.5 and above we mostly use the simpler and safer Task.Run but it isn’t uncommon to wonder how do you pass TaskCreationOptions.LongRunning as a parameter like we used to do with Task.Factory.StartNew.

The answer is that you can’t. This of course isn’t limited just to TaskCreationOptions.LongRunning. You can’t pass any of the TaskCreationOptions values. However most of them (like TaskCreationOptions.AttachedToParent) are there for extremely esoteric cases while TaskCreationOptions.LongRunning is there for your run-of-the-mill long running task.

An often suggested workaround is to go back to Task.Factory.StartNew, which is perfectly fine for synchronous delegates (i.e. Action, Func<T>), however for asynchronous delegates (i.e. Func<Task>, Func<Task<T>>) there’s the whole Task<Task> confusion. Since Task.Factory.StartNew doesn’t have specific overloads for async/await asynchronous delegates map to the Func<T> where T is a Task. That makes the return value a Task<T> where T is a Task, hence Task<Task>.

The .NET team anticipated this issue and it can be easily solved by using TaskExtensions.Unwrap (which is the accepted answer on the relevant Stack Overflow question):

Task<Task> task = Task.Factory.StartNew(async () =>
{
    while (IsEnabled)
    {
        await FooAsync();
        await Task.Delay(TimeSpan.FromSeconds(10));
    }
}, TaskCreationOptions.LongRunning);

Task actualTask = task.Unwrap();

However that hides the actual issue which is:

Task.Run with TaskCreationOptions.LongRunning doesn’t make sense for async/await.

The internal implementation creates a new dedicated thread when you use TaskCreationOptions.LongRunning. Here’s the code for ThreadPoolTaskScheduler.QueueTask:

protected internal override void QueueTask(Task task)
{
    if ((task.Options & TaskCreationOptions.LongRunning) != 0)
    {
        // Run LongRunning tasks on their own dedicated thread.
        Thread thread = new Thread(s_longRunningThreadWork);
        thread.IsBackground = true; // Keep this thread from blocking process shutdown
        thread.Start(task);
    }
    else
    {
        // Normal handling for non-LongRunning tasks.
        bool forceToGlobalQueue = (task.Options & TaskCreationOptions.PreferFairness) != 0;
        ThreadPool.UnsafeQueueCustomWorkItem(task, forceToGlobalQueue);
    }
}

But when an async method reaches an await for an uncompleted task the thread it’s running on is released. When the task is completed the rest will be scheduled again, this time on a different ThreadPool thread. That means that we created a new thread needlessly. This doesn’t only waste time for the developer but also hurts performance as creating new threads is costly (otherwise we wouldn’t need the ThreadPool in the first place).

So, if you ask yourself how to use Task.Run with TaskCreationOptions.LongRunning when your delegate is asynchronous, save yourself and your application some time and keep using Task.Run as it is:

Task task = Task.Run(async () =>
{
    while (IsEnabled)
    {
        await FooAsync();
        await Task.Delay(TimeSpan.FromSeconds(10));
    }
});

LogicalOperationStack Is Broken With Async/Await

Sat, 16 May 2015 00:00:00 +0000

Trace.CorrelationManager.LogicalOperationStack enables having nested logical operation identifiers where the most common case is logging (NDC). Evidently it doesn’t work with async/await.

The Issue

Here’s a simple example using LogicalFlow, my simple wrapper over the LogicalOperationStack:

static void Main()
{
    OuterOperationAsync().Wait();
}

static async Task OuterOperationAsync()
{
    Console.WriteLine(LogicalFlow.CurrentOperationId);
    using (LogicalFlow.StartScope())
    {
        Console.WriteLine("\t" + LogicalFlow.CurrentOperationId);
        await InnerOperationAsync();
        Console.WriteLine("\t" + LogicalFlow.CurrentOperationId);
        await InnerOperationAsync();
        Console.WriteLine("\t" + LogicalFlow.CurrentOperationId);
    }
    Console.WriteLine(LogicalFlow.CurrentOperationId);
}

static async Task InnerOperationAsync()
{
    using (LogicalFlow.StartScope())
    {
        await Task.Yield();
    }
}

LogicalFlow:

public static class LogicalFlow
{
    public static Guid CurrentOperationId
    {
        get
        {
            return Trace.CorrelationManager.LogicalOperationStack.Count > 0
                ? (Guid) Trace.CorrelationManager.LogicalOperationStack.Peek()
                : Guid.Empty;
        }
    }

    public static IDisposable StartScope()
    {
        Trace.CorrelationManager.StartLogicalOperation();
        return new Stopper();
    }

    private static void StopScope()
    {
        Trace.CorrelationManager.StopLogicalOperation();
    }

    private class Stopper : IDisposable
    {
        private bool _isDisposed;
        public void Dispose()
        {
            if (!_isDisposed)
            {
                StopScope();
            }
        }
    }
}

And the output is:

00000000-0000-0000-0000-000000000000
    49985135-1e39-404c-834a-9f12026d9b65
    54674452-e1c5-4b1b-91ed-6bd6ea725b98
    c6ec00fd-bff8-4bde-bf70-e073b6714ae5
54674452-e1c5-4b1b-91ed-6bd6ea725b98

The specific Guid values don’t really matter. Both the outer lines should show Guid.Empty (i.e. 00000000-0000-0000-0000-000000000000) and the inner lines should show the same Guid value.

Multithreading

You might say that LogicalOperationStack is using a Stack which is not thread-safe and that’s why the output is wrong. time** because every async operation is awaited when called and there’s no use of combinators such as Task.WhenAll.

The Root Cause

The root cause is that LogicalOperationStack is stored in the CallContext which has a copy-on-write behavior. That means that as long as you don’t explicitly set something into the CallContext (and you don’t with StartLogicalOperation as you just add to an existing stack) you’re using the parent context and not your own.

This can be shown by simply setting anything into the CallContext before adding to the existing stack. For example if we changed StartScope to this:

public static IDisposable StartScope()
{
    CallContext.LogicalSetData("Bar", "Arnon");
    Trace.CorrelationManager.StartLogicalOperation();
    return new Stopper();
}

The output is correct:

00000000-0000-0000-0000-000000000000
    fdc22318-53ef-4ae5-83ff-6c3e3864e37a
    fdc22318-53ef-4ae5-83ff-6c3e3864e37a
    fdc22318-53ef-4ae5-83ff-6c3e3864e37a
00000000-0000-0000-0000-000000000000

I’ve contacted the relevant developer at Microsoft and his response was this:

“I wasn’t aware of this, but it does seem broken. The copy-on-write logic is supposed to behave exactly as if we’d really created a copy of the ExecutionContext on entry into the method. However, copying the ExecutionContext would have created a deep copy of the CorrelationManager context, as it’s special-cased in CallContext.Clone(). We don’t take that into account in the copy-on-write logic.”

Solution

Use an ImmutableStack stored in the CallContext instead of the LogicalOperationStack as it’s both thread-safe and immutable so when you call Pop you get back a new ImmutableStack that you then must set back into the CallContext.

public static class LogicalFlow
{
    private static ImmutableStack<Guid> LogicalStack
    {
        get
        {
            return CallContext.LogicalGetData("LogicalFlow") as ImmutableStack<Guid> ??
                ImmutableStack.Create<Guid>();
        }
        set
        {
            CallContext.LogicalSetData("LogicalFlow", value);
        }
    }

    public static Guid CurrentId
    {
        get
        {
            var logicalStack = LogicalStack;
            return logicalStack.IsEmpty ? Guid.Empty : logicalStack.Peek();
        }
    }
    
    public static IDisposable StartScope()
    {
        // Here's where the CallContext is copied using copy-on-write
        LogicalStack = LogicalStack.Push(Guid.NewGuid());
        return new Stopper();
    }
    
    private static void StopScope() =>
        LogicalStack = LogicalStack.Pop();
}

Another options is to store the stack in the new System.Threading.AsyncLocal class added in .NET 4.6 (or Stephen Cleary’s AsyncLocal) instead which should handle that issue correctly.

Note: This came after I asked (and answered) a question on Stack Overflow: Is LogicalOperationStack incompatible with async in .NET 4.5 which has further discussion.