Draft TOC for EC++11 Concurrency Chapter

A couple of months ago, I posted a draft Table of Contents (TOC) for Effective C++11. At that point, the entries for the concurrency chapter were so rough, they weren't even in the form of guidelines. Now they are, and I'm pleased to unveil my first draft TOC for the chapter on concurrency support:

• Create tasks, not threads.
• Pass std::launch::async if asynchronicity is essential.
• Make std::threads unjoinable on all paths.
• Be aware of varying thread handle destructor behavior.
• Consider void futures for one-shot event communication.
• Pass parameterless functions to std::thread, std::async, and std::call_once.
• Use std::lock to acquire multiple locks.
• Prefer non-recursive mutexes to recursive ones.
• Declare future and std::thread members last.
• Code for spurious failures in try_lock, condvar wait, and weak CAS operations.
• Distinguish steady from unsteady clocks.
• Use native handles to transcend the C++11 API.
• Employ sequential consistency if at all possible.
• Distinguish volatile from std::atomic.

This is a draft TOC. There's nothing final about the presence, order, or wording of these Items. Furthermore, unless either your mind-reading skills are better than I expect or my mind is easier to read than I fear, it will be tough for you to anticipate what I plan to say in these Items based only on the Item titles. Still, if you see advice above that you think is either especially good or especially bad, don't be shy about letting me know about it.

I'm especially pleased with the first Item on the list ("Create tasks, not threads"), because when I came up with that wording, a number of up-to-that-point disparate thoughts fell into place with a very satisfying thud.

When I began that Item, the only thing I knew I wanted to talk about was that thread construction can throw.  In Effective C++, Second Edition, my advice about dealing with the fact that operator new can throw is "Be prepared for out-of-memory conditions," so I started thinking about guidance such as "Be prepared for std::thread exhaustion." But what does it mean to be prepared to run out of threads? With operator new, there's a new handler you can configure. There's nothing like that for thread creation. And if you request n bytes from operator new and you can't get it, you may be able to scale down your request to, say, n/2 bytes, then try again. But if you request a new thread and that fails, what are you supposed to do, request half a thread?

I didn't like where that was going.  So I decided to think about avoiding the problem of running out of threads by not requesting them directly.  The prospective guideline "Prefer std::async to std::thread" had been an elephant in the room from the beginning, so I started playing with that idea.  But one of the other guidelines I was considering was "Pass std::launch::async if asynchronicity is essential" (it's on the draft TOC above), and the spec for std::async says that it throws the same exception as the std::thread constructor if you pass std::launch::async as the launch policy and std::async can't create a new thread. So advising people to use std::async was not sufficient, because using std::async with std::launch::async is no better than using std::thread for purposes of avoiding out-of-thread exceptions.

Though my primary focus had been on figuring out how to avoid exceptions due to too many threads, another issue I wanted to address was how to deal with oversubscription: creating more threads than can efficiently run on the machine. The way to avoid that problem is to use std::async with the default launch policy, and that got me to thinking about what to call a function (or function object--henceforth simply a "function") that could be run either synchronously or asynchronously.  A raw function doesn't qualify, because if you run a raw function asynchronously on a std::thread, there is no way to get the result of the function.  (And if the function throws an exception, std::terminate gets called.) Fortunately, C++11 offers a way to prepare a function for possible asynchronous execution: wrap it in std::packaged_task. How fortuitous! I had been looking for an excuse to discuss std::packaged_task, and its existence allowed me to assign a C++11 meaning to the otherwise squishy notion of a "task".

Thus the (still tentative) Item title was born.

What I really like about it is that it's both design advice and coding advice.  At a design level, creating tasks means developing independent pieces of functionality that may be run either synchronously or asynchronously, depending on the computational resources dynamically available on the machine.  At a coding level, it means taking functions and making them suitable for asynchronous execution, either by wrapping them with std::packaged_task or, preferably, by submitting them to std::async (which does the wrapping for you).

"Create tasks, not threads" thus gives me a context in which to discuss exceptions thrown by thread creation requests, the problem of oversubscription, std::thread, std::async, std::packaged_task, and tasks versus threads. Along the way I also get to discuss thread pools and the conditions under which it can make sense to bypass tasks and go straight to std::threads. (Can you see a cross-reference to "Use native handles to transcend the C++11 API"?  I can.)

Scott