- Create tasks, not threads.
- Pass std::launch::async if asynchronicity is essential.
- Make std::threads unjoinable on all paths.
- Be aware of varying thread handle destructor behavior.
- Consider void futures for one-shot event communication.
- Pass parameterless functions to std::thread, std::async, and std::call_once.
- Use std::lock to acquire multiple locks.
- Prefer non-recursive mutexes to recursive ones.
- Declare future and std::thread members last.
- Code for spurious failures in try_lock, condvar wait, and weak CAS operations.
- Distinguish steady from unsteady clocks.
- Use native handles to transcend the C++11 API.
- Employ sequential consistency if at all possible.
- Distinguish volatile from std::atomic.
I'm especially pleased with the first Item on the list ("Create tasks, not threads"), because when I came up with that wording, a number of up-to-that-point disparate thoughts fell into place with a very satisfying thud.
When I began that Item, the only thing I knew I wanted to talk about was that thread construction can throw. In Effective C++, Second Edition, my advice about dealing with the fact that operator new can throw is "Be prepared for out-of-memory conditions," so I started thinking about guidance such as "Be prepared for std::thread exhaustion." But what does it mean to be prepared to run out of threads? With operator new, there's a new handler you can configure. There's nothing like that for thread creation. And if you request n bytes from operator new and you can't get it, you may be able to scale down your request to, say, n/2 bytes, then try again. But if you request a new thread and that fails, what are you supposed to do, request half a thread?
I didn't like where that was going. So I decided to think about avoiding the problem of running out of threads by not requesting them directly. The prospective guideline "Prefer std::async to std::thread" had been an elephant in the room from the beginning, so I started playing with that idea. But one of the other guidelines I was considering was "Pass std::launch::async if asynchronicity is essential" (it's on the draft TOC above), and the spec for std::async says that it throws the same exception as the std::thread constructor if you pass std::launch::async as the launch policy and std::async can't create a new thread. So advising people to use std::async was not sufficient, because using std::async with std::launch::async is no better than using std::thread for purposes of avoiding out-of-thread exceptions.
Though my primary focus had been on figuring out how to avoid exceptions due to too many threads, another issue I wanted to address was how to deal with oversubscription: creating more threads than can efficiently run on the machine. The way to avoid that problem is to use std::async with the default launch policy, and that got me to thinking about what to call a function (or function object--henceforth simply a "function") that could be run either synchronously or asynchronously. A raw function doesn't qualify, because if you run a raw function asynchronously on a std::thread, there is no way to get the result of the function. (And if the function throws an exception, std::terminate gets called.) Fortunately, C++11 offers a way to prepare a function for possible asynchronous execution: wrap it in std::packaged_task. How fortuitous! I had been looking for an excuse to discuss std::packaged_task, and its existence allowed me to assign a C++11 meaning to the otherwise squishy notion of a "task".
Thus the (still tentative) Item title was born.
What I really like about it is that it's both design advice and coding advice. At a design level, creating tasks means developing independent pieces of functionality that may be run either synchronously or asynchronously, depending on the computational resources dynamically available on the machine. At a coding level, it means taking functions and making them suitable for asynchronous execution, either by wrapping them with std::packaged_task or, preferably, by submitting them to std::async (which does the wrapping for you).
"Create tasks, not threads" thus gives me a context in which to discuss exceptions thrown by thread creation requests, the problem of oversubscription, std::thread, std::async, std::packaged_task, and tasks versus threads. Along the way I also get to discuss thread pools and the conditions under which it can make sense to bypass tasks and go straight to std::threads. (Can you see a cross-reference to "Use native handles to transcend the C++11 API"? I can.)