Friday, April 27, 2012

Josuttis 2E has arrived!

For C++98, the standard everyday reference work for the C++ standard library was Nicolai M. Josuttis' The C++ Standard Library.  His updated version for C++11 has been eagerly anticipated, and I was pleased to receive my copy yesterday.

I was confident that the new edition would be a worthy successor to its predecessor, but I have this whole Trust-and-Verify mentality, so I forced myself to withhold judgement until I'd had a chance to check the book's content.  I wasn't disappointed.  Based on skipping around and reading here and there (primarily about functionality new to C++11, including emplacement functions, forward_list, random numbers, tuples, async, and the atomics API, not to mention an overview of new C++11 core language features, e.g., lambda expressions, rvalue references, constexpr, and more), the second edition of The C++ Standard Library looks to be to C++11 what the first edition was to C++98:  the first place to go for a readable and insightful summary of the C++ standard library.  Language lawyers will still turn to the C++ Standard itself, of course, and Josuttis' new book doesn't cover every nuance of library interfaces any more than the first edition did, but note my use of the word readable in my description of his presentation.  The Standard is definitive, but it's not readable.  Josuttis' book is readable, and, for most people most of the time, as good as definitive. I make my living studying the arcania of C++, but when I'm coding, I turn to Josuttis when I can and the Standard only when I must.

Interestingly, the physical size of the new book is no larger than the original edition, even though the standard library has become a lot bigger.  The new book has 300 more pages, however.  (Exactly 300.  The final page number in the first edition is 799, and the final page number in the new version is 1099.  How he managed that, I don't know, but it's a neat trick.)

[The following information about electronic editions has been revised since my original post.]
In terms of electronic editions, the book is available in PDF and ePub, plus there is an edition for Kindle. This information is courtesy of comments readers left on this blog entry, and I am grateful to them for it.

I encourage you to consider buying your own copy of the new edition of The C++ Standard Library.  Having it close at hand was indispensable for C++98 programming, and I think that having it close at hand for C++11 programming will be just as important.


Tuesday, April 24, 2012

Information on the C++11 Memory Model

I've been receiving a number of inquiries about where to turn for information about the C++11 memory model (technically the memory consistency model).  Having prepared and given a talk on the topic at last year's C++ and Beyond, I can tell you that the C++11 memory model is one of the most technically challenging topics I've ever tried to master. The good news is that if your goal is simply to write well-behaved multithreaded programs, you don't need to know any of the details of the memory model.  All you need to remember is to avoid data races in your programs, and the simplest way to do that is to follow the advice that multithreaded developers have been following for years:  access shared mutable data only when you know you have exclusive access to it, typically by acquiring a mutex before the access and releasing the mutex after the access. C++11 offers RAII classes to automate the release part of this protocol (std::lock_guard and std::unique_lock), so this is easier than ever to do. In C++11, you can also use atomic data types (e.g., std::atomic<bool>) to ensure exclusive access with often greater efficiency, but conceptually these are just data types where, on each access, C++ itself handles the moral equivalent of locking and unlocking the appropriate mutex.  (In practice, special machine instructions that avoid the need for mutexes are typically employed.) So, as I said, if your interest in the C++11 memory model is to know how to write threaded programs that will run correctly, you can essentially ignore the model itself.

If, on the other hand, you really want to understand the memory model, you have your work cut out for you.  Under no conditions should you start with the C++11 Standard!  Without the proper protective gear in place, that thing could maim you for life. Instead, I suggest you start with Anthony Williams' book, C++ Concurrency in Action.  Chapter 5 has the most comprehensible description of the C++11 memory model I have seen, and in fact it's the only one I know of that points out that C++11 actually offers three different consistency models:  sequential consistency, acquire-release consistency, and relaxed consistency.  (That alone should give you some idea of what you're in for if you decide to delve into this topic.) I own Williams' book in both electronic and paper form, and I think it's not just the best current treatment of C++11's threading facilities (including, but not limited to, the memory  model), it's likely to remain the best for some time to come.

Sources that complement Williams' coverage include the following.  Note that anything published before September 2011 may differ from the final C++11 memory model, and the longer the time before September 2011, the higher the likelihood of divergence.
Good luck!

Tuesday, April 10, 2012

std::string, SSO, and Move Semantics

I'm becoming increasingly concerned that C++11's support for move semantics is leading some people to conclude that we no longer have to pay as much attention to the cost of copying objects as we used to.  "Many copies will silently become moves," they exclaim, "and moves are cheap!" Such reasoning is invalid in a number of ways.

First, not all copy requests can be replaced by moves.  Only copy requests for rvalues are eligible for the optimization.  Second, not all types support move operations that are more efficient than copying operations.  An example is a std::array<double, 1000>.  Third, even types that support efficient move operations may support them only some of the time.  Case in point: std::string.  It supports moves, but in cases where std::string is implemented using SSO (the small string optimization), small strings are just as expensive to move as to copy!

What it means to be "small" is up to the implementation.  Unless I'm misreading the source files, std::strings with a capacity of up to 15 are "small" in Visual C++ 11 beta, so std::string objects of up to that capacity will not benefit when copies become moves.  (The SSO buffer size seems to be hardwired to be 16 bytes in VC11, so the maximum capacity of a std::wstring that fits in the SSO buffer is smaller:  7 characters.)

gcc 4.7 seems to continue to use reference-counting instead of SSO for its std::string implementation.  This is non-conforming in C++11 (reference-counting may no longer be used for std::string), and it looks like gcc is aware that it must be changed. However, gcc 4.7 offers the non-standard __versa_string with a std::basic_string-compatible interface (other than the name of the template and the namespace in which it's located), and __versa_string offers the SSO.  In this case, the maximum capacity of "small" strings seems to be hardwired to 15 characters (not bytes).

While googling around for a good page to link to for the term "small string optimization" above, I came across John Ahlgren's blog entry of March 30 on how SSO interacts with move semantics.  He refers to the fact that short strings with the SSO move more slowly than longer strings as the "small string deterioration" :-)


Thursday, April 5, 2012

C++11 is Almost Here for Real!

This entry is the first under a slightly expanded charter for this blog.  Until now, I've restricted blog entries to announcements about my professional activities.  Hereafter, entries will simply be related to my professional activities and interests. The difference is subtle, and, in all likelihood, you will rarely notice it.

At the beginning of my Overview of C++11, I show a simple program to compute the most common words in a set of input files.  I write the program once using "old" C++ (i.e., standard C++98/03), then again using features from C++11. 

In 2009, when I first published the C++11 program (at that time, what became C++11 was still known as C++0x), there was no compiler that could come anywhere near compiling it. Testing the code required replacing standard C++11 library components with similar components available in TR1 or from Boost or Just Software Solutions, and language features like auto, range-based for loops, lambda expressions, and template aliases had to be replaced with typically clumsier C++98/03 constructs that were more or less equivalent in meaning.

This week I tested my simple C++11 sample program with Stephan T. Lavavej's excellent  distribution of gcc 4.7 for Windows as well as Microsoft's VC11 beta.  gcc 4.7 has lots of support for C++11, but the concurrency API still seems to be largely missing, at least for Windows, so my sample program doesn't get very far with that compiler.  [Update 6 April 2012: As noted in the comments below, when invoked in the proper manner on the proper platform, gcc 4.7 compiles and runs my program without modification!]

 The situation with the VC11 beta is a lot better.  Only two lines have to be changed.  The template alias
  using WordCountMapType = std::unordered_map<std::string, std::size_t>;
needs to be replaced by its typedef equivalent:
  typedef std::unordered_map<std::string, std::size_t> WordCountMapType;
And the z length specifier in this call to printf,
  std::printf("  %-10s%10zu\n", (*it)->first.c_str(), (*it)->second);
needs to be replaced with its VC++ equivalent, I:
  std::printf("  %-10s%10Iu\n", (*it)->first.c_str(), (*it)->second);
Other than that, the demonstration program I wrote three years ago (which, in fairness to compiler writers, was two and a half years before the C++11 standard was ratified) compiles cleanly with VC11.

If you have access to a compiler that compiles my program without modification, please let me know! The program itself is below.  You can see a more colorful version of it, along with some commentary, and an example invocation and the corresponding output, on slides 13-15 of the free sample of my C++11 training materials.


#include <cstdio>
#include <iostream>
#include <iterator>
#include <string>
#include <fstream>
#include <algorithm>
#include <vector>
#include <unordered_map>  
#include <future>

using WordCountMapType = std::unordered_map<std::string, std::size_t>;

WordCountMapType wordsInFile(const char * const fileName)   // for each word
{                                                           // in file, return
  std::ifstream file(fileName);                             // # of
  WordCountMapType wordCounts;                              // occurrences

  for (std::string word; file >> word; ) {  
  return wordCounts;

template<typename MapIt>                                            // print n most  
void showCommonWords(MapIt begin, MapIt end, const std::size_t n)   // common words
{                                                                   // in [begin, end)
  // typedef std::vector<MapIt> TempContainerType;
  // typedef typename TempContainerType::iterator IterType;
  std::vector<MapIt> wordIters;
  wordIters.reserve(std::distance(begin, end));
  for (auto i = begin; i != end; ++i) wordIters.push_back(i);

  auto sortedRangeEnd = wordIters.begin() + n;

  std::partial_sort(wordIters.begin(), sortedRangeEnd, wordIters.end(),
                    [](MapIt it1, MapIt it2){ return it1->second > it2->second; });

  for (auto it = wordIters.cbegin(); 
       it != sortedRangeEnd; 
       ++it) {
    std::printf("  %-10s%10zu\n", (*it)->first.c_str(), (*it)->second);

int main(int argc, const char** argv)   // take list of file names on command line,
{                                       // print 20 most common words within;
                                        // process files concurrently
  std::vector<std::future<WordCountMapType>> futures;

  for (int argNum = 1; argNum < argc; ++argNum) {
    futures.push_back(std::async([=]{ return wordsInFile(argv[argNum]); }));

  WordCountMapType wordCounts;
  for (auto& f : futures) {
    const auto wordCountInfoForFile = f.get();  // move map returned by wordsInFile

    for (const auto& wordInfo : wordCountInfoForFile) {
      wordCounts[wordInfo.first] += wordInfo.second;

  std::cout << wordCounts.size() << " words found.  Most common:\n" ;
  const std::size_t maxWordsToShow = 20;
  showCommonWords(wordCounts.begin(), wordCounts.end(), 
                  std::min(wordCounts.size(), maxWordsToShow));