Effective Effective Books

For the cover of the first edition of
Effective C++, I envisioned Alexander
the Great cutting the Gordian Knot.

I'm getting ready to start work on Effective C++11 (about which I'll have more to say in a later blog post), and this week I've been reviewing a manuscript for a new book in my Effective Software Development Series, so recently I've been thinking a lot about what makes an effective Effective book.

Effective books consist of a collection of technical essays ("Items"), where each essay's title comprises advice you should follow, and each essay's body consists of a rationale for the advice. In the third edition of Effective C++, for example, Item 37 is "Never redefine a function's inherited default parameter value."

This format has always seemed pretty straightforward to me, but having reviewed dozens of proposals and manuscripts for my series, it's clear that it's not straightforward to everybody. A few years ago, I wrote up the following guidelines for authors of prospective Effective books. Partly to remind myself of my own advice as I embark on a new book, and partly to have a place to point people who express interest in writing a book for my series, I thought I'd publish my guidance here.

Item 1: Keep Items short.

One of the things readers like best about Effective C++ is the brevity of the Items.  They're generally only 4-5 pages long, and that means that one can be read while waiting for a meeting to begin, during a quick break, even while using the bathroom.  I discovered this when I came out with my second book (More Effective C++), in which some items run for 10-20 pages.  Big mistake!  People want bite-sized chunks of useful information from an Effective book.  Give it to them.

Frankly, the constraint that Items be short is good for you as an author, because it forces you to think hard about what really needs to be included and what can be omitted (see below).  Also, it allows you to produce a book of typically under 300 pages, and a book of that length is something you've got half a chance of keeping in your head all at once.

Regarding overall book length, there are two ways to approach the matter.  It's a lot like packing for a trip.  You can either:

• Choose all the stuff that you think you need, then find a big enough suitcase to hold it all, or
• Choose the suitcase first, prioritize everything, then put in as many high-priority items as will fit.  

Effective books generally follow the latter strategy.

Think about your goals for your book.  Is it to tell your readers everything they need to know, or is it to tell them the most important things?  Remember that the longer your book, the less likely your readers will read all of it.  So what would you rather have?  A 250 page book that almost all readers read almost all of or a 500 page book that most readers read about half of?

It's your book, of course, and it's most important that you be happy with it, but you might bear in mind that I've published 140 Items on C++ programming spread across about 800 pages, but I did it in 3 different books.  I think my readers like it better that way, because there's less for them to digest at one time.  I know I like it better the way I did it, because I find it a lot easier to keep no more than 300 pages of material straight than to try to wrestle with 800 pages at once.  Also, I like knowing that my readers are likely to read each book all the way through.  I hate it when I work hard at writing something, only to find that some people stopped reading before they got to the end.  I have a story I want to tell them, and it's important to me that they get the entire story.

If you have a lot to say, there's no law that says you have to say it in a single book. Splitting it across multiple books may end up serving everybody better.

Bear in mind that Effective books are designed to be "the standard second book" on their topic.  You should assume that readers know the basics and have another comprehensive reference available to them.

Item 2: Put Item titles in the imperative.

People read Effective books for advice, to learn "the things the experts almost always do--or almost always avoid doing."  They want to be told what to do or what not to do, so each item should give them an order: Do this or don't do that (ideally the former--see below).  Statements are not as helpful as imperatives, so don't use statements as Item titles.

As an example, I could have worded this Item as "Imperative Item titles are good," but that's not advice, it's just a statement.  "Put Item titles in the imperative" is advice.  Your readers want advice.  Provide it.

Alas, they sometimes want advice when what they really need is information.  Fortunately, there's a way to give them both: use imperatives that begin with "Understand" or "Familiarize yourself with" or "Be aware of" or things of that ilk.  For example, if you think that people generally fail to grasp the subtleties of some topic XYZ and you feel that they really need to understand it in order to be effective software developers, consider a title like, "Familiarize yourself with the subtlties of XYZ."  It's still an imperative.

After reading your book, people will want a convenient summary of the advice it contains.  If each Item title is an imperative, the book's TOC (Table of Contents) will be a summary of its advice, and that should be one of your goals: to have the TOC summarize the advice in the book.

Item 3: Make Item advice as specific as possible.

The best advice is so specific, it can be checked by a machine.  In fact, many of the guidelines I put in Effective C++ have been incorporated into static analysis tools.  Specific advice is also good for code reviews, because it eliminates the interpretation factor; readers don't argue over what a particular Item means.  They might argue over whether a particular Item is justified (see below), but if the advice is specific, they won't argue over what it means.

If you are having trouble making a guideline specific, it could be because the topic you are writing about in that Item is too general.  If that's the case, perhaps you need to break it down into two or more separate (and more specific) guidelines.  For example, if you're trying to offer advice about the proper use of inheritance, but you find that it's hard to be specific, because interface and implementation inheritance are so different, consider writing two items, one about interface inheritance, the other about implementation inheritance.

Item 4: Word Item titles carefully.

Your TOC summarizes your book's advice.  Furthermore, your TOC consists primarily of your Item titles.  That means that the most important sentences in your book are the ones making up your Item titles.  All words in a book are worth agonizing over, but some are more agonizable than others, and the ones in your Item titles are the most agonizable of all.  They make up the advice you want your readers to remember, even if they forget everything else in the book.  As a result, you want your Item titles to (1) be easy to remember and (2) accurately summarize your advice on the topic addressed by the Item.

Resist the urge to choose titles that are catchy, but not informative.  One manuscript I read had an item on using profilers to guide optimization efforts, and the Item title was "Optimize optimally."  This is catchy, but it doesn't summarize the advice in the Item.  Item titles are different from article titles.  Authors often choose enigmatic article titles to intrigue people into reading the article.  That's not the goal in an Effective Item.  Readers already have the book. You can expect them to read it.  The goal of an Item title is to summarize in a few words the advice that will take several pages to justify.  Remember: the Item title is "what."  The Item body is "why."

When I write my books, I typically revise the wording of Items several times.  I'll start with what I think I want to say, but after I've written an Item's body, I'll realize that my justification doesn't really back the title, so I'll revise the title.  Or after I've written other Items, I'll discover that some of what what I wanted to say in Item n is now in Item m, so I'll go back and revise Item n's title to reflect the fact that it no longer addresses the concerns of Item m.

Any time my Item title is too long to fit on a single line, I'll try to come up with a shorter way to say it.  Short Item titles are easier to remember than longer ones, and my goal is for my readers to remember all the titles of all the Items.  The easier I make it for them to do that, the better my odds of achieving my goal.

You'll note that several of the other Items in this document relate to Item wordings.  That's no accident.  It's a reflection of the importance of Item titles to Effective books.  You've got just a few words to summarize the lessons of several pages of often complex material.  Choose them carefully.  They're the most important words in your book.

Item 5: Prefer guidelines that say what to do over what not to do.

Guidelines that tell people what they should do are more useful than guidelines that tell them what they should not do.  If a programmer wants to accomplish task X, knowing that they should not do A, B, or C may narrow the design space, but it doesn't really tell them how to accomplish X.  Any time you are tempted to start an item with "Avoid" or "Never", imagine the following conversation with your readers:
       You:         Avoid doing A.
       Readers:   But if I don't do A, how can I accomplish X?
       You:         Do B instead.

If you can come up with such a conversation, you probably want to word your Item as something like "Prefer B to A for X."  (See below for more information on words like "avoid" and "prefer" in Item titles.)

Item 6: Tell the truth and nothing but the truth, but not necessarily the whole truth.

Effective books are short by design, typically only about 5 pages per Item.  With such limited space, it's important to include only information readers really need to know.  Your goal should be to help your readers, not to demonstrate your mastery of history or trivia, etc.  After all, one of the things they are paying you for is deciding what they don't need to know.

At the same time, you lose your credibility if you lie to your readers, so you must never lie.  To avoid lying while also shielding them from information they don't need to know, you will need to fall back on weasel words: typically, generally, usually, etc.  You don't want to use these more than you have to, because readers will notice them, but if your choices are a two-page digression on how some bizarre machine or compiler that almost nobody uses does something in a strange way or slipping "virtually" in front of "all implementations," then "virtually" and its kin are your friends.

Item 7: Back Items with solid technical arguments.

The most common weakness I see in prospective Effective books is that the Item rationales fail to make convincing cases for the Items' advice.  This is critical!  Remember, you're telling somebody what to do.  You're probably asking them to change the way they do things.  If they're going to change their habits, and especially if they're going to let you push them around, they'll demand a good reason, and it's your job to give it to them.  You should assume that your readership is smart, experienced, and critical.  Effective books aren't introductory, so your readers already know some of what you're talking about.  Some of it they know as well as you do.  If you tell people to do X, but fail to acknowledge that it's impractical some of the time, they'll notice, and you'll lose credibility.  If you tell them that they have to do X because of Y, but Y isn't true, they'll notice, and again, you'll lose credibility.  You can't afford to lose credibility, because if you're not credible, why should
your readers take your advice?

In addition, there are almost certainly going to be exceptions to any advice you can offer, and one of the most important things you must do as an author is explain to your readers when your advice applies and when it does not.  The Item rationale is the place where you do that, at least implicitly.  For example, I spend a lot of time in my C++ books talking about how to prevent resource leaks, and much of my advice boils down to, "Do this, because it helps prevent resource leaks."  At one point, I was talking to engineers who work on the software that runs inside missiles.  Their programs run until the missile hits its target.  They don't care about leaking resources at program shutdown, because their programs never shut down.  They stop running only when the bomb goes boom.  As a result, they feel free to ignore some of my advice, and that's perfectly reasonable.  But the only way they know that that is perfectly reasonable is because my books make clear the technical motivation for the advice I give.  When that motivation fails to apply, so does the advice.

Item 8: End each Item with a summary of its advice.

The old saw is "Tell 'em what you're going to tell 'em; tell 'em; tell 'em what you told 'em."  Effective Items are similar, but not quite the same: Tell them what they need to remember; justify it; remind them of what they need to remember.

If you've worded your Items correctly (see above) what they need to remember is the Item title, a single sentence of only a few words.  The justification typically runs 4-5 pages, and that's where you deal with implementation strategies, portability issues, exceptions to the rules, explanations of why things are the way they are, etc.  By the time you're done with that, there is every likelihood that your readers have forgotten the main point you initially set out to make.  You therefore need to remind them.  It doesn't need to be long-winded; a sentence, short paragraph, or set of bullet points is all you need.  For example, for an Item with this title,

  Write operator delete if you write operator new.

I end this way (9 pages later, sigh):

Now, it's interesting to see how custom memory management routines can improve program performance, and it's worthwhile to see how such routines can be encapsulated inside a class like Pool, but let us not lose sight of the main point. That point is that operator new and operator delete need to work together, so if you write operator new, be sure to write operator delete, as well.

For the third edition of Effective C++, I added a short list of bullet points to the end of each Item called "Things to Remember." That allowed me to eliminate the summary paragraphs from many Items while still allowing me to remind readers of each Item's most important information.

Item 9: Know how to modulate the stridency of Item titles.

Virtually every rule has exceptions, but some rules have more exceptions than others.  The wording of your Item titles should reflect the likelihood that your readers will encounter exceptions.  In real life, for example, we should (almost) always buckle up for safety, (almost) never drink and drive, and (typically) consider taking public transit when it is available.  Those three words--always, never, and consider--are three of the five words I've found useful when wording Item titles.  Here's the full scale of advice urgency, from Always to Never:

•   Always
•   Prefer
•   Consider
•   Avoid
•   Never

The default is "Always," so it's generally implicit.  I could have worded one of the guidelines above  this way, for example,

Always put Item titles in the imperative.

but "Always" here adds little, so there's no point in including it. 

Because every rule has exceptions, there's an implicit "Almost" in front of any rule that begins with "Always" or "Never." However, including "Almost" in Item titles draws attention to the existence of the exceptions, and that's not what we want readers to focus on.  For example, consider how wishy-washy this looks:

Almost always put Item titles in the imperative.

Please!  Take a stand!  That's one of the things that readers like about Effective books: the authors give specific advice instead of bland platitudes.  Each Item has a rationale,  so any situation not covered by the rationale can naturally be interpreted as being an exception to the rule.  Hence the implicit nature of "Almost."

Item 10: Cross reference liberally.

Effective books are collections of essays.  Each is specific and constrained (see above), but collectively they cover a lot of territory.  The territories they cover necessarily abut and overlap, and that means that almost no Item stands alone.  When there are relationships among Items, readers need to know that.  That's where Item cross-referencs come in.  If Item A gives a detailed discussion of a topic that's relevant to Item B, B should probably have a cross-reference to A.  (A might also have a cross-reference to B.)  If Item A strengthens the rationale for Item B, B should probably have a reference to A.  If Items A and B cover disjunctive situations, both should probably refer to one another (e.g., "Item A explains why you need to do X to achieve Y, but sometimes your goal isn't Y, it's Z.  When that's the case, the advice in Item A fails to apply...").

In the end, most everything is related to everything else in some way or another.  You don't need to point out all relationships to your readers (see above about omitting information they don't need to know), but you should definitely use cross references to point out the ones that are important in making effective use of whatever technology you are writing about.

Item 11: Minimize use of footnotes.

Footnotes make a book look more formal and academic.  Effective books tend to shoot for a more casual, informal approach.  As a result, it's best to avoid footnotes whenever possible.  In general, you'll find that information you want to footnote can either be eliminated or incorporated into the body of your discussion.

Item 12: Be consistent when referring to yourself and your readers.

This is a grammatical "person" issue: first person, second person, or third person?  When you refer to the author of the book, do you say "I" (first person) or "the author" (third person)?  When you refer to the reader of the book, do you say "you" (second person) or "the reader" (third person)?  The choices are nowhere near as important as that you abide by them consistently.  I like to envision Effective books as one-on-one informal conversations between the author(s) and each reader, so I prefer using first person for the author and second person for the reader, e.g., "Now, when I say that XYZ is a good idea, I suspect you'll wonder if I'm off my rocker."  Third person works, too, though I think it's stuffy: "When considering the author's recommendation to do XYZ, the reader may be forgiven for wondering if the author is off his rocker."

Bottom line: I see too many manuscripts where authors can't decide how to refer to themselves or their readers.  Make the decision early on, write it down so you remember it, and adhere to it consistently.

Item 13: Seek out ruthless pre-publication reviewers.

No matter how careful you are, odds are that the book you write will contain errors or other problems.  That being the case, it's in everybody's interest to find out about them as soon as possible.  Ideally, you'll find out about them before the book is actually published, thus giving you a chance to fix them before your masterpiece is loosed on the world.

There are many ways to get good pre-pub feedback, but all have one thing in common: they are based on finding reviewers who are willing and able to tell you how your book can be improved and who don't pull punches.  What you want to hear is that your manuscript is wonderful and is ready for publication, but what you need to hear is that some words are misspelled, some diagrams are confusing, some topics are technically incorrect, the way you've organized things makes no sense, and that you've failed to consider some aspects of the material that are important.

Providing such feedback is hard work, and that means that you're unlikely to find people willing to give it to you unless you go looking for them.  Seek them out.  Tell them you want to know everything that is wrong with your manuscript.  Make it clear that a better book is more important to you than an unbruised ego.  Ask your friends to be reviewers only if (1) you believe they are willing to be brutal in their comments and (2) you honestly believe that your friendship can withstand the pressure.  But don't rely only on your friends.  Seek out individuals who are representative of your target audience, and ask them to tell you what they think.  (Your friends are unlikely to be good at this.  For one thing, there's a very good chance that your friends know more about what you're writing about than your average target readers does.  After all, your friends hang out with you.  Your readers don't.)

When I circulated an early draft of my second book, I got back comments from one reviewer who, having savaged the first 30-40 pages, concluded with "It doesn't get any better after that."  It took me a couple of days to recover from that, but once I did, I was able to dispassionately evaluate the comments and conclude that they were accurate.  Then I rewrote the book, making extensive revisions.  The resulting book was much better than it would have been had I received only comments telling me that this or that detail needed tweaking.

It's nice to hear nice things about what you've written.  You're much more likely to hear them after your book comes out if you've heard awful things when the book is in progress.  Seek out people who will tell you those awful things in advance so you won't have to hear them later.

New ESDS Book: Effective JavaScript

I'm very pleased to announce that the ninth book in my Effective Software Development Series has recently been published, David Herman's Effective JavaScript. In my comments to David on a pre-publication manuscript, I wrote:

These chapters are a very pleasant read.  The text flows well, and the explanations are such that I generally had no trouble understanding them, even though I don't know JavaScript.   I think this will be a very nice book.

Others who do know JavaScript seem to agree that it's turned out that way.  Reginald Braithwaite recently posted a quite positive review, and the review at Utah JS starts out this way: "Effective JavaScript by David Herman is the best JavaScript book I’ve ever read." Currently, there are six reviews at Amazon, and all award perfect fives.

If you're a JavaScript programmer, I encourage you to check out Effective JavaScript.

Scott

Recent Developments regarding C&B

2013 is only a week old, and already there's been a lot of action on the C++ and Beyond front. That's especially the case if you're willing to fully embrace zero-based indexing (thus viewing December 31, 2012, as January 0, 2013), because that means that the following announcements have come out this year:

• There will be a 2013 edition of C++ and Beyond.  Tentative dates are December 8-11.
• Videos of two additional technical presentations and one panel from C&B 2012 have now been published at Channel 9:
• Herb Sutter's "You don't know [keyword] and [keyword]".
• Herb's "Concurrency and Parallelism".
• Herb's and Andrei Alexandrescu's and my panel on "Convincing Your Colleagues".

That makes six sessions from C&B 2012 we and Channel 9 have so far made available. Two additional sessions are slated to be published at Channel 9 soon.

As always, the best way to be kept up to date on developments regarding C++ and Beyond is to do one or more of the following:

• Subscribe to the blog. This is our primary communications mechanism.  The options below currently just reflect information posted to the blog.
• Subscribe to the mailing list.
• Follow C&B on twitter.
• Like C&B on Facebook.

I look forward to another great C++ and Beyond near the end of this year, and I hope you'll be part of it. We'll announce more details when they are available.

Scott

New Revision for C++11 Training Materials

In a post in October, I said that the next revision of my C++11 training materials would come out later this year, and that time has come.  This revision is the sixth since the materials were originally published in 2010.  As always, past buyers of the materials are entitled to a free update.  If you've already purchased these materials, you should have received notification from Artima (the publisher) that a new version is available for download.

My normal practice is to publish a changelog along with each new version of the materials, but in this case, I added and removed pages in a number of places, and trying to keep track of and figure out how to describe what got changed where became burdensome, so I stopped trying.  As a result, there is no detailed changelog, but I can offer this high level list of changes:

• Coverage of enum classes has been added.
• Coverage of the interaction between noexcept and move operations in the STL has been added.
• Coverage of universal references has been added, and the example of using them for constructors has been removed.  (This blog post explains why I now think it's generally inappropriate to use universal references for constructors.)
• Coverage of numeric-string conversion functions has been added.
• Coverage of alignment control (alignof, alignas, std::aligned_storage, and std::aligned_union) has been added.
• Coverage of implementation of std::get (for std::tuple) has been added.
• Coverage of braced initializers has been extended.
• Coverage of lambdas has been extended.
• Bugs have been fixed, explanations have been improved, and information has been updated in a number of places.

The overall length of the materials has increased by about 20 pages.

Books on C++11 are beginning to come out, but they're, well, long. For example, the latest C++ Primer is 976 pages, and the upcoming C++ Programming Language is listed at 1040 pages.  If you're looking for more of a technical overview of C++11's primary features than a comprehensive treatment of essentially everything, I think my training materials (383 slides in 22-point type, plus accompanying notes) is a choice you should consider.
 
As before, you can download a free sample to see exactly what you'd be getting if you bought  the materials.  Topics covered in the free sample include:

• Copying versus moving (high-level overview).
• Simple example program:  C++98 versus C++11.
• ">>" to close nested templates.
• auto for type declarations.
• Range-based for loops.
• nullptr.
• enum classes.
• Unicode support.
• Raw string literals.

I originally wrote the following a year ago, but it's still apt:

If you're interested in a book-like publication covering the most important parts of C++11 (both language and library), I encourage you to consider purchasing my training materials.  If you like my other publications, I think you'll like these, too.  To see exactly what you'll be getting, check out the free sample.

Because this publication is in an unconventional format (annotated training materials), is available from a lesser-known publisher (Artima), and is electronic-only (DRM-free PDF), getting the word out about it has been challenging.  I'd appreciate it if you'd let people know about it, whether through blogs, tweets, social networks, email, or that most retro of communications mechanisms, face-to-face conversation.

Thanks,

Scott

Examples of good code-based digital publications?

In my last post, I wrote:

I [want] to ensure that [my next] book looks good on all target platforms, by which I mean ink on paper as well as various digital representations (e.g., PDF, HTML, ePub, etc.)  I've noticed that many books containing code examples display, er, suboptimally on one or more digital devices, and I want to avoid that.

Over the years, I've heard many complaints about the appearance of code-heavy publications (book or magazine article or blog entry, etc.) on digital devices, but I can't recall any cases of people praising the appearance of a programming piece on the device they use.  I'd like to believe that this is just because I'm wallowing in ignorance.  If you have read a code-heavy publication on a digital device where you felt the job was done well, please tell me!  I'm primarily interested in mobile devices like phones, e-readers (e.g., Kindle or Nook) or tablets.  PDF on a large laptop is not hard to get right.

If you have an example of code-heavy publishing done well on a mobile device, please email or post as a comment the following information:

• The device being used
• The title of the publication
• The publisher
• The author
• What about the experience you find especially well done

I really want to do a good job porting my book to various platforms, and the more I can find out about successes that precede me, the better I can do.

Thanks,

Scott

Publishing Code on Digital Platforms

I plan to write a new book next year (Effective C++11, about which I'll have more to say later, probably in January), and I'm laying the groundwork for it.  Part of that groundwork is doing what I can to ensure that the book looks good on all target platforms, by which I mean ink on paper as well as various digital representations (e.g., PDF, HTML, ePub, etc.)  I've noticed that many books containing code examples display, er, suboptimally on one or more digital devices, and I want to avoid that.  To that end, I've started a list of characteristics I think code displays should have, i.e., that readers should be able to take for granted in code displays.

A "code display" is code sitting in a paragraph on its own, i.e., like this:

// This code display consists of a class definition and a free function
class Widget {
public:
  explicit Widget();
  Widget(const Widget& rhs);
  Widget(Widget&& rhs);
  ...
};

std::ostream& operator<<(std::ostream& s, const Widget& w);

Here's my list so far:

• Code should be syntax-highlighted.
• Code should be copyable and pastable as text. This rules out using images for code displays.
• Code should be copyable and pastable as code. For example, if the code display has line numbers, there should be a way to copy and/or paste such that the line numbers are omitted.
• Code should be searchable. This also rules out using images for code displays.
• Viewing Code displays should not require horizontal scrolling.
• Code displays should have no bad line breaks.  This means no automatic wraparound due to the code not fitting in the available horizontal space.

There's a caveat to this "code reader's bill of rights." If you choose to view a code display in a "pathological" way, you lose the last two guarantees.  In general, this means that I expect you to have some reasonable horizontal width available on whatever device you're using.  If you view a code display on an iPhone in portrait mode, you're on your own.  If you view code on 10" tablet and set the font size to 200, you're on your own. If you view code on a giant PC screen and set the viewing window to 10 pixels wide, all bets are off.  As an author, I'm willing to do what I must to offer you these guarantees, and I'm willing to browbeat my publisher into offering them, too, but you have to give us a reasonable field to play on.

Are there other things you, as a reader, would like to be able to take for granted when viewing code displays in a technical publication?  Are there problems with code displays on digital devices that you've run into and that are not addressed above?  If so, let me know. Bear in mind that I'm trying to identify basic characteristics you should be able to take for granted, but currently cannot.  I'm happy to hear about nice-to-haves, too (e.g. it'd be nice to be able to click on or gesture over some code and have it automatically sent to an online site such as ideone or Live Work Space to be compiled and executed), but what I'm really interested in are things you should be able to take for granted.

Thanks,

Scott

More Effective C++ at 30 (Printings)

Today I received something rather remarkable:  a copy of my More Effective C++ from its thirtieth print run. So many printings is uncommon for any technical book, but it’s particularly surprising for this one, because our understanding of what it means to apply C++ effectively has undergone considerable change since the book’s initial publication in 1996. For me, that makes a thirtieth printing kind of a big deal. It’s also a somewhat ironic deal, because, for many years, More Effective C++ was a book I wasn’t especially fond of. Only its longevity has allowed me to develop an admiration and respect for it.

To some degree, books are to authors what children are to parents, and just as parents should love all their children equally, I’ve always felt guilty that I didn't have the affection for More Effective C++ as for my other C++ books, Effective C++ and Effective STL. The reason is simple: writing More Effective C++ was not a terribly pleasant experience.

That’s not what I anticipated when I began. Effective C++ had been a breeze to write, because I’d built it around guidelines and examples I’d refined through my training work with professional developers. For that book, I was so sure about what I wanted to say and how I wanted to say it, the manuscript nearly wrote itself. I probably spent more time worrying about formatting than content. Furthermore, I was excited about the prospect of becoming an author, so even the parts of the project that felt like work were fun and exciting. It was hard work, sure, but it was fun, exciting hard work.

Effective C++ was well received, and being the author of a hit was also fun. I became a columnist for C++ Report, which meant I acquired a soapbox from which to pontificate, and what’s not fun about pontificating? It was fun, fun, fun, all the time, and that was my mindset as I set about work on More Effective C++. It was going to be the fun sequel!

As best I can remember, I produced a draft manuscript without difficulty, and this was sent out for review. That’s when the fun ended. The book, it became apparent, had two problems. First, it was not technically sound. Second, it was irritating to read. One of the reviewers stopped commenting after a few dozen pages, and, having savaged what he’d seen, concluded with “It doesn’t get any better after that.” Ouch.

So much for fun.

There’s no point in asking for comments on a manuscript if you’re going to ignore them. The technical shortcomings of the material were easy to verify. What’s wrong is simply wrong. The comments on my writing style were more difficult to accept. However, these were remarks from people I respected, and if they were willing to take the time to warn me that my prose was more likely to enrage readers than engage them, I owed it to them (and to my future readers) to assume that what they saw was really there. I set the manuscript aside for a few days so I could look at it anew...and so I could prepare for the ordeal.

With my reviewers’ comments in mind, a fresh look clarified things. The manuscript was somewhere between bad and awful, and the reason was obvious. While working on it, I’d been thinking of fun, fun, and more fun. I was having fun writing, and I wanted my readers to have fun reading. But readers don’t turn to a book on C++ for entertainment. They turn to it for information and insight. I’d forgotten that, and it showed in my writing.

I started over.

“This is not a project about fun,” I told myself, “this is a project about useful information for professional C++ developers who have jobs to accomplish. Working on the book means working. There can be some fun along the way, but the essence of the project is careful work yielding practical insights into C++ and its effective application.” With that in mind, I rewrote the book, and the result (after a round of substantially less brutal reviews and the always-necessary copy-editing pass) was what was ultimately published.

Even setting aside that I wrote More Effective C++ twice, working on it was bound to be harder than Effective C++. It wasn’t my first book, so the “Oooh! I’m going to be an author!” factor was missing. I’d put all my best and most established material in Effective C++, so coming up with quality content was a bigger challenge. Furthermore, I’d changed the format of the book. Whereas Effective C++ consists of guidelines that can be explained and justified in a few pages, in More Effective C++ I wanted to do that plus explore some meatier topics—topics that could not be covered in such a format. More Effective C++ is essentially two books stapled together: one containing Effective C++-like guidelines, and another consisting of technical essays on topics like smart pointers, reference counting, and proxy classes. The result is about 50% longer than the first edition of Effective C++—another reason why More Effective C++ called for more effort.

The work was demanding, and I really did treat it as work, but I decided to allow myself one luxury—one Item motivated not by any practical consideration or compelling use case, but solely by my interest in it. One Item for fun. That Item is number 31: “Making functions virtual with respect to more than one object.” The general term for the topic is multi-methods. It’s sometimes also referred to as double dispatch or multiple dispatch, though these latter terms conflate what with how. (What we want is multi-methods. Double or multiple dispatch is how we can implement them. I’m chagrined to admit that among those who have failed to distinguish what from how is me: in More Effective C++, I describe multi-methods, double dispatch, and multiple dispatch as synonyms.)

Curiously, my “fun” Item has emerged as one of the most frequently referenced in the book. Over the years, authors have published increasingly sophisticated ways to attack the problem I explored, often using the same example I employed (handling collisions between objects in a video game). In the “Interesting Comments” section of my online errata list for More Effective C++, I refer to six books, articles, or online discussion threads that address the implementation of multi-methods. In 1998, I was even an outside reader on a doctoral dissertation focusing on the implementation of multi-methods in strongly-typed programming languages.

Also curious is the turn taken by Items 28 and 29, which together examine how non-intrusive reference-counting smart pointers can be implemented. Experience with the code I published motivated me later to advise developers to steer clear of custom code in favor of proven implementations, notably Boost’s shared_ptr (which became the basis for TR1’s std::tr1::shared_ptr, and, ultimately, for C++11’s std::shared_ptr). In Effective STL, I wrote:

The STL itself contains no reference-counting smart pointer, and writing a good one—one that works correctly all the time—is tricky enough that you don't want to do it unless you have to. I published the code for a reference-counting smart pointer in More Effective C++ in 1996, and despite basing it on established smart pointer implementations and submitting it to extensive pre-publication reviewing by experienced developers, a small parade of valid bug reports has trickled in for years. The number of subtle ways in which reference-counting smart pointers can fail is remarkable.

Remembering that I wrote this is easy, because it was quoted in the proposal to the C++ standardization committee that ultimately led to the introduction of shared_ptr (and weak_ptr) into TR1. That’s the kind of thing that warms a fellow’s heart: his inability to publish correct code being used as an argument to expand the standard library.

Compensating somewhat is the recognition that the advice I proffer in Item 33 (“Make non-leaf classes abstract”) has now largely become accepted wisdom in the C++ community. At the time, it ran counter to widespread thinking about object-oriented programming, even though I’d spoken with enough experienced C++ library designers to know that well-engineered libraries generally hewed to the line I advocated. I braced for a backlash from readers aggrieved that I was telling them to abandon the idea that adding a class to a hierarchy was a simple matter of finding a suitable base class and inheriting from it. In fact, my advice was consistent with that idea. I simply imposed a constraint on what it meant to be suitable, and that constraint flew in the face of what was at the time considered a cornerstone of object-oriented programming. That Item may be the one I’m most pleased with, because it provides a technical underpinning for a counterintuitive, yet pervasive, constraint on good hierarchy design.

More Effective C++ was one of the first books to include advice on programming with exceptions, and the essence of that guidance is all but enshrined in the Exception-Aware Programming Hall of Fame: use RAII to manage object lifetimes, prevent exceptions from leaving destructors, catch exceptions by reference, be aware of the costs that exception-based error handling may incur, etc. Interestingly, the term “RAII” doesn’t occur in the book, but the acknowledgments explain why:

The notion of using destructors to prevent resource leaks (used in Item 9) comes from section 15.3 of Margaret A. Ellis’ and Bjarne Stroustrup’s The Annotated C++ Reference Manual. There the technique is called resource acquisition is initialization. Tom Cargill suggested I shift the focus of the approach from resource acquisition to resource release.

Thanks to Tom’s advice, Item 9 is entitled “Use destructors to prevent resource leaks”—the essence of RAII. By the time I wrote the third edition of Effective C++ nearly a decade later, I’d adopted “Use objects to manage resources” as my wording for this kind of guideline, but I’m gratified to see that the exception-related information in More Effective C++ was on target three years before the appearance of what I consider the breakthrough book on programming with exceptions, Herb Sutter’s Exceptional C++.

In fact, viewing the Item title summary that is More Effective C++’s table of contents in this, the dawn of the C++11 era, I’m pleased to see that, fundamentally, all the advice there remains appropriate. Naturally, I’d tweak some things. For example, in light of the utility of some template metaprogramming techniques that I (and pretty much everybody else) was unaware of in 1995, I’d soften Item 7’s “Never overload &&, ||, or ,” to something more like “Be wary of overloading &&, ||, and ,”. Because exception specifications are deprecated in C++11, I’d replace Item 14’s “Use exception specifications judiciously” with something closer to “Prefer noexcept to exception specifications.” Such tweaks notwithstanding, the high-level advice in More Effective C++ suffers from a lot less decay than one might expect from a book in its seventeenth year. I can’t help but be a little proud of that.

The thirtieth printing of More Effective C++ didn’t produce the same book as the first one. As with all my books, I solicit bug reports and other improvement suggestions from my readers, and I keep the book’s errata list online, so everyone can see the problems that have been reported and the ones I have fixed. For each new printing, I do my best to revise the book to address outstanding issues, but sometimes my schedule is such that there’s not time to do it. (The window between being notified of a forthcoming printing and the deadline for revisions is typically rather short.) The 30 printings of More Effective C++ since 1996 have yielded 20 different versions. 

 More than 20 versions have actually been printed, because I’m counting only domestic printings. The book has been printed in alternative forms for English-speaking markets outside the United States (e.g., with a different cover design and less expensive paper for markets where North American pricing is inappropriate). It’s also been translated into a number of foreign languages, including German, Japanese, Korean, Polish, Russian, and both traditional and simplified Chinese. All in all, More Effective C++ has been printed more than 150,000 times in at least ten languages. It’s available in a number of electronic formats, as well.

My collection of More Effective C++es.  Domestic printings are on the left, international versions on the right.  Detail-oriented readers will notice 31 copies of the domestic version.  That's because there were two versions of the third printing.

My little less-loved book has surpassed any reasonable expectations an author could have. Still quietly chugging away in its seventeeth year, still providing useful information to C++ software developers—what more could I ask? As I said, I’ve grown to admire and respect this book—and even to have a parent’s affection for it.

On the Superfluousness of std::move

During my presentation of "Universal References in C++11" at C++ and Beyond 2012, I gave the advice to apply std::move to rvalue reference parameters and std::forward to universal reference parameters.  In this post, I'll follow the convention I introduced in that talk of using RRef for "rvalue reference" and URef for "universal reference."

Shortly after I gave the advice mentioned above, an attendee asked what would happen if std::forward were applied to an RRef instead of std::move.  The question took me by surprise.  I was so accustomed to the RRef-implies-std::move and URef-implies-std::forward convention, I had not thought through the implications of other possibilities.  The answer I offered was that I wasn't sure what would happen, but I didn't really care, because even if using std::forward with an RRef would work, it would be unidiomatic and hence potentially confusing to readers of the code.

The question has since been repeated on stackoverflow, and I've also received it from attendees of other recent presentations I've given.  It's apparently one of those obvious questions I simply hadn't considered.  It's time I did.

std::move unconditionally casts its argument to an rvalue.  Its implementation is far from transparent, but the pseudocode is simple:

  
  // pseudocode for for std::move
  template<typename T>
  T&& std::move(T&& obj)
  { 
    return (T&&)obj;        // return obj as an rvalue
  }

std::forward is different.  It casts its argument, which is assumed to be a reference to a deduced type, to an rvalue only if the object to which the reference is bound is an rvalue. (Yes, that's a mouthful, but that's what std::forward does.)  Whether the object to which the reference is bound is an rvalue is determined by the deduced type.  If the deduced type is a reference, the referred-to object is an lvalue.  If the deduced type is a non-reference, the referred-to object is an rvalue.  (This explanation assumes a lot of background on how type deduction works for universal reference parameters, but that's covered in the talk as well as in its printed manifestations in Overload and at ISOcpp.org.)

As with std::move, std::forward's implementation is rather opaque, but the pseudocode isn't too bad:

  // pseudocode for for std::forward
  template<typename T>
  T&& std::forward(T&& obj)
  { 
    if (T is a reference)
      return (T&)obj;          // return obj as an lvalue 
    else
      return (T&&)obj;        // return obj as an rvalue 
  }

Even the pseudocode makes sense only when you understand that (1) if T is a reference, it will be an lvalue reference and (2) thanks to reference collapsing, std:forward's return type will turn into T& when T is an lvalue reference.  Again, this is covered in the talk and elsewhere.

Now we can answer the question of what would happen if you used std::forward on an RRef.  Consider a class Widget that offers a move constructor and that contains a std::string data member:

  class Widget {
  public:
    Widget(Widget&& rhs);       // move constructor
  
  private:
    std::string s;
  };

The way you're supposed to implement the move constructor is:

  Widget::Widget(Widget&& rhs)
  : s(std::move(rhs.s))
  {}

Per convention, std::move is applied to the RRef rhs when initializing the std:string.  If we used std::forward, the code would look like this:

  Widget::Widget(Widget&& rhs)
  : s(std::forward<std::string>(rhs.s)
  {}

You can't see it in the pseudocode for std::forward, but even though it's a function template, the functions it generates don't do type deduction.  Preventing such type deduction is one of the things that make std::forward's implementation less than transparent.  Because there is no type deduction with std::forward, the type argument T must be specified in the call.  In contrast, std::move does do type deduction, and that's why in the Widget move constructor, we say "std::move(rhs.s)", but "std::forward<std::string>(rhs.s)".

In the call "std::forward<std::string>(rhs.s)", the type std::string is a non-reference. As a result, std::forward returns its argument as an rvalue, which is exactly what std::move does.  That answers the original question.  If you apply std::forward to an rvalue reference instead of std::move, you get the same result. std::forward on an rvalue reference does the same thing as std::move.

Now, to be fully accurate, this assumes that you follow the rules and pass to std::forward the type of the RRef without its reference-qualifiers.  In the Widget constructor, for example, my analysis assumes that you pass std::forward<std::string>(rhs.s).  If you decide to be a rebel and write the call like this,

  std::forward<std::string&>(rhs.s)

rhs.s would be returned as an lvalue, which is not what std::move does. It also means that the std::string data member in Widget would be copy-initializd instead of move-initialized, which would defeat the purpose of writing a move constructor.

If you decide to be a smart aleck and write this,

  std::forward<std::string&&>(rhs.s)

the reference-collapsing rules will see that you get the same behavior as std::move, but with any luck, your team lead will shift you to development in straight C, where you'll have to content yourself with writing bizarre macros.

Oh, and if you make the mistake of writing the move constructor like this,

  Widget::Widget(Widget&& rhs)
  : s(std::forward<Widget>(rhs.s))
  {}

which, because I'm so used to passing std::forward the type of the function parameter, is what I did when I initially wrote this article,  you'll be casting one type (in this case, a std::string) to some other unrelated type (here, a Widget), and I can only hope the code won't compile.  I find the idea so upsetting, I'm not even going to submit it to a compiler.

Summary time:

• If you use std::forward with an RRef instead of std::move, and if you pass the correct type to std::forward, the behavior will be the same as std::move.  In this sense, std::move is superfluous.
• If you use std::forward instead of std::move, you have to pass a type, which opens the door to errors not possible with std::move.
• Using std::forward requires more typing than std::move and yields source code with more syntactic noise.
• Using std::forward on an RRef is contrary to established C++11 idiom and contrary to the design of move semantics and perfect forwarding.  It can work, sure, but it's still an anathema.

Scott

"Universal References in C++11" now online at ISOCpp.org

The October Overload article I blogged about earlier, "Universal References in C++11," is now available online at the spanking new ISOCpp.org website. In conjunction with my video presentation at Channel 9 on the same topic, this means you now have your choice of three different ways to experience my pitch for adding "universal reference" to the accepted C++ vocabulary:

• Overload version: Monochrome PDF and, if you're a subscriber, hardcopy.
• Channel 9 version: Video of me presenting the information, along with the original presentation materials.
• ISOCpp.org version: HTML version with syntax-colored code examples and live hyperlinks.

I really think that the idea of universal references helps make C++11's new "&&" syntax a lot easier to understand. If you agree, please use this term and encourage others to do it, too.

Scott

The Standard works in Circular Ways

Rummaging around in the C++11 standard isn't all fun and games, but from time to time you come across true gems.  For example, this is from 2.14.7/1 in the standard:

The pointer literal is the keyword nullptr. It is a prvalue of type std::nullptr_t.

And this is from 18.2/9:

nullptr_t is defined as follows:

namespace std {
  typedef decltype(nullptr) nullptr_t;
}

So nullptr is of type nullptr_t, and nullptr_t is defined to be the type of nullptr.

Um, okay.

Scott

Video for "Adventures in Perfect Forwarding" Now Available

On Saturday, June 2, Facebook sponsored a one-day C++ conference and asked me (and others) to give a presentation.  I chose an abridged and updated version of a talk from C++ and Beyond 2011, "Adventures in Perfect Forwarding."  Judging by the dates on the comments below the video, it's been available since July, but I found out about it being online only today. (Apparently I don't have enough friends.)

The video is broken into two parts, each about 30 minutes long:

• Part 1

• Part 2

Due to technical difficulties at Facebook during the talk, some audio is missing and other audio is out of sync with the video.  My understanding is that these issues can't be addressed, so we have to live with the limitations of the recording.

Facebook does not appear to have made the presentation materials available, but you can grab them here (PDF).

I hope you enjoy this talk.

Scott

Updated Release of "Effective C++ in an Embedded Environment"

In April 2010, I made the presentation materials for two of my training courses available for purchase. Purchasers get free updates for life. The first set of materials, An Overview of the New C++ (C++11), has since seen five update releases, and a sixth will occur sometime later this year. The other course, Effective C++ in an Embedded Environment, has not been revised. Until now. A new version has just been published, and if you have purchased these materials, you should have already been notified about how to download the new version.  If you have not (e.g., because you asked Artima, the publisher of these materials, not to send you email), Artima asks me to remind you that

You can log into Artima with the account you used to purchase the book, click on "Your Settings", and redownload anytime. If you've forgotten your password, you can get a reminder here.

It would be a mistake to interpret the greater frequency of updates for the C++11 materials as meaning that I pay less attention to the materials on using C++ in embedded systems.  I don't.   The simple fact is that C++11 was in a state of flux up until final standardization, and since then compilers have been adding increasing feature support, plus I've been learning more about the features in C++11 and how to use them.  There's been a lot happening in the C++11 world, and I've been working hard to keep my training materials current (while at the same time not bombarding you with updates).

The world of C++ in embedded systems has been stabler, and so have my materials devoted to that topic.  Even so, the changelog for the materials (available with your update) lists some five dozen changes affecting around 80 pages—about 25% of the total slides.  None of the revisions are earth-shattering, but a lot of little things have been fixed or clarified or otherwise improved. I have a strong incentive to keep these materials in as good a shape as I can: they're the ones I use myself.

I hope you enjoy the latest versions of my training materials on the use of C++ in embedded systems.  As always, you can download a free sample of the materials (the first 30 pages) from the materials' sales page.

Scott

Copying Constructors in C++11

Today I read Martinho Fernandes' post, "Some pitfalls with forwarding constructors." What caught my attention was his claim that a forwarding constructor (i.e., a templatized constructor taking a universal reference) is a copy constructor.  "Can't be true," was my initial thought, because in the past it was true that it couldn't be true. But then I kicked myself.  "Crawl out of the 1990s," I told myself (meaning 1998, the year of the first C++ standard).  C++11 has new rules, and with new rules comes a new game.

Setting aside edge cases we need not fritter away time on, the copy constructor for a class C takes a parameter of type const C&.  If there is no such function declared in a class and such a function is needed (because it is used somewhere), it is automatically generated.  This is absolutely true in C++98 and is typically true in C++11, and the difference between absolutely and typically is not of interest here.  Where I'm going with this does not require acknowledging the nooks and crannies along the way.

In C++98, you could use a template to write what I called a "generalized copy constructor," and you did it like this:

class Widget {
public:
  template<typename T>
  Widget(const T&);
  // ...
};

The key thing to recognize was that this template, though it could be instantiated to yield the signature of a copy constructor, is itself not a copy constructor, and that means that the class does not declare one.  That being the case, if the copy constructor is needed, the compiler generates one as usual, because that's the rule: if there is no user-declared copy constructor (i.e., constructor taking a const Widget& parameter) and one is needed, the compiler generates one.  The fact that such a function could be generated from a user-declared template is irrelevant.

When the compiler sees a request to copy an object, it still has to consider the possibility of calling a template instantiation, but the template instantiation has the same signature as the implicitly-generated copy constructor, and (1) the copy constructor is a function (not a template function) and (2) when a function and template function are equally good matches for a function call, overloading resolution chooses the function.  The end result is that copy constructors can never be generated from templates.

To the best of my knowledge, all this remains true in C++11.

What changes in C++11 is the possibility of writing a generalized copy constructor taking a universal reference--a universal copy constructor:

class Widget {
public:
  template<typename T>
  Widget(T&&);        // now takes universal reference
  // ...
};

Consider now this code:

  Widget w;
  Widget wcopy = w;           // copy w to wcopy

This is a request to copy w. Widget has no copy constructor, so the compiler generates one with the default signature. The resulting class looks like this:

class Widget {
public:
  Widget(const Widget&);         // copy constructor (compiler-generated)  
  template<typename T>   
  Widget(T&&);        // universal copy constructor

  // ...
};

When the compiler see the request to copy w, it knows it can call the copy constructor, just like it could in C++98, but it also knows that it can instantiate the template. Unlike the C++98 template, however, this one lacks a const-qualification on its parameter, so the resulting instantiation also lacks one. After instantiation, the class effectively looks like this:

class Widget {
public:
  Widget(const Widget&);         // copy constructor (compiler-generated)  
  Widget(Widget&);               // instantiated template

  // ...
};

Overload resolution now kicks in to choose from these two candidate functions. (Technically, one is a template function, not a function, and to be really technical about it, it's a template specialization, not a template instantiation, but none of that matters here.)

The parameter to these functions would be the object being copied.  That's w.  w is not const.  A call to the compiler-generated copy constructor would require the addition of const to get an exact match, but the call to the instantiated template requires no such addition.  As a result, it's a better match.  The copy constructor will not be called to copy w.  Instead, the instantiated template will.

The situation changes if we copy a const Widget:

  const Widget cw;
  Widget cwcopy = cw;           // copy a const Widget

Now we're back in C++98-land. The compiler-generated copy constructor and the template instantiation have the same signature (both take const Widget& parameters), so the function wins.

At least that's how I view it.  gcc 4.7 agrees.  Given this program,

#include <iostream>

class Widget {
public:
  Widget(){};

  Widget(const Widget&) { std::cout << "Widget copy ctor  "; }

  template<typename T>
  Widget(const T&) { std::cout << "Generalized Widget copy ctor  "; }

  template<typename T>
  Widget(T&&) { std::cout << "Universal Widget ctor  "; }
};

void endLine() { std::cout << '\n'; }

int main()
{
  Widget w;

  {
    std::cout << "Create Widget from Widget:\n";
    std::cout << "  Direct init w/parens: ";   Widget wcopy1(w);  endLine();
    std::cout << "  Copy init           : ";   Widget wcopy2 = w; endLine();
    std::cout << "  Direct init w/braces: ";   Widget wcopy3 {w}; endLine();
    endLine();
  }

  const Widget cw;

  {
    std::cout << "Create Widget from const Widget:\n";
    std::cout << "  Direct init w/parens: ";   Widget wcopy1(cw);  endLine();
    std::cout << "  Copy init           : ";   Widget wcopy2 = cw; endLine();
    std::cout << "  Direct init w/braces: ";   Widget wcopy3 {cw}; endLine();
    endLine();
  }
}

gcc 4.7 produces this output:

Create Widget from Widget:
  Direct init w/parens: Universal Widget ctor
  Copy init           : Universal Widget ctor
  Direct init w/braces: Universal Widget ctor

Create Widget from const Widget:
  Direct init w/parens: Widget copy ctor
  Copy init           : Widget copy ctor
  Direct init w/braces: Widget copy ctor

Nothing surprising there, but if I use auto to deduce the type of the copy, gcc's output changes. That is, this source code,

int main()
{
  Widget w;
  {
    std::cout << "Create auto from Widget:\n";
    std::cout << "  Direct init w/parens: ";   auto wcopy1(w);   endLine();       // now using auto
    std::cout << "  Copy init           : ";   auto wcopy2 = w;  endLine();
    std::cout << "  Direct init w/braces: ";   auto wcopy3 {w};  endLine();
    endLine();
  }

  const Widget cw;
  {
    std::cout << "Create auto from const Widget:\n";
    std::cout << "  Direct init w/parens: ";   auto wcopy1(cw);   endLine();
    std::cout << "  Copy init           : ";   auto wcopy2 = cw;  endLine();
    std::cout << "  Direct init w/braces: ";   auto wcopy3 {cw};  endLine();
    endLine();
  }
}

produces this output:

Create auto from Widget:
  Direct init w/parens: Universal Widget ctor
  Copy init           : Universal Widget ctor
  Direct init w/braces: Universal Widget ctor  Universal Widget ctor

Create auto from const Widget:
  Direct init w/parens: Widget copy ctor
  Copy init           : Widget copy ctor
  Direct init w/braces: Widget copy ctor  Universal Widget ctor

This is surprising. Where are those extra constructor calls coming from? (In a comment on my last post, Julain posted results for clang, which did not show such extra calls.)

I had a theory.  To test it, I added a move constructor to Widget,

class Widget {
public:
  Widget(){};

  Widget(const Widget&) { std::cout << "Widget copy ctor  "; }
  Widget(Widget&&) { std::cout << "Widget move ctor  "; }           // added this

  template<typename T>
  Widget(const T&) { std::cout << "Generalized Widget copy ctor  "; }

  template<typename T>
  Widget(T&&) { std::cout << "Universal Widget ctor  "; }
};

then reran the test (using auto-declared variables).  The revised results?

Create auto from Widget:
  Direct init w/parens: Universal Widget ctor
  Copy init           : Universal Widget ctor
  Direct init w/braces: Universal Widget ctor

Create auto from const Widget:
  Direct init w/parens: Widget copy ctor
  Copy init           : Widget copy ctor
  Direct init w/braces: Widget copy ctor

No extra constructor calls. Funky, no?

Some of the following analysis is incorrect, but rather than simply edit the post to remove the wrong information and replace it with correct information, I'll add comments like this.

gcc is making an extra call to the universal copy constructor only when I use brace initialization.  My theory is based on the observation that when brace initialization is used to initialize a std::initializer_list object, the values in the braced list are copied into a temporary rvalue array.  There are no std::initializer_list objects in my code, but let's suppose, for reasons I cannot fathom, that the use of auto in the above code somehow prompts gcc to treat a braced initializer as if it were being used with a std::initializer_list.

There are std::initializer_list objects in my code, because, as a prospective guideline for Effective C++11 puts it, "Remember that auto + {} == std::initializer_list." The statement

auto wcopy {w};

creates an object of type std::initializer_list<Widget>, not, as I had thought when I wrote the original post, an object of type Widget.

If that were the case, the to-be-copied Widget would be copied into the array, where it would be an rvalue that would then be moved into the object being initialized.  That is, given

auto wcopy {w};

w would be copied into an array, and a request would then be made to move this array element into wcopy. But in the code where I declare no move constructor, no move constructor can be generated, because I've declared a copy constructor, and if you declare a copy operation, no move operators are generated.  So in the code that requests a move of the rvalue in the array, the move is accomplished by calling a copy operation.

Because the object in the array is a copy of the actual initialization argument, it's not const, even if what it's a copy of is.  In this code,

const Widget cw;
auto cwcopy {cw};

though cw is const, the copy of cw in the array is not.  That means that when the compiler has to choose between the copy constructor (which takes a const Widget& parameter) and the instantiated universal copy constructor (which, in this case, takes a Widget& parameter), the universal copy constructor is the better match.

Remember, this is all a theory.  But it does explain the output from gcc.  In particular, it explains:

• Why there is an extra call to a copying operation:  because the first copy operation copies the initializing object into the array, and the second one copies it from the array to the target of the initialization.
• Why the extra call is to the universal copy constructor: because any cv qualifiers present on the initializing object are absent from the copy in the array, and the universal copy constructor is a better match than the conventional copy constructor for non-const objects.
• Why the extra call goes away when I declare a move constructor: because then the move constructor can be called on the rvalue in the array, but calls to copy constructors and move constructors can be optimized away under certain conditions (which are fulfilled here).  Such call elision is generally not permitted for other functions, including the universal copy constructor.   Which means that when the move constructor is missing and cannot be generated, the universal constructor must be called, but when the move constructor is present, it need not be called.  

Are we having fun yet?

If my theory is correct, I think it identifies a bug in gcc, because, as far as I know, the only time the contents of a braced initializer list are copied into a temporary array is when they're being used with a std::initializer_list object.  (It may happen in calls to std::async and the variadic std::thread constructor, too, but I'm too lazy to look it up right now.)

I no longer believe that this is a bug in gcc, because gcc is doing exactly what I'd guessed:  initializing a std::initializer_list<Widget> object. I just didn't think it was supposed to.  Now I do.

Anyway, bottom line: in C++98, copy constructors could not be generated from templates, but in C++11, they can be, practically speaking.  At least that's my current understanding.  Thanks to Martinho Fernandes for his post that sent me down this particular rabbit hole.

Thanks also to Xeo for pointing out the error in my original post.

Scott

Posting C++ Code

I decided that if I'm going to be posting code, I might as well figure out how to make it look like code. In a comment on my last post, Nuno Barreiro's suggested I use SyntaxHighlighter, so I decided to give it a try.

Never in the history of software development has there been a tool with more explicit no-brains-are-required installation and usage tutorials.  My favorite is this one, because, really, how much more can you ask than for screen shots with arrows pointing where to click and "Click this" directives?  I threw caution to the wind and did exactly as I was told. Security be damned, I wanted my code to be syntax-highlighted, and I was in a hurry!

It didn't work.  The syntax of my code was not highlighted, and I got an error telling me that the brush I needed couldn't be found. An hour or so of googling and trial and error and more googling and more trial and more error ensued before I figured out that the name of the brush I needed was "c++", not "C++". That despite the fact that the documented brush name is "C++" and the name of the JavaScript file is shBrushCpp.js.  (Alias brush names are "cpp" and "c", both of which also work, but by the time I started playing around with the aliases, I was beyond noticing that the file name contained "Cpp", but the alias is "cpp".  My bad.)

To get code to display properly, you have to enter it in HTML mode (at least in Blogger, which is the platform I'm using), and you have to remember to transform all opening angle brackets ("<") into their fetching HTML equivalents ("&lt;"). I use Centricle.com's Encode/Decode HTML Entities page for that.

To get C++ code highlighting in Blogger, then (and I'm posting this here more for me than in the hope it might be useful to somebody else) you first blindly point, click, copy, and configure as per this page (initial setup) and this page (additional configuration). For each code fragment you want to appear in your blog, you:

1. Write the code elsewhere, presumably where you can easily edit and test it. I use Emacs, but I've heard tell that there are other code and text editors.
2. Transform the code into HTML-friendly form via something like the Encode/Decode HTML Entities page.
3. Copy the transformed code into the HTML editor in Blogger.  Put said code into a pre block as follows:

<pre class="brush: c++">
Your code goes here
</pre>

In the words of technical support professionals everywhere, it works for me.

As a demonstration, here's some code I've been playing around with this evening.  The goal is to see which of several potential constructors is called when an object (an lvalue, if you insist on knowing) is copied. Try to predict what it will do before you feed it to your compiler. When I run it under gcc 4.7, I'm surprised by what I see.

#include <iostream>

class Widget {
public:
  Widget(){};

  Widget(const Widget&) { std::cout << "Widget copy ctor  "; }

  template<typename T>
  Widget(const T&) { std::cout << "Generalized Widget copy ctor  "; }

  template<typename T>
  Widget(T&&) { std::cout << "Universal Widget ctor  "; }
};


void endLine() { std::cout << '\n'; }


int main()
{
  Widget w;

  {
    std::cout << "Create Widget from Widget:\n";
    std::cout << "  Direct init w/parens: ";   Widget wcopy1(w);  endLine();
    std::cout << "  Copy init           : ";   Widget wcopy2 = w; endLine();
    std::cout << "  Direct init w/braces: ";   Widget wcopy3 {w}; endLine();
    endLine();
  }


  {
    std::cout << "Create auto from Widget:\n";
    std::cout << "  Direct init w/parens: ";   auto wcopy1(w);   endLine();
    std::cout << "  Copy init           : ";   auto wcopy2 = w;  endLine();
    std::cout << "  Direct init w/braces: ";   auto wcopy3 {w};  endLine();
    endLine();
  }

  const Widget cw;

  {
    std::cout << "Create Widget from const Widget:\n";
    std::cout << "  Direct init w/parens: ";   Widget wcopy1(cw);  endLine();
    std::cout << "  Copy init           : ";   Widget wcopy2 = cw; endLine();
    std::cout << "  Direct init w/braces: ";   Widget wcopy3 {cw}; endLine();
    endLine();
  }

  {
    std::cout << "Create auto from const Widget:\n";
    std::cout << "  Direct init w/parens: ";   auto wcopy1(cw);   endLine();
    std::cout << "  Copy init           : ";   auto wcopy2 = cw;  endLine();
    std::cout << "  Direct init w/braces: ";   auto wcopy3 {cw};  endLine();
    endLine();
  }
}

Update 10/17/12: Having run across this blog post, I also gave Google's Prettify a try. I don't like the result as well as that of SyntaxHighlighter, but judge for yourself.  Here's a prettified version of the Widget class above:

class Widget {
public:
  Widget(){};

  Widget(const Widget&) { std::cout << "Widget copy ctor  "; }

  template<typename T>
  Widget(const T&) { std::cout << "Generalized Widget copy ctor  "; }

  template<typename T>
  Widget(T&&) { std::cout << "Universal Widget ctor  "; }
};

Update 6/12/20: I removed support for SyntaxHighlighter and Google Prettify from the blog, because neither seems to be supported any longer. I did only cursory testing, but I'm hoping that the result will be that code examples in my blog posts continue to look like code, except they won't be syntax-highlighted.

Parameter Types in Constructors

[This is the first time I've tried to include code fragments in a blog post, and let's just say I was surprised at how badly Blogger deals with them.  I apologize for any formatting problems, and I welcome suggestions on how to get Blogger to swallow code displays without throwing a fit.]


I recently went through Sumant Tambe's presentation materials from his Silicon Valley Code Camp presentation, "C++11 Idioms."  He argues that an emerging idiom is to pass arguments to constructors by value, because this takes advantage of move opportunities when they are available.  I was surprised to read about this emerging idiom, in part because I had not heard of it (I'm supposed to be clued in about this kind of stuff) and in part because it runs contrary to my own thinking on the subject, which is to use perfect forwarding. 

His presentation goes on to discuss the general problem of efficient parameter passing, based on a trio of blog posts by Boris Kolpackov.  (Part 1 of that trio is here.) In this discussion, Sumant does mention perfect forwarding, but he argues that it has some significant drawbacks.  First, he notes, it requires use of a template.  Second, he remarks that you'll need to restrict the template arguments to the types you want, "most likely using enable_if."  Third, he says you need to query the types to determine whether they are rvalues, and he shows this code:

#define IS_RVALUE(TYPE, VAR) \
(std::is_rvalue_reference<decltype(std::forward<TYPE>((VAR)))>::value)

He concludes that "it is too much noise!"   

Regarding the use of templates, well, I have to say that if you view the use of templates as a notable downside in C++ software development, you should probably rethink your choice of programming language.  For at least the past decade, templates have been as mainstream as classes.  

Regarding the code needed to find out if an argument was an rvalue, that information is encoded in the deduced type of the perfect forwarding function template, so all you really have to say is:
!std::is_reference<T>::value
Hardly intimidating.

That leaves the issue of restricting template argument types and the use of enable_if.  The case that Boris (and hence Sumant) considers is a operator+ for matrix objects, but I want to reconsider the first problem Sumant addresses (how to declare constructor arguments) with perfect forwarding in mind, because, as it turns out, enable_if plays a role here that I think clarifies the situation.

Suppose I have a Widget class with a std::string data member.  In C++98, I'd initialize it like this:

class Widget1 {
public:
  Widget1(const std::string& n): name(n) {}

private:
  std::string name;
};

The problem here is that if an rvalue is passed to the Widget1 constructor, name is copy-initialized from n instead of being move-initialized.  That's easy to fix using perfect forwarding, which is designed to optimize this kind of thing:

class Widget2 {
public:
  template <typename T>
  Widget2(T&& n): name(std::forward<T>(n)) {}

private:
  std::string name;
};

No muss, no fuss, no enable_if, no testing for whether the argument passed to n was an rvalue--none of that stuff.  Very nice.

But let's suppose there's another way to initialize a Widget.  Instead of passing a name, we can pass an ID number, and the name can be looked up from the ID number.  In C++98, adding this would be trivial:

class Widget3 {
public:
  Widget3(const std::string& n): name(n) {}
  Widget3(int id): name(findNameFromID(id)) {}

private:
  std::string name;
};

Adding this to the version of Widget with the perfect forwarding constructor, however, leads to grief.  But not immediately. The class will compile without any trouble, and simple tests will work as expected:

class Widget4 {
public:
  template <typename T>
  Widget4(T&& n): name(std::forward<T>(n)) { std::cout << "T&&\n"; }
  Widget4(int id): name(findName(id)) { std::cout << "int\n"; }

private:
  std::string name;
};

Widget4 w1("Hello");      // calls perfect forwarding ctor
Widget4 w2(22);           // calls int constructor

But slightly trickier tests fail.  For example, if the type of the passed ID isn't exactly int--e.g., a long or a short or an unsigned--the perfect forwarding constructor is a better match, and the code will try to initialize a std::string with a numerical value.  This makes no sense, and the compiler will exhibit no reluctance in telling you that.  Given

Widget4 w3(22L);          // pass a long

gcc 4.7 says this:

ctors.cpp: In instantiation of 'Widget4(long int &&)':
ctors.cpp:40:18:   required from here
ctors.cpp:18:42: error: invalid conversion from 'long int' to 'const char *' [-
    fpermissive]

basic_string.h:487:7: error: init. arg 1 of 'basic_string<
        char; _Traits = char_traits<char>; _Alloc = allocator<char>, _Traits
      , _Alloc
    >::basic_string(
        const char; _Traits = char_traits<char>; _Alloc = allocator<char> *
      , const _Alloc &
    )' [-fpermissive]
In file included from c:\users\scott\apps\mingw\bin\../lib/gcc/i686-pc-
    mingw32/4.7.0/../../../../include/c++/4.7.0/string:54:0,
from ctors.cpp:1

The fundamental problem is that perfect forwarding and overloading make very bad bedfellows, because perfect forwarding functions want to take everything. They're the greediest functions in C++.

To curb their avarice, you have to limit the conditions under which they can be instantiated, and that's where enable_if comes in.  And where enable_if comes in, any semblance of beauty leaves.  For example, if you'd like to disable the perfect forwarding template for integral types (thus allowing the int overload to handle those types), you could code it this way:

class Widget5 {
public:
  template <typename T>
  Widget5(T&& n, typename std::enable_if<!std::is_integral<T>::value>::type* = 0)
    : name(std::forward<T>(n)) { std::cout << "T&&\n"; }
  Widget5(int id): name(findName(id)) { std::cout << "int\n"; }

private:
  std::string name;
};

This code behaves as we'd like, assuming (sigh) we'd like to treat a char as a number:

  Widget5 w3a("Sally");           // calls perfect fowarding ctor
  Widget5 w3b(44);                // calls int ctor
  Widget5 w3c(44L);               // calls int ctor
  Widget5 w3d((short)44);         // calls int ctor
  Widget5 w3e('x');               // calls int ctor

If more constructor overloads are added, or if the constraints on when the perfect forwarding constructor is applicable become more complicated, the enable_if-related code can become, er, unpleasant. Such unpleasantness is presumably the noise that Sumant Tambe refers to when he rejects perfect forwarding.

In those cases, I think his preferred solution to the problem of declaring constructor parameters--take everything by value--is reasonable.   The generated code isn't quite as efficient as you'd get from perfect forwarding, but the overloading rules are a lot easier to understand, and the resulting code is easier to both read and write.

I still think that perfect forwarding is the language tool you should prefer when you need to write constructors and similar functions (e.g., setters) that simply shuttle values from one place to another.  That's what it's designed for.  If you need to overload such functions, however, things get very messy very quickly, and except in the most demanding of performance-sensitive applications and libraries, I think it's reasonable to fall back on pass-by-value as the parameter-passing mechanism.

But these are still early days in C++11 programming, and we're all still learning.  I welcome your thoughts on how to declare constructor parameters, the proper role of perfect forwarding, and the wisdom of passing parameters by value.

Scott

Video for "Universal References in C++11" Now Available

A couple of days ago, I blogged that my article on "Universal References" had been published by Overload, and I said a video of me presenting this topic at C++ and Beyond would soon be available.  Soon has become now. Servers are standing by, and on-demand viewing is available at Channel 9.  Many thanks to Charles Torre and to Microsoft for recording and hosting the presentation. I hope you like it and find it useful.  If you do, please tell others about it, because I'm trying to inject the term universal reference into the C++ vocabulary, and I can't do it alone.

Scott

New Article: "Universal References in C++11"

One of the most difficult aspects of C++11 for me to wrap my head around was the meaning of "T&&" in type declarations.  “It means rvalue reference,” I was told, and that was fine until I was also told that in some cases, it actually means lvalue reference.  I fought with this seeming contradiction for a couple of years before my head cleared and I decided that we need to distinguish declarations for rvalue references from declarations for things that look like rvalue references, but may not be.  I call these latter things universal references.  I hope the term catches on, because I think it makes the matter a lot easier to understand.

I gave a talk on universal references at this year's C++ and Beyond, and a video of that talk should go live sometime this month.  (I'll let you know when.)  I also wrote an article on the topic for ACCU's Overload, and the issue with the article (PDF) has just been published.  This is from the beginning of that article:

Given that rvalue references are declared using “&&”, it seems reasonable to assume that the presence of “&&” in a type declaration indicates an rvalue reference. That is not the case:

Widget&& var1 = someWidget;       // here, “&&” means rvalue reference

auto&& var2 = var1;               // here, “&&” does not mean rvalue reference

template<typename T>
void f(std::vector<T>&& param);   // here, “&&” means rvalue reference

template<typename T>
void f(T&& param);                // here, “&&” does not mean rvalue reference

In this article, I describe the two meanings of “&&” in type declarations, explain how to tell them apart, and introduce new terminology that makes it possible to unambiguously communicate which meaning of “&&” is intended. Distinguishing the different meanings is important, because if you think “rvalue reference” whenever you see “&&” in a type declaration, you’ll misread a lot of C++11 code.

I hope you find the article both interesting and useful.

Scott

The Projucer

A lot of uninvited stuff shows up in my electronic mailbox. There's spam, sure, plus my share of links, stories, and pictures from friends and family members who are more easily entertained than I am.  I also get questions and comments and links to C++- and software-related things people think---or hope---I might be interested in. I skip a lot of the "entertainment" I get from friends and family (none of whom read this blog, so I'm not worried about offending them), but in the realm of C++, I look at everything I get, because it's part of my job as a C++ consultant. Most of the stuff I see I dismiss without action, because I don't think it's interesting or I don't see how it's useful or I simply don't have time to really analyze and pursue it.  There are, after all, only so many hours in a day.

Julian Storer
(unless he's using
a picture of
somebody else)

Every once in a while, something piques my interest, and not always for the reason the sender imagines.  Such was the case about three weeks ago, when I got a message from Julian Storer. He wrote (with minor editing and link addition on my part):

For the last decade I've been running a big C++ cross-platform toolkit called JUCE. Won't attempt to describe it in this email - there's plenty of info about it on my website if you want to learn more.

I've just been making some announcements about a new tool that I'm aiming to release soon, and being proud of what I've managed to cook-up, I thought it'd be worth pestering some of my C++ guru heros to have a look at it. It's something that nobody's managed to do in C++ before, so I think you'll probably find it interesting!

The TL;DR is: I've built an IDE that uses Clang and the LLVM JIT engine to provide real-time interactive execution of C++ GUI code, and to allow GUI editing by automatic refactoring of user-code... Hard to sum it up in a sentence, but if you saw the Bret Victor demo that everyone was talking about a few months ago, you'll get the gist - he

was showing off interactive javascript, and I've managed to do the same trick in C++.

Hope you can spare 10 mins to check it out!

I did check it out, and not just because he called me a C++ guru hero. Though I had not watched all of  Bret Victor's demo, I had watched part of it, so I had some idea what was involved, and the idea that something similar could be done for C++ intrigued me.  Also, when people tell me they have done something in C++ that nobody has done before, that tends to catch my attention. Most important, at the time I was trying to figure out what to do with Julian's email, I had much more pressing things to work on that I didn't want to attack yet, so I needed a way to tell myself I was working without actually, you know, working.  So I toddled over to the screencast Julian had put together and invested the six minutes and thirty-six seconds it runs.

It's a nifty demo, but the application area is GUI development, and that's not really an area I know a lot about.  So when he ran the GUI from the compiled application, changed the application source code, then showed how the GUI updated itself, I was impressed with what must be happening behind the scenes, but I didn't have to hold onto my hat, nor was I concerned that my socks would be knocked off.  And when he modified the running GUI and some literals changed in the source code, I thought it was, well, nice. But then I saw something that made me replay that part of the video several times.  Not only did the literals in the source code change when he moved a GUI widget around, he was also able to have new statements added to the source code by interactively editing the running GUI.  He was editing the program source code through the GUI.

Maybe this is old news.  Maybe Visual Studio and Eclipse and various other IDEs for GUI development allow you to create and edit source code via both text (traditional source code) and generated GUI (arguably an alternative form of source code).  As I said, I'm not a GUI development guy, so I have no idea what the state of the art in this area is.  What I do know is that Julian's demo is consistent with the idea that there is no such thing as canonical source code.  C++ text is source code, of course, but there's no reason that the GUI generated from that source code can't be viewed as source code, too.  Since they're both source code, it makes sense that the underlying program can be edited through either.  One could argue that each representation of the program (C++ text and displayed GUI) is simply a view of that program, and the program itself is something more fundamental--something more abstract that transcends the concrete syntax and semantics of whatever view is used to see and manipulate it.

I'd like to believe that this is a reasonable argument, because, in essence, this was the topic of my PhD research many years ago.  My dissertation, Representing Software Systems in Multiple-View Development Environments, explored how one could represent programs such that they could be viewed and edited through any of a variety of concrete syntaxes ("programming languages").  Let me save you the trouble of reading my dissertation by summarizing my conclusion: It's hard.

I'm guessing that Julian was not expecting that his demo would remind me of the research I was pursuing over 20 years ago, but it did.  And that's why I'm blogging about it. Besides, as far as I know, he has done something in C++ that nobody has done before, and groundbreakers like that deserve encouragement. If you're wise in the ways of GUI development with C++, I encourage you to watch Julian's screencast and offer him feedback to help him continue his work.  I'm sure he'd be pleased to hear from you.

Scott