Today I got email about some information in Effective Modern C++. The email included the statement, "An expression never has reference type." This is easily shown to be incorrect, but people assert it to me often enough that I'm writing this blog entry so that I can refer people to it in the future.
Section 5/5 of the Standard is quite clear (I've put the relevant text in bold):
If an expression initially has the type “reference to T” (8.3.2, 8.5.3), the type is adjusted to T prior to any further analysis. The expression designates the object or function denoted by the reference, and the expression is an lvalue or an xvalue, depending on the expression.There'd clearly be no need for this part of the Standard if expressions couldn't have reference type.
If that's not enough to settle the matter, consider the type of an expression that consists of a function call. For example:
int& f(); // f returns int&
auto x = f(); // a call to f
What is the type of the expression "f()", i.e., the type of the expression consisting of a call to f? It's hard to imagine anybody arguing that it's not int&, i.e., a reference type. But what does the Standard say? Per 5.2.2/3 (where I've again put the relevant text in bold and where I'm grateful to Marcel Wid for correcting the error I had in an earlier version of this post that referred to 5.2.2/10):If the postfix-expression designates a destructor (12.4), the type of the function call expression is void; otherwise, the type of the function call expression is the return type of the statically chosen function (i.e., ignoring the virtual keyword), even if the type of the function actually called is different. This return type shall be an object type, a reference type or cv void.It's very clear that expressions can have reference type. Section 5/5 takes those expressions and strips the reference-ness off of them before doing anything else, but that's not the same as the reference-ness never being present in the first place.
Scott
Hi,
ReplyDeleteI have another example of expression with reference type:
int a = 1, b = 2, c = 3;
(b += c) = a;
Also lambda is expression with reference type.
I think by the same argument one could say that function parameters may be array types:
ReplyDelete> After determining the type of each parameter, any parameter of type “array of T” or “function returning T” is adjusted to be “pointer to T” or “pointer to function returning T,” respectively. [dcl.fct]/5
The wording could easily be changed to "if the type would otherwise be a reference type/array type," and then the wording would no longer support your claim.
Finally, the wording in the second quote does not refer to the type of an expression, just the the 'result type' of a function, which is not necessarily identical to the type of a function call expression calling the function.
@Anonymous
I believe even by Scott's argument that expression does not have a reference type; it's simply an lvalue of type int, because the built-in += operator is specified to return "return an lvalue referring to the left operand."
---
I think it's good to understand what is meant when somebody says, informally, 'the expression has a reference type', but I'm not sure there's a real, practical need to take the terms 'before' and 'after' too literally when the spec uses them this way.
@Seth: I'd be interested to know what type you think the expression "f()" has, according to the Standard.
ReplyDeleteDetermining the type of the expression `f()` includes applying [expr]/5. That is, one cannot be said to have determined the type until all applicable rules have been applied.
ReplyDeleteSo I would say that the expression `f()` formally has type `int` (and the expression is an lvalue). I might well informally describe `f()` as having int reference type.
@Seth: But what type do you start with, i.e., to what type to you apply "the applicable rules"?
ReplyDeleteTo start with, no type at all. Then my understanding would be updated/corrected to 'int&' by the application of [expr.call]/3. Then it would be updated/corrected to 'int' by the application of [expr]/5.
ReplyDelete@Seth: So the type of the call expression is int&, per 5.5.2/3. The adjusted type (per the wording in 5/5) is int.
ReplyDeleteInterestingly, 5/5 seems to be the motivation for the decltype rule in 7.1.6.2/4 bullet 3 that add a reference-qualifier to the type of lvalue expressions. That is, 5/5 takes the reference-qualifier away from expressions with reference type, and, for lvalues, 7.1.6.2/4 bullet 3 adds it back. (I am aware that 7.1.6.2/4 bullet 3 yields reference types for some lvalue expressions that were not originally of reference type, e.g., index operations into an array, but my sense is that its main job is to restore reference qualifiers that 5/5 removed.)
So if you were asked to describe the type of the parameter in `void foo(int x[5])` would you say its type is an array of 5 ints? And then only when asked about the "adjusted type" of the parameter say that the type is an int pointer, per [dcl.fct]/5's wording that 'After
ReplyDeletedetermining the type of each parameter, any parameter of type “array of T” [...] is adjusted to be “pointer to T”'?
I don't believe that these figurative uses of 'before' and 'after' should be taken literally and the 'intermediate' states referred to by the standard should not be treated as part of the program's actual meaning. There is no literal timeline in which the program means one thing and then means another as additional rules from the spec are applied. The program has one meaning, and that meaning is the final result of applying the whole standard.
But again, this is only when speaking formally. If distinguishing between an expression's 'non-adjusted type' and its 'adjusted type' is helpful in explaining some point then I think it's perfectly reasonable to speak informally in such terms.
Nor does such an informal description need any cover from the standard; if the standard in fact used the wording 'if an expression's type would otherwise be "reference to T" the expression's type is actually T,' it could still be reasonable to talk about expressions with reference types.
@Seth: Personally, I'd say that the parameter in your example is declared to be an array of 5 ints, but it's treated as if it had been declared to be a pointer.
ReplyDeleteFWIW, the situations where I tend to run into trouble with people regarding whether expressions can have reference type are when I make statements such as this (from Effective Modern C++): "During template type deduction, arguments that are references are treated as non-references, i.e., their reference-ness is ignored." One reader wrote "An expression never has reference type, ... [so this] doesn't make much sense." I think that the idea that a variable of type int& doesn't have a reference type is absurd and is not helpful, especially in understanding how type deduction works.
Of course, a single comment from a single reader means nothing, but I've had this kind of discussion a number of times with a variety of people. Hence this blog post. Until 5/5 gets applied, an expression can have reference type, and that pre-5/5 type corresponds much more closely to intuition than the post-5/5 "adjusted" type.
> I think that the idea that a variable of type int& doesn't have a reference type is absurd and is not helpful
ReplyDeleteWell a variable and an expression that is just a variable's names are distinct and do not necessarily have the same type. But I understand exactly what you mean, and I agree that speaking informally of an expression having a reference type is perfectly reasonable and may be helpful for explication.
In particular I can see why one might prefer such terminology over a more formal usage of the standard's value category terms; it's both more succinct and more accessible, as many programmers aren't familiar with the value categories.
On the other hand, if people keep bringing up this point I have to wonder; are they just engaging in pedantry, or are they actually failing to understand what you mean due to this informal usage? If the former then I think it can safely be dismissed with a "yeah, but you know what I mean," without harming your mission of making people better C++ programmers.
@Seth: Your perspective seems to be entirely pragmatic, which I appreciate. The Standard, on the other hand, engages in semantic gymnastics leading to the odd formality that given a call to a function that returns a reference, the (post-adjustment) type of the call expression is a non-reference, but if you check to see if the result of the call is a reference (via std::is_reference<decltype(the function call)>::value), the result is true: it's a reference. As I said, 5/5 takes the reference away, and decltype adds it back. It'd be an interesting exercise to see how hard it would be to eliminate this now-you-see-it-now-you-don't behavior from the Standard. (One thing that'd have to be done would be to change the specifications for some of the operations in the C subset, e.g., to have pointer dereferences and array index operations return T& instead of simply lvalues of type T.)
ReplyDelete[5.1.1p8]: An identifier is an id-expression provided it has been suitably declared [...] The type of the expression is the type of the identifier.
ReplyDelete[8.3.2p1]: [...] then the type of the identifier of D is “derived-declarator-type-list reference to T.”
I think the paragraphs above settle the matter once and for all, don't they?
Regarding the type of "f()", [5.2.2p3]: [...] the type of the function call expression is the return type of the statically chosen function [...] This return type shall be an object type, a reference type or cv void.
So, the type of "f()" is "int&". The moment it is used, its type is adjusted to "int", because we want to work with the object or function being referenced, not with the reference itself (that's how I think about it).
@bogdan: Thanks for the backup :-) I agree, the sections you site for identifiers as expressions seem to make the case pretty unequivocally.
ReplyDeleteDoes the following count as an expression that returns a reference?
ReplyDeleteDoes the following also count as a function that only accepts an array (not an array that has decayed into a pointer) and returns an array (without having it decay into a pointer)?
#include <iostream>
int ( & add_two( int( & a )[ 3 ] ) )[ 3 ] {
for ( auto & e : a ) {
e += 2;
}
return a;
}
std::ostream & operator << ( std::ostream & o, int const ( & a )[ 3 ] ) {
for ( auto const & e : a ) {
o << e << '\n';
}
return o;
}
int main() {
int vec3[ 3 ]{ 1, 2, 3 };
std::cout << "vec3:\n" << vec3 << '\n';
int ( & result )[ 3 ] = add_two( vec3 );
std::cout << "result:\n" << result << '\n';
std::cout << "vec3:\n" << vec3 << '\n';
std::cout << "add_two( vec3 ):\n" << add_two( vec3 ) << '\n';
std::cout << "vec3:\n" << vec3 << '\n';
//-----------------------------------
// TWO CASES THAT DO NOT WORK:
//-----------------------------------
//-----------------------------------
// CASE 1) Arrays of a different size:
//-----------------------------------
//int vec4[ 4 ]{ 1, 2, 3, 4 };
//add_two( vec4 ); // compile-time error: error C2664 : 'int (&add_two(int (&)[3]))[3]' : cannot convert argument 1 from 'int [4]' to 'int (&)[3]'
//-----------------------------------
// CASE 2) Pointers to arrays of the same size:
//-----------------------------------
//int * vec3_ptr = new int[ 3 ]{ 1, 2, 3 };
//std::cout << "vec3_ptr:\n";
//for ( size_t i = 0; i < 3; ++i ) {
// std::cout << vec3_ptr[ i ] << '\n';
//}
//std::cout << '\n';
//add_two( vec3_ptr ); // compile-time error: error C2664: 'int (&add_two(int (&)[3]))[3]' : cannot convert argument 1 from 'int *' to 'int (&)[3]'
//add_two( *vec3_ptr ); // compile-time error: error C2664: 'int (&add_two(int (&)[3]))[3]' : cannot convert argument 1 from 'int' to 'int (&)[3]'
//delete[] vec3_ptr;
}
/*
Output:
vec3:
1
2
3
result:
3
4
5
vec3:
3
4
5
add_two( vec3 ):
5
6
7
vec3:
5
6
7
*/
From above, the question:
ReplyDeleteDoes the following count as an expression that returns a reference?
should have read:
Does the following count as an expression that has a reference type?
... and I should have followed that up with:
Or are you guys writing about something different?
;-)
The very first quote says that, for the purpose of any analysis whatsoever ("prior to any further analysis"), expressions don't have reference type, because those that start out that way are immediately adjusted.
ReplyDeleteSo while technically you're right, expressions can have reference types, such knowledge does not serve any purpose in understanding the language. The slightly simplified "expressions never have reference type" is not technically true, but more useful than the exact truth for understanding the language.
Your second quote, on the other hand, says nothing to support your point. The "result type" in that quote is not the type of the function call expression, it's the declared result type of the function.
Note that the removal of the reference type from types of expressions leads to an identical treatment of variables of type T and variables of type T& as well as any lvalue expression of type T, not only in the context of deduction. So this might make the spec easier, despite the additional rule in [expr]p5.
ReplyDeleteFor example, some expressions require that one of the operands has an integral type. If expressions observably could have reference type, this had to be adjusted so that both integral types and references to integral types are matched.
I therefore agree with @Seth that a distinction between the type of a variable and the type of an expression referring to that variable makes sense (to me). Also in the light of variables of rvalue-reference type: All expressions that are names, i.e. all id-expressions, are lvalues - with the exception of enumerators. (Class member access expressions can be rvalues, but they're not id-expressions.)
There is of course an asymmetry between variables and return types: For the former, we treat references and non-references equally as lvalues, whereas for return types, the value category is determined by the reference qualifier or lack thereof.
By talking about expressions of reference type, it might be possible to lose sight of this asymmetry.
This asymmetry is also the reason for the different rules of decltype(expression) vs decltype((expression)) - but there's no excuse to call the whole thing decltype if it's only sometimes about the declared type of a variable.
As far as I know, you cannot observe the initial type of an expression prior to applying [expr]p5, so talking about the initial type to me only makes sense if it has explanatory value (and not because there indeed is, in some sense, an expression of reference type).
P.S. I'm resubmitting this comment since I'm unsure if my earlier submission worked.
Since I wrote that mail to Scott, let my clarify some points:
ReplyDeleteIt is true that an expression can have (initially) a reference type. In your example the function call expression f() has type int& (by 5.2.2/3 not 5.2.2/10 which is about the value category of the expression). But as soon as you recognize that the type is a reference, you adjust the type to int.
That's why I tell people that an expression never has reference type. Because it is the adjusted type which matters (in overload resolution, template argument deduction, auto, decltype, ...).
But if you take the standard literally then it is true that an expression can have (initially) reference type, although you cannot observe this initial type.
@dyp:
ReplyDelete> By talking about expressions of reference type, it might be possible to lose sight of this asymmetry.
In my opinion, it's the other way around, and here's why:
As you mentioned, the function call expression 'f()' works very differently depending on whether 'f' is declared as 'int f()' or 'int& f()'. If we refused to talk about expressions of reference type, then the expression 'f()' would have type 'int' in both cases, which I'd say would make it more likely to lose sight of the asymmetry you talked about.
Also, such an interpretation is incorrect from the standard's point of view, as the type of a function call expression is the same as the function's declared result type (standard reference in my previous answer).
So, I think talking about expressions of reference type makes a lot of sense, as it helps us understand why the expression is treated differently later on, even though the reference has been removed 'for further analysis'.
@dyp and @Marcel Wid:
Consider this function definition, where 'B' is a class type.
void f(B b, B& br)
{
b.mf();
br.mf();
}
The 'initial', before adjustment, type of the expression 'b' is 'B', while for 'br', it's 'B&' (standard reference in my previous answer). Those 'initial' types are essential for interpreting the two full expressions, and they're very observable to me - they can change the way the full expressions are evaluated. Saying that 'it is the adjusted type which matters' may actually be confusing.
Sure, we can construct other ways to think about this, but I think the way using the 'initial' reference type has its merits, and, given that the standard explicitly defines expressions of reference type, I think there are reasons to prefer it.
@Marcel Wid: Thanks for pointing out that I should have cited 5.2.2/3 instead of 5.2.2/10 for the call expression. I've updated the post to reflect this.
ReplyDelete@dyp: As I hope I made clear in an earlier comment, my interest in the matter is for explanatory purposes, so perhaps we can agree that there is value in that realm for the idea that expressions can have reference type. Here's what I wrote in the earlier comment:
ReplyDeleteThe situations where I tend to run into trouble with people regarding whether expressions can have reference type are when I make statements such as this (from Effective Modern C++): "During template type deduction, arguments that are references are treated as non-references, i.e., their reference-ness is ignored."
In essence, what I'm doing here is describing the effect of 5/5. It prevents readers from wondering why template type deduction doesn't deduce a reference type when a reference variable is passed, and it also prevents them from wondering why passing a reference variable to a reference parameter doesn't yield a reference to a reference.
As for the standard itself, if you want to truly understand it, you have to recognize that expressions can start with a reference type (e.g., per 5.5.2/3), can have it stripped off (per 5/5), and can have it added back again (e.g., per 7.1.6.2/4 bullet 3).
Overall, I think that the notion that expressions can't have reference type is harmful for general explanatory purposes and is insufficient for purposes of understanding the Standard. As an informal shorthand for the more nuanced Standard-based truth, it's fine among those who already understand the nuances, but the population of people who fall into that category probably numbers in the low dozens.
(I was a bit sloppy in the usage of the term "reference qualifier" in my previous comment. What I meant were the ptr-operators that are added to object types, not the ref-qualifiers added to member function types.)
ReplyDelete@bogdan
Well, the asymmetry is present in the value category of the expression.
In the example, there is no difference in the decomposition or semantics of the expressions b.mf() and br.mf(). The initial types do not seem relevant to me in those examples. Yes, they do refer to objects of different scopes; yes, the types of the function parameter matter for the function call expression f(obj0, obj1). But for the class-member-access expressions b.mf and br.mf, those differences do not matter. In both cases, the object-expression is an lvalue. Overload resolution will select the same member function (declaration) for both function-call expressions. (Polymorphism doesn't change that.)
The semantics of an expression are determined by two independent pieces of information. We can express these pieces for example as:
- The kind (id-expression, function call, ..) of the expression and its initial type.
- The value category and the type after removing references.
We can also express these pieces of information in other ways, much like the basis vectors of a vector space.
One has to admit that there are expressions of reference type in the Standard. But I think the origin of the "there are no expressions of reference type" is a simplification; a model of the specification. As far as I can see, you can form a very precise model of the specification with the simplification of "there are no expressions of reference type": Whenever an expression is analyzed, its type loses its referenceness. This referenceness is so brittle that you cannot even look at it with decltype to observe it; it seems unobservable to me.
If you talk about expressions having reference type in the context of anything but the mere existence of those expressions, then I think that is a model of the actual spec. Such a model can be useful, for example if it is simpler than the actual rules. Typically, we restrict the applicability of such a model to achieve simplification: Newtonian mechanics is a good model as long as the objects have the right size and mass and are not too fast.
@Scott Meyers
ReplyDeleteYou are a far more experienced teacher than I am. I wonder though if it is useful to talk about exceptions of the referenceness of expressions, a la "in this context it does care, in that context it does not care whether or not the expression has reference type".
What I'm wondering is if it is possible and useful to teach another model of expressions to programmers that are not very experienced with Standardese. It seems to me the model you're referring to is akin to "expressions can have reference type, and there are contexts where this referenceness is ignored". The model that's behind "there are no expressions of reference type" to me necessarily includes describing expressions via the reference-less type and its value category. Once there is value category, we don't need the referenceness of an expression any more. In fact, the two are not "linearly dependent", to use the language of the vector basis example from my previous comment. If we do not include value category, we need not only to know the referenceness, but also the kind of the expression (variable, function call, cast?) to determine the semantics.
The reason why I'm wondering is that to me, a model that only involves the two value categories and the reference-less type seems simpler than a model that needs to add exceptional rules and distinguish between variables and return values. (But then again, I'm not a very experienced teacher.)
Neither model is entirely correct, as far as I can see. The former ("there are expressions of reference type, and there are contexts where it doesn't care") needs to introduce rules e.g. in type deduction where there are no such rules in the Standard. The latter ("there are no expressions of reference type") does not acknowledge that there are actually expressions with reference type in the Standard.
Deciding which model is better can of course not be done solely on this point. A model is good if it has a clear application area where it is sufficiently precise; and then it needs to be as simple as possible.
@dyp: A fundamental problem is that variables can be of reference type. There's no way to hide that from people. If x is of type int&, x is a reference. That has implications you can't get away from, e.g., it has to be initialized.
ReplyDeleteBut if you pass x to a template function, what type will be deduced for x? Its type is int&, so what is the obvious conclusion? But the obvious conclusion is wrong. So how do we explain that?
One way is to split hairs and say that x, as a variable, is an int&, but x, as an expression, is an int. But that also requires explaining when something is a variable and when it's an expression, and it's frankly a can of worms. It's like trying to explain how a variable of type int&& is an lvalue, which is something I have extensive experience with, and trust me when I tell you that it's initially very confusing for virtually everybody (including me).
I haven't tried teaching things that way, so I can't compare it to what I do, which is to tell people that for type deduction purposes, an argument's reference-ness is ignored. What I can say is that it's easy to motivate that, because if I have a template that seems to declare a by-value parameter (i.e., that declares a parameter of type T), it'd be counterintuitive if passing a reference argument turned that into a T& parameter.
Of course, matters get even more confusing when universal references enter the picture, because then lvalue arguments are deduced to have a reference type. Which means that, strictly speaking, a variable of type int& is an expression of type int that's deduced to have a type of int&. But my way's no better: the variable has type int&, but its reference-ness is ignored for type deduction purposes, but then a & is slapped back on. There's just no way to make that sequence of events look anything but arbitrary, IMO.
(@Scott Meyers: Apologies for the partial hijacking of the discussion, I'll shut up in a minute, I promise...)
ReplyDelete@dyp:
I have to disagree with your analysis of the 'void f(B b, B& br)' example. I think there is a fundamental difference in semantics between 'b.mf()' and 'br.mf()': if 'mf' is virtual, then, of the two expressions, 'b.mf()' is the only one for which we can tell for sure, at compile time, what function will actually be called (or, to put it differently, expression 'b' is the only one for which the static type and dynamic type of the object referred to will always be the same). It seems to me that the fact that 'br.mf()' can result in a different function being called than the one that would be chosen statically is an essential property of 'br' being a reference type (in this case); it's essential for both the compiler and the human reading the code, and it's very observable.
This property of 'br' is not a consequence of either the value-category or the type-after-removing-references properties in this case. So, in your subsequent description (nice one, by the way), I think you need one more basis vector to truly form a basis in your model; that could be the declared-type, but then that's not linearly independent of type-after-removing-reference. I see that as a problem with the model.
I only intervened in the discussion because I thought the statement 'expressions don't have reference type' was too strong and a bit dangerous, since it runs contrary to explicit statements in the standard. I also think the statement 'the initial reference type of an expression is not observable' is too strong, and one of the reasons is the example above.
In terms of what is the best model for teaching this C++ vector-space-of-properties, I think both alternatives have advantages and disadvantages of their own, and, if I had to teach it... I'd probably choose a third way :-) but it would probably include expressions of reference type.
How do you teach the following?
ReplyDeleteint&& f();
void g(int&& x)
{
int&& y = f();
int&& z = x; //ERROR!
}
Both expressions, x and f(), have (initial) type int&&. In my model, where an expression has 2 invariants, namely its (adjusted) type and its value category, I can easily explain the bahavior: The expression f() hat type int and is an xvalue (or more general an rvalue), while the expression x also has type int but is an lvalue. Again, I claim that it's the adjusted type AND the value category of an expression that matter.
Of course, there is no single best model to teach these topics, but I want to motivate, why the model "expressions never have reference type" is IMO a good starting point.
"Overall, I think that the notion that expressions can't have reference type is harmful for general explanatory purposes and is insufficient for purposes of understanding the Standard. As an informal shorthand for the more nuanced Standard-based truth, it's fine among those who already understand the nuances, but the population of people who fall into that category probably numbers in the low dozens."
ReplyDeleteAs I said earlier, it may be better for pedagogical reasons to speak of expressions having reference types, but I think it's absolutely incorrect to say that understanding expressions as having non-reference types (and having value categories) is "insufficient for purposes of understanding the Standard."
That is, although the standard refers to expressions as having intermediate reference types, that language could easily be replaced, and whenever any part of the standard refers to an expression's type, template deduction rules, decltype, etc., it is always talking about the expression's final type. Nor is understanding that it is the final type which is formally significant an 'informal shorthand'.
To the contrary, it is an 'informal shorthand' to skip the value category conversions and act as though these expressions have reference types in these contexts.
---
The reference type was added to C++ as the programmer's handle for controlling expression value categories from C, in support of C++'s goal of giving programmers power similar to the built-in constructs.
It's clear that what C++ tries to do is keep reference types and value categories consistent to this end. Since accessing a 'referred-to object' requires having a glvalue, reference types get converted to glvalues for expressions. And since types don't have value categories, getting a type from an expression (e.g., decltype) means transforming the expression's value category into a reference qualifier on the resulting type.
In particular I wouldn't say that [expr]/5 is what motivates the decltype rules for producing reference types. Instead both rules, along with several others, all share the same motivation; to make reference qualifiers on types synonymous with value categories on expressions.
As such, eliminating the back and forth between the two would, I think, mean completely eliminating value categories from the standard and replacing all that language with language to deal with expressions of reference types.
---
Here's another argument: it is too much of a stretch to argue that "the expression's type" should be understood to refer to an expression's 'inital' type rather than it's final type. "An expression's type" is it's final, adjusted type, and is therefore never a reference type.
This is why I say that the according to the standard, the type of `f()` is formally `int`, with an lvalue value category.
@Seth:
ReplyDelete[...]although the standard refers to expressions as having intermediate reference types, that language could easily be replaced, and whenever any part of the standard refers to an expression's type, template deduction rules, decltype, etc., it is always talking about the expression's final type.
Understanding the Standard requires dealing with it as it's written, not as it could be written, and 5.2.2/3 is an example of where the Standard refers to an expression's type as potentially being a reference, i.e., where, when the Standard talks about an expression's type, it's not talking about its final/adjusted type.
There is no way to reconcile the idea that expressions never have reference type with this part of the Standard, and that's why I say that the shorthand "expressions never have reference type" is insufficient for understanding the Standard.
@Marcel Wid: I'd explain the behavior of your function g as follows:
ReplyDelete1. As function return types, rrefs are rvalues.
2. The initalization of y is fine, because rrefs can be initialized with rvalues.
3. All parameters are lvalues, regardless of type, so x is an lvalue.
3. The initialization of z is illegal, because you can't initialize an rref with an lvalue.
Scott
To clarify my earlier statement: there is no part of the standard in which an expression's type is taken to mean anything other than its 'final' type, excepting by necessity only the part that specifies the algorithm for determining that final type. For example, the specifications for template type deduction and decltype specifiers will refer to the types of expressions but never mean their initial types.
ReplyDelete5.2.2/3 ([expr.call]/3) describes a step in the algorithm for determining an expression's type.
"and that's why I say that the shorthand "expressions never have reference type" is insufficient for understanding the Standard."
If you're only talking about specification for determining an expression's type, okay. But understanding, say, the template type deduction does not require ever knowing about an expression's 'initial' type. An 'initial' type is never needed for understanding any other part of the standard.
@Scott Meyers
ReplyDeleteI do acknowledge that there variables of reference type necessitate to deal with reference type. In an earlier comment, I've alluded to the asymmetry of variables of reference type versus function calls whose initial type is a reference.
You say that it would be splitting hairs to differentiate between some x as a variable and the same x as an expression. I think a programmer already needs to understand that, since initializing a reference is fundamentally different from using it after the initialization; for example:
int y = 42;
int x = y; // not an assignment, binding to a reference
x = y; // an assignment
y = x; // an expression with the same behaviour as the above
This is not just about the sign = in two contexts. In the initialization of a reference, we deal with the reference itself and need semantics that are different from the semantics of initializing the type referred to. When later dealing with the variable of reference type, it is as if we were dealing with the object referred to.
I know only two occasions when we deal with variables themselves, instead of expressions referring to those variables: Initialization and decltype. The latter does two things, one of which is to yield the declared type of a variable.
Both occasions need special treatment anyway. Therefore, I fail to see the can of worms you're mentioning. Would you mind giving an example?
For me, a lot of rules just fell into place after realizing that effectively, expressions don't have reference type (that is, in every context where we analyze expressions or where the type of an expressions is required for the semantics of some super-expression, they don't have reference type).
One example is the lvalueness of an expression that refers to a variable of rvalue-reference type. As you probably know, those kind of expressions originally were specified as rvalues, before it was realized that this leads to problems. As I see those problems, they're all related to the handle one gives to the rvalue by binding it to a name: with a name, I can refer to the same object multiple times, and the name can refer to some subobject of the actual (complete) object. Though I don't have a concise rule of what an lvalue is, being a name is a property that is sufficient to being an lvalue because of the identity that a name gives to an object. (Enumerators are names but rvalues, but they could be seen as a form of literals, that is, expressions referring to a value directly. Also, they're not variables.)
And this is also how the Standard specifies that (id-)expressions referring to variables of rvalue reference type are lvalue-expressions: There simply is no dedicated rule, the rule for id-expressions indiscriminately applies to all variables.
For forwarding references, the spec has been deliberately broken as far as I understand it: an inconsistency has been introduced to solve the forwarding problem. Since the inconsistency is in the Standard, it might not be reconcilable with the remaining rules even in a simplified model.
P.S. Resubmitting a comment again. There seems to be an issue? Sometimes, I'm not getting a confirmation.
@dyp: I'm sorry you're having trouble posting. As far as I know, that's a Blogger issue, and there's nothing I can do to resolve the problem. If you (or anybody else) is aware of something I can do to improve the situation (short of moving my blog to a different platform), please let me know.
ReplyDeleteThe distinction between variables and expressions that you draw is actually the distinction between declarations and expressions. I agree that that's something developers already have to understand, but even that has unfortunate consequences. C programmers moving to C++ tend to have great difficulty in understanding that in a declaration, "&" means reference, but in an expression, "&" means address-of. Except, of course, in the type specification part of a cast expression, where it can mean reference again. Another can of worms.
Given a variable x, you seem to want to say it's a variable in declarations, but an expression everywhere else, except when passed by itself to decltype, when it's a variable again. That explanation works, I guess, but note that formally (per 7.1.6.2/4 as well as the grammar summary at the beginning of 7.1.6.2), decltype takes only expressions. Which means that in "decltype(x)", x is an expression that's treated as a variable. I thus disagree with your assertion that "in every context where we analyze expressions or where the type of an expressions is required for the semantics of some super-expression, they don't have reference type." In some (but not all) expressions used as arguments to decltype, they have reference type. It's this kind of special-case behavior that I consider a can of worms.
As an aside regarding your view of names and lvalues, note that "this" is effectively a variable name that's not an lvalue. (In principle, the expression "&this" makes perfect sense and has obvious semantics, but it's not permitted, because "this" is defined to be an rvalue.)
@bogdan
ReplyDelete(I have not seen exceptions to this rule, so I'll state it as an absolute)
The Standard does not discriminate in the (immediate) context of any other expressions between expressions of lvalue-reference type and variables that are lvalues. This of course is also the case for calls of member functions as in `b.mf()` vs `br.mf()`. Both cases follow the same rule in the specification. (So again, the spec does not use the initial type of an expression to define any semantics but the value category of said expression. That's what I mean with the initial type of the expression is not observable.)
Which function is actually called depends on another property that you'd have to add in both simplified models: the dynamic type of the object expression (the LHS of `.`) The fact that for `br.mf()` the function being called is not known from the immediate context is not unique to expressions of reference type. And similarly, the function that `br.mf` refers to could have been declared as `final` in some base class, so we could restore knowledge of which function is being called at compiler time. In a simplified model that has no expressions of reference type, one could say that a dynamic dispatch is always performed, and a compiler is free to perform static analysis to avoid the dynamic dispatch under the as-if rule.
If you insist that the property of the static type of an expression can differ from the dynamic type of the expression is important, then I'd like to see in which parts of the Standard this property is actually used. I would guess that we can omit this property from the model "expressions don't have reference type" and lose only a small part of its area of applicability (if any at all), while keeping it simple at its valid area of applicability.
As Scott Meyers mentioned earlier: dereferencing a pointer does not yield an expression of reference type currently in the Standard. So when you claim that there's a fundamental difference between `b.mf` and `br.mf`, you'd have to introduce a rule that either covers `br.mf` and some `p->mf`, or a separate rule for both, or make `*p` yield an expression of reference type. I think that is more complicated than just to say that the abstract machine always performs a dynamic dispatch (for virtual functions).
@dyp:
ReplyDelete(This turned into a novel, sorry... But, hey, you started it :-) )
On discriminating between 'b.mf()' and 'br.mf()':
Yes, both cases follow the same rule in the standard, but I think there's more to that rule than it seems at first glance.
Here's how the standard defines virtual function calls:
[5.2.2p1]: [...] Otherwise, its final overrider (10.3) in the dynamic type of the object expression is called; such a call is referred to as a virtual function call. [...]
There's no mention of any special dispatch mechanism that has to be implemented in all cases and maybe optimized away sometimes, or of whether the dynamic type of the object is known or not. And, (paradoxically, you may think) that's why knowing the dynamic type is important: if the dynamic type is statically known, then the final overrider is statically known too, and it just needs to be called; that's all that the standard requires.
And that's what happens in practice, as I'm sure you know. On any compiler that I know of, with or without optimizations enabled, the call to 'b.mf()' will not involve anything else than a simple, direct function call, because the rule in [3.8p7.2] ensures that there will never be an object of a different dynamic type in there (or else we'd have undefined behaviour). This is independent of any optimization under the 'as-if' rule; this is simply doing what the standard says, without introducing anything that would have you 'pay for what you don't use'.
On the other hand, if the dynamic type is not statically known, then the only other way to identify what function to call is at runtime, and so the implementation is obviously required to provide a mechanism for the final overrider to be identified and called at runtime, which makes 'br.mf()' essentially and observably different from 'b.mf()' in my example.
Expecting any kind of runtime polymorphism from 'b.mf()' makes no sense, while 'br.mf()' is one of the ways to achieve it. A model that tries to help people understand how things work in C++ should make this very clear, and I don't see how it could do that if it ignores the only difference between the two expressions, that is, references.
Regarding pointers:
The rule for expressions of pointer type is just as simple as for reference types: 'p->mf()' is the same as '(*p).mf()' and the fact that the object expression is obtained by dereferencing a pointer is essential and must not be ignored.
Such an expression is essentially different from a 'b.mf()' from my example. I don't see how trying to hide that can help anyone understand what happens in a C++ program.
This also nicely reflects the fact that one can think of references as entities that have some of the properties of pointers and some of the properties of names.
@dyp:
ReplyDeleteA few more bits and pieces:
> Which function is actually called depends on another property that you'd have to add in both simplified models: the dynamic type of the object expression
Yes, of course that property is needed, but I said something else too: in that example, the fact that 'b' is not a reference conveys essential information about the dynamic type of the object, namely that it will always be the same as the static type, and that in itself is also a property of the expression.
> The fact that for `br.mf()` the function being called is not known from the immediate context is not unique to expressions of reference type.
I never said it were. I specifically said that in that case, of the two expressions, the reference one has that property and the other one doesn't.
Yes, there are other expressions with this property, and all those expressions involve pointers or references in one way or another. Just like I can't ignore a pointer dereference, I can't ignore a reference. They all have an essential common trait; I guess I would call it 'the pointer-like trait'.
> And similarly, the function that `br.mf` refers to could have been declared as `final` in some base class, so we could restore knowledge of which function is being called at compiler time.
That would amount to 'adjusting the experiment so that it conforms to the model', and that's not how good models are validated.
> In a simplified model that has no expressions of reference type, one could say that a dynamic dispatch is always performed, and a compiler is free to perform static analysis to avoid the dynamic dispatch under the as-if rule.
Yes, one could say that. And the programmer reading the code will have to do that static analysis as well. And what will be the first thing the programmer or the compiler will look at when statically analysing that example? Whether the object expression actually is of a reference type or not. Removing this from the model when you're clearly going to resort to it when needed just seems counterproductive to me.
@dyp:
ReplyDelete(This post was supposed to appear before the other one, but it looks like it was overwritten instead.)
(This turned into a novel, sorry... But, hey, you started it :-) )
On discriminating between 'b.mf()' and 'br.mf()':
Yes, both cases follow the same rule in the standard, but I think there's more to that rule than it seems at first glance.
Here's how the standard defines virtual function calls:
[5.2.2p1]: [...] Otherwise, its final overrider (10.3) in the dynamic type of the object expression is called; such a call is referred to as a virtual function call. [...]
There's no mention of any special dispatch mechanism that has to be implemented in all cases and maybe optimized away sometimes, or of whether the dynamic type of the object is known or not. And, (paradoxically, you may think) that's why knowing the dynamic type is important: if the dynamic type is statically known, then the final overrider is statically known too, and it just needs to be called; that's all that the standard requires.
And that's what happens in practice, as I'm sure you know. On any compiler that I know of, with or without optimizations enabled, the call to 'b.mf()' will not involve anything else than a simple, direct function call, because the rule in [3.8p7.2] ensures that there will never be an object of a different dynamic type in there (or else we'd have undefined behaviour). This is independent of any optimization under the 'as-if' rule; this is simply doing what the standard says, without introducing anything that would have you 'pay for what you don't use'.
On the other hand, if the dynamic type is not statically known, then the only other way to identify what function to call is at runtime, and so the implementation is obviously required to provide a mechanism for the final overrider to be identified and called at runtime, which makes 'br.mf()' essentially and observably different from 'b.mf()' in my example.
Expecting any kind of runtime polymorphism from 'b.mf()' makes no sense, while 'br.mf()' is one of the ways to achieve it. A model that tries to help people understand how things work in C++ should make this very clear, and I don't see how it could do that if it ignores the only difference between the two expressions, that is, references.
Regarding pointers:
The rule for expressions of pointer type is just as simple as for reference types: 'p->mf()' is the same as '(*p).mf()' and the fact that the object expression is obtained by dereferencing a pointer is essential and must not be ignored.
Such an expression is essentially different from a 'b.mf()' from my example. I don't see how trying to hide that can help anyone understand what happens in a C++ program.
This also nicely reflects the fact that one can think of references as entities that have some of the properties of pointers and some of the properties of names.
@Scott Meyers
ReplyDeleteThe posting issue might be related to the "preview" feature. When I try to preview my comment and then use the "Publish" button in the preview window, it seems to fail.
First of all, since I'm tired of using a sentence two refer to either model, I'll introduce some bad names:
"there are NO Reference-Type EXpressions" -> Nortex
"EXpressions Can hAve Reference Type" -> Excart
This might not be very relevant, but I purposely made a distinction between initialization and usage, not between declarations and expressions: Mem-initializers in the mem-initializer-list of a constructor are not declarations, but have the same reference-binding semantics that appear in the definitions of variables of reference type.
"C programmers moving to C++ tend to have great difficulty in understanding that in a declaration, "&" means reference, but in an expression, "&" means address-of." Interesting! Though I can understand why this is not obvious. I'd probably try to differentiate between the postfix type-operator "&" and the prefix expression-operator "&". But I guess I'm digressing.
The reason why I mentioned decltype specifically as a kind of exception to the "the referenceness of an expression is ignored" is because I think the decltype language feature is fundamentally broken: it yields two related, but not identical pieces of information about its operand - depending on what form the operand has. If the operand is a variable or function, it tells us the declared type of that entity. Otherwise, it tells us the type of the operand expression plus its value category. This leads to weird inconsistencies you'll probably know:
int x;
decltype(x) // int
decltype((x)) // int&
int&& y = move(x);
decltype((y)) // int&
as @Seth already said, decltype does not simply restore the referenceness of an expression.
[to be continued... sorry for the long comment]
@Scott Meyers
ReplyDelete[...continued]
Therefore, I think decltype is in itself a can of worms. If you know a simple explanation of the weird behaviour of decltype in the Excart model, please let me know. Assuming we don't know any such simple explanation, I don't think it's a good example of a difficult-to-explain special case in the Nortex model.
I can only agree that decltype does take only expressions. But I don't quite know what you want to say with "x is an expression that's treated as a variable". If we go the language-lawyer way and follow the strict wording of the spec, if the expression is an id-expression or class-member-access, the expressions is only used to tell us what entity it is referring to. The type of that entity is used as the resulting type; the type of the expression is not itself used in that case. I would also guess that decltype takes an expression because that's the easiest way to specify it grammatically; for example, A::x to "name" some variable in a base class or namespace necessarily is an expression, not a an identifier.
I don't like playing language-lawyer here, since I think the whole discussion is only useful if we discuss the Nortex and Excart models. The Standard itself, as it is written, cannot be precisely described by Nortex, since - as you have pointed out - it is incorrect as a claim. In the Nortex model, one could easily explain decltype in a way very similar to the Standard: if the expression names a variable or function directly, the resulting type is the declared type of that variable or function. Otherwise, the resulting type is the type of the expression, plus an & if it is an lvalue, or an && if it is an xvalue.
On the side-topic of "this": That is an interesting remark I'll have to think more about. Currently, I'd say that it's more like a literal, simply because it fundamentally is more like a value, not an object. I might describe it as a value whose meaning is "the address of the current class instance (subject)". Also, I'm stuck at the "obvious semantics" that an expression "&this" should have. I don't really understand what you're alluding to.
@dyp: Since the original point of my post was to establish that expressions can have reference type, and you seem to agree that that's true for the Standard as it's written (which is the only Standard we have), I'll put you down as agreeing with me, take my winnings, and go home :-)
ReplyDeleteRegarding "this", the only reason it seems more like a literal than an object is because it's defined to be an rvalue. If, instead, it were defined to be an lvalue of type T* const (for a class T) or, for const objects, const T * const, you be able to treat it like any other const lvalue, and the type yielded by "&this" would be T * const * (or, for const objects, const T * const *).
When teaching C++ we should describe things "as is", i. e. as they are in standard. Or, we should give a warning like "currently we are not strict". So, if I wrote a C++ book, I would give a warning that I'm not strict at the beginning, then give some informal text about references and types, but then proceed to actual terminology, i. e. every expression has value category and (final) type, which is always non-reference.
ReplyDeleteWe should have some convention about what "type of expression" is. Can it be reference or no. Everywhere the standard refers to "type of expression" it assumes that it cannot be reference (including 4.1/1 [lvalue-to-rvalue conversion] and 7.1.6.2/4 [decltype definition]), with the exception for 5/5 and some places where we construct type of expression, for example mentioned 5.2.2/3. So, (final) "type of expression" never can be reference (but intermediate type can be reference).
Let's assume we declare "int a". What is type of expression "a" now? Of course, it is "int", even if we will consider pre-5/5 type. We can say that it is "int &", but this would be our local convention only, which will contradict the standard. Okey, so "a" is int. It is lvalue of type "int". Now we declare "int &b = a". What is type of expression "b"? This expression is indistinguishable from "a". I. e. they both are lvalues, they both can be assigned to. If we have the following two functions:
void f (const int &);
void f (int &);
then "f (a)" and "f (b)" will both call the second variant. Only way to distinguish "a" and "b" is decltype: decltype (a) and decltype (b) are different things (but in this case decltype gives us type of variable and not type of expression). So, this is pedagogically right way to think that "a" and "b" have the same type. Okey, so expression "b" has no reference type, and so (by analogue) any other expression has no reference type.
>pre-5/5 type corresponds much more closely to intuition than the post-5/5 "adjusted" type.
In this example, "a" and "b" are nearly the same and so should have the same type. But pre-5/5 types are different. So, post-5/5 is closer to intuition (at least to intuition of experienced programmer).
Now about that br.mf example. b and br declared differently, so they are initialized differently. b initialized as a copy of actual argument and br initialized as a reference to actual argument. But after this initialization expressions b and br have the same type: they both are lvalues of type B.
@bogdan {
Now about that argument about virtual functions in br.mf example. As well as I know, the standard doesn't describe where virtual function table (vft) look up can be omitted. So, in b.mf () vft look up typically omitted, but this is implementation detail, this is just optimization. So, b and bf still are lvalues of type B.
Well, I can agree that this b.mf example shows that referenceness sometimes matter. But "expression cannot have reference type" is too good convention.
}
@Unknown:
ReplyDelete> We should have some convention about what "type of expression" is. [...]
Agreed. How about this convention: The type of an expression is what the standard says it is. This is what I use; it has the advantage that it doesn't need any disclaimers.
> [...] Now we declare "int &b = a". What is type of expression "b"? [...]
Please read my first post in this thread, where I indicate the sections in the standard that clearly and unambiguously specify that the type of the expression "b" is "int&". Not the "intermediate" type; this is the type. Saying that expression "b" has type "int" is simply incorrect. I think it's important to restate this, as you're making this wrong assertion several times in your post.
The significance of reference types in expressions and the usefulness of building new conventions, different from the ones in the standard, about the types of expressions are subjects that have been discussed at length in this thread. Valuable arguments have been brought up on both sides, and I don't think it's worth repeating the same ideas.
The same goes for "br.mf()", whether direct calls are just optimizations or not, and so on. I've made my point on that in a previous post.
(I'm that "Unknown")
ReplyDelete@bogdan
> The type of an expression is what the standard says it is.
Unfortunately, the standard doesn't say what is "the type of expression". The standard has pre-5/5 type and post-5/5. And, according to the standard, both are "type of expression". So, we should have some agreement.
> I indicate the sections in the standard that clearly and unambiguously specify that the type of the expression "b" is "int&"
Okey, but 4.1/1 [lvalue-to-rvalue conversion] and 7.1.6.2/4 [decltype] use term "type of expression" and assume it is never reference. So, it is not clear from the standard that type of expression "b" is "int &". (And yes, this is inconsistency in the standard. Ideally, the standard should be fixed to say what "type of expression" is.)
@Askar Safin:
ReplyDelete> Unfortunately, the standard doesn't say what is "the type of expression". The standard has pre-5/5 type and post-5/5. And, according to the standard, both are "type of expression". [...]
I disagree. I believe the standard does say what the type of an expression is: it's what it says it is in the section that defines that expression. Everything else are transformations applied to the expression as part of the "analysis" that [5p5] refers to.
If you want to think of expressions as having multiple types, it's your choice, if you think that makes things easier to understand, but I don't think that's what the standard says.
Yes, there are inconsistencies in the standard's wording on this subject, but I tend to see them as unimportant, because all of them (that I know of) can be solved by the classic "yes, but you know what I mean". What's important here is to understand the semantics of references (part name, part pointer, as I like to say), and once that's clear, everything else falls into place, the wording being less important. There are other places where the standard is ambiguous or inconsistent and clarification is more difficult or impossible, and I'd prefer to see the committee focus on those.
@bogdan
ReplyDelete> can be solved by the classic "yes, but you know what I mean"
I understand the standard so: type of expression is non-reference type (because we use this in 4.1/1) and all other parts of the standard which contradict this, can be solved using "you know what I mean" :)
@Askar Safin:
ReplyDeleteI'm afraid you're looking for inconsistencies in the wrong place. There's no inconsistency between [4.1p1] and [7.1.6.2p4], on the one hand, and expressions having reference type on the other hand. The expressions that those paragraphs are referring to are undergoing "analysis"; they have already been transformed so that reference types have been replaced by the referred types. There is no contradiction here.
When I agreed that there are inconsistencies, I was referring to cases like:
-on the one hand, [5.2.9] and [5.2.10], which define the results of some casts to reference types as having non-reference type;
-on the other hand, [5.4p1], which says "The result of the expression (T) cast-expression is of type T [...]", which means that, if T is U&, the result of the cast has reference type, and then goes on to say that this cast is actually defined in terms of the other casts.