On Thu, 2015-12-03 at 15:33 -0500, Jason Merrill wrote:
> On 12/03/2015 09:55 AM, David Malcolm wrote:
> > This patch adds bulletproofing to detect purged tokens, and avoid using
> > them.
> >
> > Alternatively, is it OK to access purged tokens for this kind of thing?
> > If so, would it make more sense to instead leave their locations untouched
> > when purging them?
>
> I think cp_lexer_previous_token should skip past purged tokens.
Sorry if this is a silly question, but should I limit the iteration e.g.
by detecting a sentinel value? e.g.
parser->lexer->buffer->address () ?
Or is there guaranteed to be an unpurged token somewhere beforehand?
Out of interest, the prior tokens here are:
(gdb) p end_tok[0]
$25 = {type = CPP_GREATER, keyword = RID_MAX, flags = 0 '\000',
pragma_kind = PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-1]
$26 = {type = CPP_NAME, keyword = RID_MAX, flags = 0 '\000', pragma_kind
= PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-2]
$27 = {type = CPP_LESS, keyword = RID_MAX, flags = 0 '\000', pragma_kind
= PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-3]
$28 = {type = 86, keyword = RID_MAX, flags = 1 '\001', pragma_kind =
PRAGMA_NONE, implicit_extern_c = 0, error_reported = 0,
purged_p = 0, location = 202016, u = {tree_check_value =
0x7ffff19dfd98, value = <bdver1-direct,bdver1-agu 0x7ffff19dfd98>}}
(gdb) p end_tok[-4]
$29 = {type = CPP_KEYWORD, keyword = RID_NEW, flags = 1 '\001',
pragma_kind = PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 0, location = 201890, u =
{tree_check_value = 0x7ffff18a8318,
value = <identifier_node 0x7ffff18a8318 new>}}
where the previous unpurged token is:
(gdb) p end_tok[-3].purged_p
$31 = 0
(gdb) call inform (end_tok[-3].location, "")
../../src/gcc/testsuite/g++.dg/cpp0x/nsdmi-template14.C:11:14: note:
B* p = new B<N>;
^
which would give a range of:
B* p = new B<N>;
^~~~~
for the erroneous new expression, rather than:
B* p = new B<N>;
^~~~~~~~
if we used the location of the purged token (the CPP_GREATER).
I prefer the latter, hence my suggestion about not zero-ing out the
locations of tokens when purging them.
