How does `yield` and `await` production parameters expand into `*` and `async` before identifiers

Question

Can anyone explain how the following production expands into * and async keywords:

BindingIdentifier[Yield, Await]:
   Identifier
   [~Yield]yield
   [~Await]await

Based on the TypeScript parsing code I see that it checks for * and async keywords, so I assume here that BindingIdentifier_Yield matches *identifier and BindingIdentifier_Await matches async identifier but I can't seem to trace that expansion using the above grammar.

I know I can expand the identifiers [Yield, Await] according to the spec:

A production may be parameterized by a subscripted annotation of the form “[parameters]”... A parameterized production is shorthand for a set of productions defining all combinations of the parameter names, preceded by an underscore, appended to the parameterized nonterminal symbol

So the above is expanded into:

BindingIdentifier:
   Identifier
   [~Yield]yield
   [~Await]await

BindingIdentifier_Yield:
   Identifier
   [~Yield]yield
   [~Await]await

BindingIdentifier_Await:
   Identifier
   [~Yield]yield
   [~Await]await

But how then BindingIdentifier_Yield and BindingIdentifier_Await is expanded into * and async? I suspect that the explanation is here:

[~Yield]yield
[~Await]await

but I'm not sure. Any help is welcome!

It doesn't. I think you may be misunderstanding in this case. The only logic happening in here is saying that those two words are only allowed as identifiers outside of generators or async functions. Like `function* foo(){ var yield; }` and `async function foo(){ var await; }` are not allowed since those are not allowed as `BindingIdentifier`s. — loganfsmyth, Sep 05 '17 at 15:48
Are you sure the parsing code you're talking about applies to this grammar? It sounds more like you are looking at code that parses the beginning of a function in general, rather than a BindingIdentifier. — loganfsmyth, Sep 05 '17 at 15:51
@loganfsmyth, thanks! yes, indeed I'm looking at the [functionexpression parsing code](https://github.com/Microsoft/TypeScript/blob/master/src/compiler/parser.ts#L4520) and `parseOptionalToken` looks for `AsteriskToken` so I decided to look that grammar of identifier. [Here is](https://www.ecma-international.org/ecma-262/8.0/index.html#prod-FunctionExpression) the production for fuction expression `functionBindingIdentifier[~Yield, ~Await]`. If you follow `BindingIdentifier` grammar it wil navigate to the grammar I posted — Max Koretskyi, Sep 05 '17 at 15:54
How is this not a duplicate of [What are \[Yield, Await, In, Return\] in EcmaScript grammar](https://stackoverflow.com/questions/46022919/what-are-yield-await-in-return-in-ecmascript-grammar)?! — Bergi, Sep 05 '17 at 16:53
@Bergi, this question is about particular application of the knowledge I learnt from the linked question. It's as if you have a question `what is promise` and then another `why promise here doesn't return correct result`. I see them as different questions — Max Koretskyi, Sep 05 '17 at 19:43

loganfsmyth · Accepted Answer · 2017-09-06T17:12:13.147

1

The check for the * token in Typescript exists to handle both

14.1:

FunctionExpression[Yield, Await, Default]:
    function BindingIdentifier[?Yield, ?Await]opt ( FormalParameters[~Yield, ~Await] ) { FunctionBody[~Yield, ~Await] }

vs

14.4:

GeneratorExpression[Yield, Await, Default]:
    function* BindingIdentifier[?Yield, ?Await]opt ( FormalParameters[+Yield, ~Await] ) { GeneratorBody }

since both function and generator expression can be expanded from PrimaryExpression:

PrimaryExpression:
    this
    Literal
    ArrayLiteral
    ObjectLiteral
    FunctionExpression       <---------
    ClassExpression 
    GeneratorExpression      <---------
    AsyncFunctionExpression
    RegularExpressionLiteral
    TemplateLiteral

These checks also overlap for function declarations, and method syntax.

The production

BindingIdentifier [Yield, Await]:
    Identifier
    [~Yield] yield
    [~Await] await

expands to

BindingIdentifier:
    Identifier
    yield
    await

BindingIdentifier_Yield:
    Identifier
    await

BindingIdentifier_Await:
    Identifier
    yield

BindingIdentifier_Yield_Await:
    Identifier

So yield and await identifiers are not allowed in cases where +Yield and/or +Await has been used in the grammar. You can see in the examples above, they use

FormalParameters[~Yield, ~Await]
FunctionBody[~Yield, ~Await]

whereas the generator uses

FormalParameters[+Yield, ~Await]
FunctionBody[+Yield, ~Await]

Since the generator says +Yield instead of ~Yield, it means that

function foo(){ var yield; } // works
function* foo(){ var yield; } // not allowed

edited Sep 06 '17 at 17:12

answered Sep 05 '17 at 15:59

loganfsmyth

156,129
30
331
251

thanks a lot for your help! can you please show also in your answer that both `FunctionExpression` and `GeneratorExpression` can be expanded from [PrimaryExpression](https://www.ecma-international.org/ecma-262/8.0/index.html#prod-PrimaryExpression) – Max Koretskyi Sep 05 '17 at 16:08
1

Added a bit more to the answer FYI. – loganfsmyth Sep 05 '17 at 16:08
amazing explanation, very much appreciate! – Max Koretskyi Sep 05 '17 at 16:10
are you by any chance available for mentoring sessions on codementor.io? – Max Koretskyi Sep 05 '17 at 19:44
I do, with the same username. I've never actually done it though! Happy to give it a shot. – loganfsmyth Sep 05 '17 at 20:37
great, thanks, will look you up there. how did you learn ecmascript grammar? are you also good with algorithms? – Max Koretskyi Sep 06 '17 at 09:01
Yeah I should be able to answer a good portion of the questions you'll have about the spec. I learned by basically reading the spec a lot over time. – loganfsmyth Sep 06 '17 at 16:15
In the production for `BindingIdentifier_Yield`, the second rhs should be `await`, not `yield`. – Michael Dyck Sep 06 '17 at 16:59
@MichaelDyck Good catch, fixed. – loganfsmyth Sep 06 '17 at 17:12

How does `yield` and `await` production parameters expand into `*` and `async` before identifiers

1 Answers1