4

Why is f'{'one'}' invalid syntax? Seems to me like if f-strings were sensible, as soon as a { appears, everything inside of { ... } should be considered Python code.

What's the reason the language designers decided to make the string's end quote be considered as a string end, even inside of { ... } ?

What advantages come from this?

theonlygusti
  • 11,032
  • 11
  • 64
  • 119

1 Answers1

6

Due to Python’s string tokenizing rules, the f-string f'abc {a['x']} def' is invalid. The tokenizer parses this as 3 tokens: f'abc {a[', x, and ']} def'. Just like regular strings, this cannot be fixed by using raw strings.

https://peps.python.org/pep-0498/#escape-sequences

Why didn't they "fix" the tokenizer to make it possible? Possibly backwards compatibility. Possibly it just isn't worth it when an easy fix is to simply mix " inside ' for the common cases.

The reason the syntax acts like it does is that languages are built upon blocks (tokens) defined by a grammar that shows how tokens are built. Python's f-strings reused two existing building blocks. First, everything between the f' and trailing ' are valid string contents. If you remove the f, your Python program's syntax will still be correct (though the contents of the string will not be dynamic). Your example, '{'one'}' fails this condition. The second condition is that the code between { and } is a valid Python expression. 'one' is a valid, expression, but it must pass both conditions. The next alternative \'one\' is a valid string literal but it's an invalid expression (cannot paste \'one\' in the REPL, for example) - so it doesn't pass both conditions, either. Changing this would make f-strings take a different path through the parser and probably require duplicate code where it wasn't needed before. More detail on tokenization, grammars, and compiler design is beyond the scope of this short answer.

nemec
  • 1,044
  • 9
  • 35
  • 1
    Was just about to post this. In general, questions of the form "why did the Python developers do X" can be answered in more detail than you would ever want by a quick lookup of the relevant PEP(s). :) – Samwise Mar 21 '22 at 23:45
  • @Samwise okay but there is no detail here – theonlygusti Mar 22 '22 at 00:17
  • @theonlygusti I added some details to the answer about how Python's f-strings are tokenized in practice, but you might need to study about compiler design if you want to know why these restrictions happen at the Python interpreter level – nemec Mar 22 '22 at 01:01
  • 1
    Your answer is good and detailed enough (thank you), but the PEP about it (if this is it) isn't really – theonlygusti Mar 22 '22 at 01:03
  • @theonlygusti did you read the part about tokenizing, second paragraph here? https://peps.python.org/pep-0498/#specification – Samwise Mar 22 '22 at 14:09
  • @Samwise it doesn't explain why, just gives the same information as in the question (plus some other restrictions this question isn't asking about) – theonlygusti Mar 22 '22 at 15:09
  • 1
    **When tokenizing source files, f-strings use the same rules as normal strings, raw strings, binary strings, and triple quoted strings. That is, the string must end with the same character that it started with: if it starts with a single quote it must end with a single quote, etc. This implies that any code that currently scans Python code looking for strings should be trivially modifiable to recognize f-strings (parsing within an f-string is another matter, of course).** seems pretty clear to me. ¯\_(ツ)_/¯ – Samwise Mar 22 '22 at 15:13
  • @Samwise you think the main motivation behind this design decision was so that code which scans Python source to look for strings can be written as easily as possible? – theonlygusti Mar 22 '22 at 20:00
  • @theonlygusti, ...if you're going to disregard documentary evidence, all that's left is opinion. Opinions are off-topic here. (But I'd argue so is documentary evidence as well, when it doesn't answer a question about how one goes about the practice of _writing software in_ Python; also see [What is the rationale for closing "why" questions on a language design?](https://meta.stackexchange.com/questions/170394/what-is-the-rationale-for-closing-why-questions-on-a-language-design)). – Charles Duffy Apr 19 '23 at 18:48