Repository navigation
python -m pegen python is broken #96424
Description
If I run PYTHONPATH=Tools/peg_generator ./python.exe -m pegen python Grammar/python.gram on main branch, the result file is broken.
There are lots of invalid syntax construct that come from C directly.
Some examples:
@memoize
def interactive(self) -> Optional[mod_ty]:
# interactive: statement_newline
mark = self._mark()
if (
(a := self.statement_newline())
):
return _PyAST_Interactive ( a , p -> arena )
self._reset(mark)
return NoneIt also has this at the bottom:
return children
KEYWORDS = ('return', 'import', 'from', 'raise', 'pass', 'del', 'yield', 'assert', 'break', 'continue', 'global', 'nonlocal', 'def', 'if', 'class', 'with', 'for', 'try', 'while', 'as', 'elif', 'else', 'in', 'except', 'finally', 'None', 'True', 'False', 'or', 'and', 'not', 'is', 'lambda')
SOFT_KEYWORDS = ('match', 'case', '_')
void *
_PyPegen_parse(Parser *p)
{
// Initialize keywords
p->keywords = reserved_keywords;
p->n_keyword_lists = n_keyword_lists;
p->soft_keywords = soft_keywords;
// Run parser
void *result = NULL;
if (p->start_rule == Py_file_input) {
result = file_rule(p);
} else if (p->start_rule == Py_single_input) {
result = interactive_rule(p);
} else if (p->start_rule == Py_eval_input) {
result = eval_rule(p);
} else if (p->start_rule == Py_func_type_input) {
result = func_type_rule(p);
} else if (p->start_rule == Py_fstring_input) {
result = fstring_rule(p);
}
return result;
}
I am attaching full file to this issue: https://gist.github.com/sobolevn/37cfd12ad9d2c157e72e7589747f4ed2
Pinging @isidentical and @pablogsal
Activity
pablogsal commentedon Aug 30, 2022
The grammar in python.gram is a grammar for the C backend. There is nothing broken here: you just cannot use the Python backend with this grammar.
Pegen is executed on every PR to check that the parser is updated so we have checks to know is not broken.
If you want a grammar that works with the python backend, check the 3rd party pegen module:
https://github.com/we-like-parsers/pegen
pablogsal commentedon Aug 30, 2022
Closing as not a bug.
sobolevn commentedon Aug 30, 2022
Thanks!