⇦

2024-10-17

Adding syntax to the cpython interpreter

Condensed version of this cool blog post.

Let's add some new syntax to Python! Making a small change is not so hard.

Our aim is to make ternary statements default to None as they do in Ruby:

>>> "hello" if 2 + 2 == 4
"hello"
>>> "hello" if 2 + 2 == 5
None

In existing Python, we get an error:

File "<python-input-0>", line 1
    "hello" if 2 + 2 == 5
    ^^^^^^^^^^^^^^^^^^^^^
SyntaxError: expected 'else' after 'if' expression

First, let's clone and build Python:

git clone git@github.com:python/cpython.git
cd cpython
./configure
make

Now let's run the Python interpreter we built and check it works:

./python.exe
>>> 2 + 2
4

Now lets's change the grammar so that if we don't have an else condition we default to None.

First find the following in Grammar/python.gram:

expression[expr_ty] (memo):
    | invalid_expression
    | invalid_legacy_expression
    | a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) }
    | disjunction
    | lambdef

And change it to:

expression[expr_ty] (memo):
    | invalid_expression
    | invalid_legacy_expression
    | a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) }
    | a=disjunction 'if' b=disjunction { _PyAST_IfExp(b, a, _PyAST_Constant(Py_None, NULL, EXTRA), EXTRA) }
    | disjunction
    | lambdef

Now lets regenerate the c files from the grammar:

make regen-pegen
git diff  # to see what changed

And compile the interpreter again:

make

Now we have our shiny new ternary expressions:

./python.exe
>>> print("hello" if 2 + 2 == 4)
hello
>>> print("hello" if 2 + 2 == 5)
None
>>>

Yay!