⇦

2020-08-29

Evolving backend → frontend rendered Python / JS apps with dnjs

With very little ceremony or surprise, dnjs lets you specify configuration , generate static html/css , and render html on the backend/frontend across languages. This page is even written with it. A word of warning: the project is still in its development stages.

What's the problem?

In most worlds, when you're building a web app, there is a very clear divide between:

  1. "Let's build a backend-rendered page in Python / JS /... rendering the html with some funky templating language. We can sprinkle in some ad-hoc JS as we get there."
  2. "Let's build a fully frontend-rendered app with React / Vue /..."

A lot of the time, teams plump for option 2, even when the user experience might be worse (slow loading times, lack of hyperlinking, etc), and the developer experience might be worse (lots of client state, hard to test, etc).

dnjs takes the following positions:

It attempts to resolve these with a pure subset of JS that is reasonable to implement in host languages and has a standardised way of describing DOM nodes . Being able to share things across the backend and the frontend means we can evolve our app over time. We can start with simple html forms, then progress to frontend-rendering as the complexity requires - while reusing previously written components .

Let's walk through an example where we span this transition.

An example app

In this post, we will build the following simple web app, where we can add todos and check them if they're done. The data will be persisted on the backend:

The 3 ways we will build it are:

Aside: running the examples

The source for the above can be found in the examples/todo directory of the dnjs repo. It has the following structure: 

├── app.py
├── shared
│   ├── components.dn.js
│   └── style.dn.js
├── static
│   ├── backend.js
│   ├── classic.js
│   ├── declarative.js
│   └── style.css
└── templates
    └── page.dn.js

You should be able to run the server with: 

git clone git@github.com:leontrolski/dnjs.git
cd dnjs; python3 -m venv .env; source .env/bin/activate
pip install dnjs fastapi uvicorn aiofiles python-multipart

examples/todo/bin/serve
open 'http://localhost:8000/backend'

Building the app with backend

We're going to use the Python library FastAPI for our backend. Let's start by looking at the body of the html we want to end up with: 

<h1 class="todo-bold todo-red">Hello Ms Backend</h1>
<div id="todo-list">
    <form id="todoListForm" method="POST">
        <ul>
            <li class="todo-todo">
                hullo
                <input class="doneCheckbox" name="doneCheckbox" value="0" type="checkbox" />
            </li>
            <li class="todo-todo">
                goodbye
                <input class="doneCheckbox" name="doneCheckbox" value="1" type="checkbox" checked />
            </li>
        </ul>
        <input name="newMessage" value="" placeholder="message" autocomplete="off" />
        <button>add todo</button>
    </form>
</div>

As you can see, we're going to render just a normal html <form> . When we submit the form, it will POST one newMessage with value="some message" and many doneCheckbox s, each with value="index" - the indexes of todos that are checked.

An aside on html forms

Html forms are really not a thing of beauty, they map poorly to json structures, you can't nest them, they can only convey strings, etc. Lots of frontend devs in particular seem to have forgotten the weirdo ins and outs of how they work, for example: that <button> s and <input type="checkbox"> s have (name, value) s that are only are POSTed when they are clicked/checked.

Despite these shortcomings, they are well worth using for simple forms (that is to say - most forms). Not using any JS means having basically zero state in the browser, and that makes reasoning about/testing the app way easier. Like way way easier.

Back to the app

Let's have a look at the dnjs templating code we use to render the page from Python , the base page is in examples/todo/templates/page.dn.js and looks like: 

import { TodoList } from "../shared/components.dn.js"

const title = [classes.bold, classes.red]

export default (data) => (
    ...
    m("h1", {class: title}, "Hello ", data.username),
    m("#todo-list", TodoList(data.state)),
    ...
)

The components for the form itself can be found in examples/todo/shared/components.dn.js and look a bit like: 

export const Todo = (todo) => m("li",
    todo.message,
    m("input.doneCheckbox", {name: "doneCheckbox", value: todo.i, checked: todo.done, type: "checkbox"}),
)

export const TodoList = (state) => m("form#todoListForm",
    {method: "POST"},
    m("ul", state.todos.map((todo, i) =>Todo({...todo, i: i}))),
    m("input", {name: "newMessage", value: state.new}),
    m("button", "add todo"),
)
The equivalent handlebars code would look something like this. 
<h1 class="{{classes.bold}} {{classes.red}}">Hello {{data.username}}</h1>
<div id="todo-list">
    <form id="todoListForm" method="POST">
        <ul>
            {{#each state.todos}}
                <li>
                    {{message}}
                    <input class="doneCheckbox" name="doneCheckbox" value="{{@index}}" {{#if done}}checked{{/if}} type="checkbox" />
                </li>
            {{/each}}
        </ul>
        <input name="newMessage" value="{{state.new}}" />
        <button>add todo</button>
    </form>
</div>

We will render the page in examples/todo/app.py with: 

@app.get("/backend", response_class=HTMLResponse)
def get_backend() -> str:
    return dnjs.render(
        templates / "page.dn.js",
        PageData(
            username="Ms Backend",
            state=State(todos=_todos.todos, new=""),
        )
    )

We're rendering the dnjs (a subset of JS ) with Python ... 🤯

In the same file, we handle POST s with: 

@app.post("/backend", response_class=HTMLResponse)
def post_backend(newMessage: str, doneCheckbox: List[int]) -> str:
    if newMessage:
        _todos.todos.append(Todo(message=newMessage, done=False))

    for i, todo in enumerate(_todos.todos):
        todo.done = i in doneCheckbox

    return render(...)
Note, for demo purposes, we are storing the todos serverside with an in-memory structure _todos .

In case you can't be bothered to run the example application, here's a video of me clicking around the backend app with the console on.

Adding some frontend rendering with classic

The users of our app were posting lots of frequent comments, we decided it wasn't acceptable to have a page reload every time it happened. This means we had to add a sprinkling (23 lines) of JS to:

The bulk of this - examples/todo/static/classic.js - is as follows: 

import { Todo } from "../shared/components.dn.js"

...

todoListFormEl.onsubmit = async e => {
    // prevent the <form> from actually submitting
    e.preventDefault()

    ...

    // construct the todo data and send to the backend
    const data = {
        message: newMessageEl.value,
        done: false
    }
    await fetch(
        "/classic/todos",
        {method: 'POST', body: JSON.stringify(data)}
    )

    // make the Todo DOM node and append it to the DOM
    const newTodo = Todo(data, null)
    todoListUlEl.appendChild(m.makeEl(newTodo))

    ...
}

The important thing here is that we were able to reuse the same Todo component that we used on the backend on the frontend . Cool huh!

dnjs2dom

In the classic app, we use the teeny dnjs2dom library to turn the m(...) components into actual DOM nodes. This library enables the line todoListUlEl.appendChild(m.makeEl(newTodo)) .

Let's show a video of this in action:

Going whole-hog frontend rendering with declarative

It's a bit contrived with this simple app, but let's imagine our UI's complexity has tipped over to the point where we want everything super dynamic. To achieve this, we will use the (excellent) mithril - which dnjs is handily compatible with.

Here's the line from examples/todo/static/declarative.js where we mount our components to the DOM with mithirl: 

m.mount(todoListEl, {view: () => TodoList(state, actions)})

Note again - we were able to reuse the same TodoList component that we used on the backend on the frontend.

mithril wraps any {onclick: f, onsubmit: g, ...} functions and rerenders the DOM as necessary in similar way to React . Below is an example of one of the functions we pass via m("form", {onsubmit: add}, ...) . 

const add = (e) => {
    e.preventDefault()
    if (!state.new) return
    state.todos.push({message: state.new, done: false})
    state.new = ""
    m.request({
        url: "/declarative/todos",
        method: "PUT",
        body: {todos: state.todos}
    })
}
Note that if we want to render the same component from the backend, we just pass in {onsubmit: None} and it is ignored by dnjs .

Let's show a video of our declaratively rendered app in action:

Bonus css section

Our app has many pages, so we'd like to share/compose/namespace our css. Handily dnjs can convert from examples/todo/shared/style.dn.js : 

const _classes = {
    bold: {
        "font-weight": "bold",
    },
    red: {
        "color": "red",
    },
    ...
}

// namespace
export default Object.fromEntries(
    Object.entries(_classes)
    .map(([k, v], _) => [".{namespace}-{k}", v])
)

Via: 

dnjs shared/style.dn.js --css > static/style.css

To: 

.todo-bold {
    font-weight: bold;
}
.todo-red {
    color: red;
}

In examples/todo/shared/style.dn.js we also export the names themselves of the css classes, so in the rendering code, we can refer to eg: style.classes.bold and it will resolve to the name: "todo-bold" .

Similar to the advantages in html-rendering land, having your css as "just data" means you can easily compose/transform it, again without resorting to a whole new DSL like Sass .