Runtime Math Parsing Is the Solution Typst Deserves

This is a response to Laurenz Mädje's post The Math Mode Problem where he discusses five options, A-E, that Typst could use to resolve a syntactic problem in its math mode syntax. I'm not going to rehash the problem description here, Laurenz is a clear writer and an empathetic maintainer. You should read his post first; I can wait. You should also consider reading the replies on the Typst Forum post or in the GitHub PR conversation.

However this post is actually rather late. I originally wanted to write a response within a few days, but I've had difficulty with the writing, and it has now been a few months. In that time Laurenz has already made the choice of option B and merged the relevant PR. And with the 0.14 release imminently approaching, that choice is likely to be made permanent.

Laurenz, I realize it's rude to ask you to consider another opinion on something you thought was settled (especially near a release), and for that I'm sorry. However, I'm publishing this because I not only think that getting this right is important for Typst, but because I think I can change your mind.

I do think the choice of option B was smart. It was a pragmatic choice for Typst and improved over option A, and I've even found myself drawn to it at times while writing this. However, I think option B has a number of downsides that haven't been considered, and I think it would be the wrong choice for Typst.

To explain why, I'm going to discuss the following points:

• Two reasons why option B is harmful for new users
• Some concerns with your arguments around syntax highlighting under option B
• A defense of why runtime math parsing doesn't change the status quo for third party tools
• And an explanation of why it's ok for Typst to prioritize human writability over human readability in its math syntax.

Instead of option B, I believe runtime math parsing, Laurenz's option C, was thrown out too quickly. In this post I intend to reveal its upsides and defend its downsides, and hopefully show why Typst deserves runtime math parsing.

And in addition to this post, I've published an implementation of runtime math parsing as Pull Request 7003. The PR description goes over implementation considerations.

Two harms of option B for new users

In Laurenz's post, he justifies option B with the argument that Typst's users shouldn't need to care about the difference between math functions and Typst functions. This view is intended to benefit new users, but I actually think it creates effects that harm new and learning users in particular.

For new users, option B (1) hides typesetting affordances such as converting sin(x) to sin^2(x) or f(x) into f'(x) and (2) breaks trust in the composability of math in Typst.

Option B hides typesetting affordances

Typst functions and mathematical functions don't have the same affordances: ways in which they can be interacted with. A user learning Typst will benefit from knowing the difference because they will know when to make use of typesetting affordances and when to make use of scripting affordances to accomplish their goals.

My example is the difference between converting sin(x) to sin^2(x) and sqrt(x) to root(3, x). Squaring the sine vs. cubing the root.

When writing with pencil and paper these are very similar things: just write the number smaller and up to the side. But in Typst's math mode, squaring the sine relies on the typesetting affordance of adding an attachment, while cubing the root relies on the scripting affordance of using a different function with an extra argument.

If a new user believes that sqrt and sin are similar, and has only seen examples of either squaring the sine or cubing the root, then they're likely to try the method that works for one on the other and get confused. For example, sin(2, x) just renders "2,x" inside the parentheses, and sqrt^3(x) renders the text "sqrt" monospace with a superscript 3.

These are obviously incorrect, but the explanation of why they're incorrect is dependent on the difference between Typst functions and mathematical functions. These differences aren't going away. And for a learning user to become a good user of Typst, they need to know the distinction.

Option B breaks trust in composability

One of the three design principles on Typst's README is Power through Composability. Composability means that elements of the language should work well together so that users can reuse parts of their document without fear. For example, when users write cal(F) in math mode, the intent is to create a single element, the caligraphic F, and users should expect that element to compose cleanly with other math syntax by being placeable anywhere in an equation without issue.

But option B requires that true Typst function calls placed after attachments, like A_cal(F), now need extra parentheses to parse and evaluate correctly: A_(cal(F)). If you did write A_cal(F), we now generate a warning: "function literal used as subscript; hint: wrap the entire function call in parentheses," and we leave it to the user to fix.

An experienced Typst user can learn that cal(F) is only ever invalid when immediately following attachments, but a learning user that encounters the error message in A_cal(F) will break their confidence in the composability of functions in math.

The main counterpoint is that this break is in a rarely used location, an edge case in the syntax that will only be encountered rarely. But breaking the composability assumption anywhere in the language makes the perception of Typst overall more brittle: small edge cases add up and assumptions rely on trust. But trust is built slowly, and shattered in an instant. Users shouldn't have to question where in Typst's syntax function calls may or may not be atomic, because they always should be.

Laurenz's argument for option B when merging the PR was that it was the most predictable choice, but that argument didn't consider the way option B makes Typst less predictable as a whole language. Runtime math parsing treats Typst functions as atomic units, and persists the user expectations that are built in code mode and markup mode into math mode. Runtime math parsing doesn't break trust in Typst's composability.

What is Syntax Highlighting For?

Laurenz also makes an argument about syntax highlighting for math under option B that I have some concerns with. I'll discuss this over two quotes from his post:

[Option B] means that you can consistently reason about precedence in equations of a field you're unfamiliar with, where you're unsure whether something is a symbol or a custom function. You just need to know: Blue highlighting = function call.

This breaks down when you have false positives, or expressions with the identifier(...) syntax that aren't functions, such as implied multiplication. Highlighting the expression $2 sigma(1 + Gamma(x))$ would color both sigma and Gamma in blue, despite only the latter being a mathematical function.

Another example is the expression quad(...), quite common in modular arithmetic: 3 equiv 1 quad(mod 2). If a user is meant to learn the meaning of unfamiliar symbols by trusting the highlighter, then they're going to have a very difficult time figuring out what a blank space 1em wide is meant to mean as a mathematical function.

I think my disagreement here is in the point of syntax highlighting. I think the point of highlighting Typst math isn't to make the meaning of the mathematics clearer (that's the job of the layout engine!). The point of syntax highlighting is to make the meaning of the Typst source code clearer.

Syntax highlighting helps you instead of tricking you just because the highlighter has no access to runtime information.

My response to this is that the highlighter should have access to runtime information, but only as a progressive enhancement.

Using type information to inform highlighting is already common in IDEs for static and dynamic languages. If you open Rust in VSCode and turn off Rust Analyzer, you'll see a ton of elements change color because their types are not knowable syntactically. For example, without Rust Analyzer, closure variables highlight differently when declared vs. when called—the call is highlighted like a normal function. But with Rust Analyzer enabled, closures consistently highlight as variables to make it clear that they aren't global function definitions.

For Typst, I think it would make the most sense to highlight math on a best-effort basis when we lack runtime information, and to extend the highlighter with runtime information as a progressive enhancement in IDEs such as the web-app or via language servers. Then we can make it so that true Typst function calls are highlighted the same in both math mode and code mode.

Runtime type information will also be useful for highlighting when custom types are added. It will likely be desirable to color types and their constructors differently from other identifiers, and there don't seem to be any plans to syntactically differentiate those two, so runtime information would be needed to do so.

The Downsides of Runtime Math Parsing

Runtime math parsing is seriously contentious. It modifies the entire execution model of the language and makes Typst's math mode more complex and overall less portable for external tools. It will also be necessarily slower than normal parsing, and leaves future options for incrementally parsing math off the table.

But I don't think that moving to runtime math parsing actually changes the status quo for third party tools that would use Typst's math syntax. In particular, I don't think there's any reasonable subset of Typst's math syntax that wouldn't either require the use of a substantial amount of the Typst compiler or would be something that isn't actually Typst.

My view is that runtime math parsing may be an odd implementation detail and an extra burden on tool writers, but I don't think it's an undue burden because it reduces the harms of option B for Typst's users, and should make writing Typst less annoying than otherwise.

Runtime math parsing doesn't really change the status quo

For a third party tool that would actually process or use Typst's math syntax, I don't think runtime math parsing really changes the status quo.

For example, if you're writing about matrices and you want to use square brackets as delimiters, Typst has you covered: just use the delim argument for matrices: mat(a,b;c,d, delim: "["). But you probably don't want every matrix in your file to look like that, so you use a set rule: set math.mat(delim: "["). But set rules are not some simple little feature. Their implementation is wired through half the compiler! If your tool understands set rules, runtime math parsing seems paltry in comparison. And if a tool using Typst's math syntax doesn't allow or understand set rules, then its users are going to want some other non-set-rule method to customize their matrices.

Instead, what if you want to write an equation like 4327 + 817623 = answer and have answer computed for you? Well, you'll write 4327 + 817623 = #(4327 + 817623) and then require the entire runtime system to know what comes after the equals sign, and suddenly you have exactly what you need to get runtime math parsing working.

The ability to easily accomplish these kinds of small tasks is part of what makes Typst so compelling and nice to write with in the first place.

But what if you don't want a complex implementation? If you just want to take a rough sketch of Typsts's math syntax and plaster it into a Javascript library that makes reasonable MathML output then you totally can, and you'll get users. But if you strip out standard library compatibility, variables, code-mode, show rules, set rules, etc. then I don't think those users would be using Typst at all.

Typst's math syntax is a user interface for writing math, and I don't think we should compromise on the user experience for the sake of third party tools.

Human Readability vs. Human Writability

Finally, I'd like to address the issue that runtime math parsing also hurts the human readability of Typst. How an equation might parse is now dependent on the types of defined variables that may be 500 lines up in your document.

I think this is a fair point, but I disagree with it because I consider Typst's normal context of use. The main way that most people write Typst is with a live, incremental update of the document in a different pane of their editor. This means that checking whether something renders as you expect is a glance away.

Typst is a purpose-built language: it produces a rendered document. If you aren't looking at the output at least every once in a while, you're going to mess up. That's not an argument for making the syntax less predictable, but it gives us leniency when we do want to improve the writing experience of Typst (at the expense of reading just the source code). And as a few other commenters have already shown, option B makes for a particularly worse writing experience in the way it adds extra parentheses and particularly doubled parentheses.

So overall my answer to the question, "Why should Typst's math syntax prioritize writing over reading?" is that the thing meant for reading is the rendered document. Typst's math syntax should mirror math notation where it can and should try to avoid senseless ambiguity, but when there's disagreement, I think making writing easier should win out.

This argument applies similarly to Typst's markup mode, but actually doesn't apply to code mode. The reason is related to the fundamental role of syntax in programming languages, but explaining that is part of why this post has been delayed so long, so I'm cutting it and maybe I'll write about it in the future.

Conclusion

Hopefully I've made a convincing argument that runtime math parsing is the solution Typst deserves for the math mode problem. However, the implementation of option B is still merged in the repository, and will be included in the release for 0.14 if there's no action.

I don't think we should delay 0.14's release on this issue, but if you either agree with these arguments or think they merit more discussion, I'd humbly request that we temporarily revert the merged PR (i.e. git revert 7dc99e9) so that users don't have to go through two sets of changes.

Thank you for reading this, Laurenz, the Typst team, and everyone else. I'm again sorry for the delay in getting this out, but I'm glad that it is. This issue has taken a lot of my personal energy for far too long, and I'm excited to focus on new topics for writing.