Let's Fix User Interfaces
If you play a musical instrument, or paint, or cook, you have experienced the idea of creative fidelity. Not all creative tools are the same — some help you express your imagination better than others. Think of a dull knife, or an untunable guitar. Bad at getting your ideas into reality: poor creative fidelity.
Today, anyone involved in building software — notably software with graphical user interfaces like web pages or mobile apps — has experienced more than one squished tomato, or discordant strum.
Our ideas take 1,000,000 paper cuts on their way to shipping, no matter your job title, and no matter what technologies you use. If we consider user interfaces as a medium for art, creative fidelity is low. But it doesn't have to be this way.
It's not you. It's our tools.
And I mean this lovingly, for everyone on this planet who interacts with screens and computers, both creators and consumers.
Our tools for building software — especially user interfaces — could be better. Should be better.
Imagine the lift to the wellbeing of humanity if we could grease the gears of our interface with machines. To make ourselves more creative, to empower more creators, and to enrich our lives with more art and personality. As humanity spends ever more time in front of screens1, the quality and diversity of these experiences become ever more important.
What if creating software could feel like drawing, or painting, or writing poetry? What if anyone who can use a smartphone could contribute directly to a codebase through visual and verbal tools?
Abracadabra
If I could wave a magic wand and change one thing about the past, it would be to erase the differences between Webflow and Figma. Webflow builds software, but it is constrained by the layout system & technical rules of HTML, which is why creative fidelity is so low. You don’t get to draw in Webflow; you must build in a very complicated way.
On the other hand, Figma offers high creative fidelity, but is fundamentally disconnected from code. As a result, Figma is scoped to "drawing pictures" instead of "drawing software." Wrong medium, for this particular goal.
And the heart of the problem is that Webflow vs. Figma, or SwiftUI vs. Photoshop — they speak different languages. In a formal sense: these languages are not bijectively equivalent2, which is why we can't freely translate back and forth between design and code.
But what if “an HTML” were designed exactly to be the backbone of a vector design tool, as well as the backbone of a user interface engine? Let's try something wild — let's imagine a language that describes `R ⋃ F
`, the union of `R: anything you can express in React`,
and `F: anything you can express in Figma
`.
Let’s play this thought experiment out, by building it.
A design tool for codebases
Let's start by building a rendering engine like Figma's. This must be a professional vector design tool, even as it builds software.3 The coordinate system of this rendering engine must be the same as a vector design tool’s, and the vocabulary must include all of the same concepts, e.g. Group, Rectangle, Path, Text.
Why does Figma feel so unlike every other browser experience? Because it's built at the systems level, not limited by JavaScript's stuttering garbage collector.4 Our practical choices for building at this level are C, C++, or Rust. We can use constructs like reference-counted pointers at this layer, operating over a fixed span of linear memory instead of allocating to the JavaScript heap and trigger unpredictable, frame-dropping garbage collections, i.e. stutters.
Another cool thing about building at the systems level is we can target browsers through WebAssembly, but also can build to pure native apps for just about any device on the planet.
Now that we have this rendering engine, instead of packaging it away into a design tool, let's expose it through a declarative language, so that you can write it by hand in addition to drawing it. This is that “HTML-like” language we imagined a minute ago, and it looks exactly like React:
<Group>
<Button />
<SomeCustomComponent />
<Rectangle />
</Group>
To write a <Rectangle/>
is to draw a <Rectangle/>
. And vice-versa.
Let’s sprinkle in an expression language, so this can (1) remain 100% declarative, just data, so it can be read & written deterministically, but also (2) allow you to express anything you might express with React and (3) forge a connection with programming languages, so this whole thing can attach to a programming language in the spirit of HTML.
<Group>
<Button @click=handle_click />
<SomeCustomComponent some_prop={some_data + 5} />
<Rectangle fill={hsl((num_clicks)deg, 100%, 50%)} />
</Group>
Snap it together with templated reusable components, compile across platforms, customize and extend with Rust (or TypeScript, coming soon.)
We’ve already built all of the above — it’s free and open source and compiles into extremely fast (240FPS) native apps or WebAssembly. Check out Pax Engine on GitHub.
A more creative future
Finally, let’s build a vector design tool, using the language above as the source of truth, reading & writing that language with every operation. This makes it so you can hand-code a user interface declaration alongside visually designing it. Use the best tool for the job and for your skillset.
The latest design can be opened at any time in the project’s journey, just like opening a codebase with VSCode, or opening a design file with Figma.
Critically, the design now lives in source code and ships to production — contributions are made as pull requests and non-developers can contribute to the same codebase as developers. Artists, creatives, marketers — anyone who can use a spreadsheet or a design tool or a notepad — now becomes a builder of software.
That’s Pax. And we’re doing this all so we can have a more creative, more collaborative, more inclusive future — a better interface between humans and machines.
Pax Create is our design tool for making cross-platform apps & websites, built around the open source Pax Engine. As of June 2024, Pax Create is pre-release, but we are getting close to our first launch. Sign up for early access or learn more at pax.dev.
Flash proved this was possible in a creative tool, and it was exactly this connection of disciplines that elevated Flash websites and cartoons to an entirely different league of creative fidelity than anything we've seen since.
GPU rendering is an important piece of the puzzle, too, namely for scaling to complex projects — but perhaps surprisingly, that pesky garbage collector is the number one reason why Figma feels “smooth” and most other things in the browser don’t. See e.g. https://www.researchgate.net/publication/253255632_Sporadic_frame_dropping_impact_on_quality_perception