Performance magic behind Motiff

How hard can building a UI editor be? When our team decided to embark on creating an editor for the AI era two years ago, we didn't think it was going to be that tough. With the plenty of open-source solutions available, we assumed that integrating some AI capabilities with these existing solutions would suffice to develop a product. However, as we delved deeper into the project, we realized that the performance demands of the editor made what seemed like standard features extraordinarily difficult, such as:

• Moving layers in bulk
• Opening files
• Switching pages

You might shrug off these functionalities as basic - we did too, at first...

It’s Just Drawing a Few Rectangles, How Hard Could It Be?

Motiff is packed with designer-friendly features such as auto layout and component instances, which introduce a complex web of data relationships. For instance, changing a layer's size means its parent frame needs rearranging; altering a vector layer necessitates a redraw of its associated Boolean layer; tweaking a component might impact hundreds or thousands of its derived instances.

At the end of 2021, as we began crafting Motiff, the computation challenges in manipulating this data web didn't fully register with us.

Just days into the project, by leveraging Skia's C++ code, our front-end engineers conjured up a demo that could draw a few rectangles using the good ol' React + Redux stack. Encouraged by this early success, the team managed to nail down most of the editor functionalities in a few months. It looked like we were on the brink of release, but then we hit two major snags: we hadn’t integrated the component instances yet, and we hadn’t conducted any performance stress tests.

It was only when we started to grapple with these issues that we had our "oh no" moment. Between struggling with complex logic and watching our frame rates hit the single digits, it dawned on us: we had strayed from the right path from the start. We had to go back to the drawing board and rework our computational architecture, painstakingly extracting and transferring the relevant logic from React + Redux to a new C++-based framework.

In hindsight, Motiff's unique feature set led us down this path:

• Compared to traditional GUIs: Features like component instances trigger extensive computations, and many designers are accustomed to stacking numerous layers on an infinite canvas. They navigate quickly through zooming and selecting multiple layers for bulk movement. In contrast, traditional GUI scenarios don't typically face such intensive update and rendering requirements. Or if they do, a bit of lag is often deemed acceptable. Systems like React + Redux and Vue are designed with the mindset of updating a small piece of a massive GUI. Skia, serving as the foundational library for traditional GUIs, isn’t optimized for handling mass updates either.
• Compared to gaming: Games don't require high precision in rendering; objects not visible within the viewport can be ignored or not rendered. Distant objects can be simplified or merged, and sometimes even completely disregarded. When porting triple-A titles to the handheld Switch console, these techniques are employed to reduce computational demands. However, Motiff's users expect highly accurate rendering, even for rapid zooming on an infinite canvas. What’s more, running Motiff in the memory-constrained environment of WebAssembly means we can't simply transplant batching techniques from games wholesale.

These contrasts forced us to think outside the box, drawing on the best practices from GUI and game architectures but grounded in first principles and original thought. Next up, I'll dive into the specific optimizations we deployed for three distinct business scenarios.

Precision Updates - Smaller Moves, Faster Speed

In Motiff, design drafts are viewed as a Key-Value (K-V) database, where each design element (such as rectangles, circles, text, etc.) is identified by a key consisting of a unique identifier (NodeId) and a property key (PropKey). Each key corresponds to a Value, which can be a number, string, color code, position coordinates, etc., depending on the type of property.

For example, if we draw a rectangle in Motiff, the data in the files can be represented as:

It can also be seen as a Map to help us understand the content of this blog post. For such a Map, we have three basic operations:

• Create: Create a K-V Pair
• Update: Update a K-V Pair
• Remove: Delete a K-V Pair

Any editing action is a combination of such operations in this K-V database. For instance, when a designer starts a new interface design through Option copy, the moment the mouse drags, Motiff needs to create thousands of new K-V Pairs in the files through the Create operation.

When the size of a layer changes, its parent frame also needs to be re-layout. In the initial demo version, Motiff accomplished this by adding extra logic in the property update function, as shown below:

setWidth(width: number) {
    this.width = width
    this.getAncestors().forEach(node => node.reLayout())
}

However, this method of immediate updates, while ensuring data consistency, could also lead to performance issues, especially during bulk editing. For example, in bulk editing layers:

The code in this scenario means the editor needs to adjust the position of each selected layer. However, However, stuttering often occurs during the adjustment process.

In Motiff, the scale of the computations associated with such manipulations is much larger than the volume of data being directly updated. So, it's easy to conceive that a reasonable approach is to trigger "related" recalculations only for the updated data after its update.

To enhance performance, Motiff adopted a strategy of batch updating, by combining multiple fragmented update operations into a single batch operation. The merged algorithm makes it convenient for programmers to optimize performance, reducing unnecessary computations. In theory, we could first batch update a selection of Nodes directly.

During the entire bulk editing process, each attribute of each node would only be computed once. This is the foundation of Motiff's data computation algorithm.

Motiff encapsulates these algorithms for maintaining files consistency into separate Systems. When the files is modified by the user's direct actions, related Systems are triggered to execute. They run in a certain order, gradually updating the files‘ related properties to ensure the files’ final consistency.

Here are two sets of data, moving 1024 rectangles inside a Boolean frame, comparing individual updates to batch updates:

• Individual updates: 20647 ms
• Batch updates: 54 ms

Opening Large Files - A Distinctly Different Challenge from Editing

Precision updates are based on change events. However, it is not necessary to convert files recovered from compressed archives into an intermediary state of change events initially. At the same time, the processing algorithm based on change events needs to consider both the existing files data and the new modifications, making its algorithm complexity higher. To optimize the speed of opening files, each System has implemented a fast batch tree traversal algorithm specifically for bulk creation scenarios. Compressed archives are first decompressed and deserialized into an initial node tree, and then each System traverses this tree to populate the properties that need to be calculated.

Here are two sets of data for opening file with 4000 rectangles, comparing an optimized algorithm to a non-optimized algorithm:

• Non-optimized algorithm: 758 ms
• Optimized algorithm: 341 ms

Besides the opening speed, another challenge is whether there is enough memory to open large files. A Node's properties can number in the hundreds, but often only a few dozen are used. If defined in the most intuitive way, each Node as a Struct with hundreds of properties would consume a lot of memory to indicate that this Node does not have this property. However, if not using a Struct and instead storing objects like dynamic typed languages using a Map, it would cause a lot of memory fragmentation and slow down the property read/write speed. This leads to our memory management strategy not being able to choose a single solution but needing to make different memory layout choices for different properties.

Here are two sets of data, holding ten thousand rectangles, comparing optimized memory layout to using a Map:

• Storing with Maps entirely: 183 MB
• Optimized memory layout: 4.3 MB

In addition to more precisely using memory, we have also supported new features of the latest version browsers, raising the memory limit from 2GB to 4GB. This allows us to put more layers within a single page.

Currently, Motiff can ensure a smooth editing experience for users working with files that contain up to a million layers on a single page.

High-Performance Rendering Process - The Final Round of Smoothness Calculation

In the early stages of development, we chose Skia as our rendering engine. Skia is an open-source graphics library, and it serves as the rendering engine for Chrome, Android, and Flutter. With Skia, we could easily draw various layers:

void draw(SkCanvas* canvas) {
    SkPaint p;
    p.setColor(SK_ColorRED);
    p.setAntiAlias(true);
    p.setStyle(SkPaint::kStroke_Style);
    p.setStrokeWidth(10);
    canvas->drawLine(20, 20, 100, 100, p);
}

At the outset, our team held an optimistic view that since Skia was the rendering engine behind a multitude of mainstream GUI libraries, it would naturally excel in efficiently rendering Motiff as well. Our assumption was straightforward: simply utilize Skia's drawing API to update layers following any changes in their data, enabling us to render the design drafts directly within the browser. Nonetheless, upon deploying Skia for these tasks, the anticipated ideal performance fell short of our expectations, and the underlying issues were far from straightforward:

1. 1.Skia forces a reranking of rendering instructions that do not match the performance we could achieve with manual arrangements in understanding business scenarios. Moreover, this reranking itself incurs a significant performance cost.
2. 2.Due to the need for reranking, Skia accumulates rendering instructions, reranks them, and then executes all rendering instructions at once. This makes it difficult to limit rendering time within a specified range.
3. 3.For the same path, there are different rendering algorithms. Skia selects the algorithm based on fixed rules, whereas we could choose the most optimal algorithm based on our business scenarios.

Eventually, we used WebGL to implement Motiff’s own rendering engine, completely saving the instruction reranking time, while also optimizing paths and effect algorithms, making the overall performance 2.27 times that of the initial version implemented with Skia.

The complexity of achieving high-performance rendering goes beyond this. Any rendering stuttering can easily be noticed by users, particularly when zooming in and out of the canvas area, thereby affecting the user experience. However, blindly reducing the rendering time per frame would significantly prolong the overall rendering time, resulting in users seeing blanks in the files content for an extended period. The biggest challenge in rendering is how to render more content within a single display refresh cycle, while minimizing the probability of exceeding the time limit.

The scale of files is limitless, while the performance of devices is limited. Obviously, it's impossible to guarantee that all visible files content can be rendered within a single display refresh cycle, regardless of the file size. We use a technique called "tiling", which divides the screen into numerous fixed-size areas. The rendering target is no longer the entire screen but each individual tile.

Tiling has many benefits. It not only divides tasks but also establishes a unit of cache. For example, during the editing process by users, only the tiles within the editing area need to be updated, while the other tiles remain unchanged.

During the zooming process of the user's canvas area, new tiles can also replace previous ones with different zoom ratios, as the difference in rendering content is merely a change in zoom ratio.

If a tile contains a lot of content, it might not be completely rendered within a single display refresh cycle. In our initial rendering engine, either we forced the rendering of this tile, causing a timeout, or we further divided this tile, extending the overall rendering time. In the current Motiff rendering engine, we have the ability to interrupt the current rendering task and continue this rendering task in the next display refresh cycle, thus minimizing the overall rendering time.

In addition, we developed a unique data structure dedicated to rendering, named RenderTree. This structure is finely tuned to contain precisely the data necessary for rendering, which guarantees a continuous stream of data in memory. By harnessing the principle of spatial locality in CPU memory access, we've significantly enhanced access efficiency. Concurrently, we've implemented caching for the outcomes of computations related to rendering. This strategy effectively minimizes the computational load needed during the rendering phase, especially when files are in a static state and not undergoing edits.

A Brief Recap

In the world of Motiff, nothing takes precedence over performance. The content of this blog post represents just a portion of the efforts we’ve made toward enhancing Motiff’s performance. Through essential improvements in data storage, memory management, update protocols, and rendering tactics, we’ve empowered designers to manipulate design projects that consist of millions of layers. The broader the capability of the tool in handling extensive design projects, the smaller the creative limitations imposed on the designer. With ongoing changes in design methodologies, Motiff is on a continuous journey to upgrade its framework for computing data, aiming to fulfill the ever-escalating expectations for power and stability from designers.