Performance evaluation report

Motiff is a productivity tool for designers, where users work dozens of hours a week.

While innovative AI features are essential for boosting efficiency, the experience of basic operations like dragging, zooming, and copying also significantly impacts a designer's productivity. Motiff emphasizes the performance of engineering efforts in every feature to make sure that designers can enjoy a fully functional editor with a smooth and stable user experience.

Overview

This performance evaluation aims to simulate common and necessary high-frequency operations in designers' daily work and record the performance during these operations. For a more intuitive understanding and measurement of Motiff's performance, we conducted the same operations on Figma and recorded its performance as a reference under the same testing environment.

The results in this report are from our evaluation completed in May 2024. Like all performance evaluations, results may vary due to differences in devices, resolution, operating systems, network bandwidth, and even the number of programs running in the background. We strive to make the evaluation results as fair and objective as possible, but your results may vary.

Details regarding the testing environment for this evaluation are as follows:

• Device: iMac (M1, 2021)
• Screen: 24-inch 4.5K Retina display, 2240 x 1260
• Memory: 16GB
• System: macOS Sonoma 14.5
• Browser: Chrome 125.0.6422.113 (arm64)
• Example files: To ensure that the example files sufficiently cover the real needs of designers' work, we used the Google Material Design suite (Material 3 Design Kit) as the basis for the sample files. We are grateful for their effort. Detailed information about all the sample files used in this evaluation can be found in the appendix.

Performance evaluation checklist

The performance concerns of different design teams are affected by their own workflow, project complexity, collaboration scope, etc. After referring to the workflow of many design teams and work preferences, we identified "smoothness," "speed," and "stability" as three important factors with widespread impact:

• Smoothness During creation, designers frequently perform various operations on canvas and layers, such as moving and zooming canvas, dragging layers, and bulk copying. Ensuring the smoothness of these frequent operations requires a seamless editing experience and instant visual feedback, both of which depend on excellent rendering effects.
• Speed Creativity should not be interrupted. The long and frequent waiting times during various operations, especially opening files and switching pages, can be very frustrating.
• Stability As the business grows and the number of collaborators increases, design files also grow larger. Gradual lagging and even crashing can bring work to a halt. The ability to manage large multilayered files stably and have higher layer limits is necessary.

To evaluate different dimensions of performance, we set up different observation indicators. The following measurements are used in this evaluation:

Performance evaluation results are as follow.

Smoothness: Rendering effects

During the evaluation, we used the same layer content to perform the same zooming, moving, and other continuous operations in Motiff and Figma respectively. For these kinds of continuous operations, we used FPS and jank rate as quantitative indicators metrics to assess the performance more precisely than direct visual observations.

FPS

FPS, or frames per second, represents the average number of frames rendered per second. It is a key indicator of visual smoothness. the higher the FPS, the lower the delay in visual feedback.

• 24 FPS: The standard frame rate for traditional movies that is considered to provide a natural and comfortable viewing experience.
• 30 FPS: Used by many TV shows and video games. It is a little bit smoother than 24 FPS and is often used in action scenes.
• 60 FPS: Used for high-resolution sports broadcasts and the most high-quality games. It offers a very smooth and realistic visual experience for fast-moving scenes.

In actual practice, the instantaneous performance of FPS can fluctuate due to several factors such as the graphics performance of the device, screen resolution, and the complexity of the layers. Considering the characteristics of different scenarios, we believe that maintaining an FPS above 50 ensures an excellent user experience, while an FPS below 20 results in noticeable jank and poor performance.

The following chart shows the FPS performance we sampled in this evaluation:

As you can see, Figma's FPS performance is almost consistent at an ideal level of 60 when zooming or moving the canvas. Although Motiff shows some fluctuations at times, the overall visual experience remains smooth. There is still a need to learn from Figma's stable performance. In terms of dragging and batch copying, thanks to the optimized mechanism of the self-developed rendering engine, Motiff still stays above the target range of 50 FPS in scenarios where a large amount of computation is required.

Jank rate

Even with excellent FPS performance, if the refresh rate is uneven and varies in speed, the visual experience can still feel choppy. Therefore, we introduce the jank rate as an indicator to assess whether the refresh rate is consistent. When the duration of any single frame exceeds twice the average duration of the previous three frames, it is recorded as a jank. Meanwhile, the corresponding jank duration will also be recorded. The jank rate is calculated as the proportion of jank duration to the total duration within a unit of time. A lower jank rate indicates smoother and more seamless visual feedback.

You can see that even though the FPS on the left reaches the ideal level of 60, it is still not as smooth as the right side with an FPS of only 20 when the jank rate reaches 20%. Similarly, the jank rate can also fluctuate due to various factors.

In actual practice, we believe that the target range for the jank rate should be below 20%, as exceeding this threshold can result in an unsmooth visual experience.

The following chart shows the jank rate performance we sampled during this evaluation:

In comparison, it's noticeable that fluctuations in the FPS indicators often appear with the increase in the jank rate. From a visual perspective, when FPS maintains stable performance, minor fluctuations in the jank rate do not affect the smoothness of operations. Similarly, thanks to optimizations of its self-developed rendering engine, Motiff's jank rate performance currently stays within the target range of below 20%. With further optimization in the control of jank rate performance following the stability of FPS, we will continue to enhance our performance.

Summarizing the performance in the "smoothness" aspect of the evaluation:

The performance evaluation in terms of "smoothness" is summarized below:

• For zooming and moving canvas operations, Motiff and Figma are evenly matched. Figma performed better in maintaining stability, while Motiff aims for further improvement.
• In dragging layers, Figma exhibits noticeable "frame drops" and jank, while Motiff provides a smoother visual experience with more prompt responses to layer movements
• In bulk copying of layers with the Option command, Motiff not only copies faster than Figma but also provides a more apparent sense of smoothness.

Speed: File loading speed and page switching speed

To ensure that the actual scenarios of most design files are covered. We used 3 design files with different sizes for this evaluation:

• Total layers: 60,000; Single page layers: 30,000
• Total layers: 120,000; Single page layers: 60,000
• Total layers: 360,000; Single page layers: 180,000

Excluding interference caused by network fluctuations, we conducted 5 tests for each file, taking the average as the final evaluation result.

File loading speed

As the number of layers in the file increases, the file loading speed will also slow down. Overall, Motiff performs faster on all three files with different sizes and has great performance in controlling latency for large files.

Below are the detailed data from the 5 evaluations:

Page switching speed

In terms of page switching speed, thanks to optimizations in lightweight storage and resource loading strategies, Motiff achieves almost seamless page transitions. Even in the case of large files with 360,000 layers (every single page has 180,000 layers), page switching can be completed in 0.8 seconds.

Below are the detailed data from the 5 evaluations:

We also used the developer tools in the browser to keep a detailed record of the front-end loading speed for page switches. You can refer to Appendix 3 for details.

To summarize the performance in terms of "speed":

• Regarding file opening time, as the number of layers in the file increases, the time taken also increases. However, Motiff performs excellently by taking a shorter time overall and showing better performance as the time taken increases.
• In terms of page transition time, Motiff has an outstanding performance. Even for files with large layers (360,000 layers), page switching speed can be completed within 1 second, whereas Figma requires a longer wait.

Stability: Layer limits

After chatting with many friends who are using Figma, we have found that as design content continues to increase in a single file, designs become increasingly choppy, and memory shortage warnings may occur.

Although there is a certain limit to the number of layers a single page can support, Motiff aims to increase this limit so that smooth editing experiences can still be supported even with larger numbers of layers.

The following evaluation represents an extreme performance competition. After optimizing various performance aspects, we tested the limits of both Figma and Motiff under extreme conditions.

The competition results are listed below:

• Motiff still smoothly completes a series of editing operations even after reaching a final count of 1.08 million layers.
• Figma's performance peaks at 480,000 layers, after which it enters recovery mode during subsequent operations.

Stable and reliable performances are comprehensive results. Motiff's performance with a million layers benefits from optimizations made to its self-developed rendering engine for smoothness. Strategies for lightweight storage and resource loading speed are equally important. This is evident in its performance when bulk copying a large number of layers.

Fundamentally, it benefits from a high-performance underlying that accurately high-performance architecture schedules and efficiently allocates computational resources. We will continue our commitment to improve and explore ways to push the limits further.

Appendix

Sample files for testing

Download links：

[1] Smooth_ 60K Layers on 1 page

[2-1] Speedy_ 60K Layers on 2 pages

[2-2] Speedy_ 120K Layers on 2 pages

[2-3] Speedy_ 360K Layers on 2 pages

[3] Robust_ Single page capacity test

Rendering performance monitor

1. 1.Install and use the browser extension: Tampermonkey
2. 2.Configure the monitoring code

// ==UserScript==
// @name         Smooth performance
// @namespace    <http://tampermonkey.net/>
// @version      2024-05-20
// @description  FPS & Jank rate
// @author       Motiff
// @match        <https://www.figma.com/*>
// @match        <https://motiff.com/*>
// @icon         <https://www.google.com/s2/favicons?sz=64&domain=motiff.com>
// @grant        none
// ==/UserScript==

(function () {
  var stutterPanel = document.createElement("div");
  stutterPanel.setAttribute("id", "monitor");
  stutterPanel.style.position = "fixed";
  stutterPanel.style.left = "65%";
  stutterPanel.style.top = "15px";
  stutterPanel.style.color = "red";
  stutterPanel.style.zIndex = 10000;
  document.body.append(stutterPanel);

  let lastRenderTime = performance.now();
  let lastStatisticsTime = lastRenderTime;
  let renderCount = 0;
  let maxRenderDuration = 0;
  let renderDurations = [];
  let stutterDurationSum = 0;

  function render() {
    const now = performance.now();
    const renderDuration = now - lastRenderTime;
    renderDurations.push(renderDuration);
    if (renderDurations.length > 3) renderDurations.shift();
    const avgRenderDuration =
      renderDurations.reduce((a, b) => a + b, 0) / renderDurations.length;
    if (renderDuration > 2 * avgRenderDuration) {
      stutterDurationSum += renderDuration;
    }
    maxRenderDuration = Math.max(maxRenderDuration, renderDuration);
    lastRenderTime = now;
    renderCount += 1;

    const statisticsDuration = now - lastStatisticsTime;
    if (statisticsDuration > 1000) {
      const fps = (renderCount / statisticsDuration) * 1000;

      const date = new Date()
      const currentTime = date.getHours() + ":" + date.getMinutes().toString().padStart(2, '0') + ":" + date.getSeconds().toString().padStart(2, '0');
      const logTime = (date.getMonth() + 1).toString().padStart(2, '0') + "-" + date.getDate().toString().padStart(2, '0') + "-" + date.getFullYear() + " " + date.getHours() + ":" + date.getMinutes().toString().padStart(2, '0') + ":" + date.getSeconds().toString().padStart(2, '0') + "." + date.getMilliseconds();
      console.log(`${logTime} ｜${Math.floor(fps * 100) / 100} ｜${Math.floor(stutterDurationSum / statisticsDuration * 10000) / 100} ｜${Math.floor(maxRenderDuration * 100) / 100} `);
      stutterPanel.innerHTML = currentTime
          + " ｜ FPS: " + fps.toFixed(2)
          + " ｜ Jank rate: " + Math.floor(stutterDurationSum / statisticsDuration * 10000) / 100 + "%"

      lastStatisticsTime = now;
      renderCount = 0;
      maxRenderDuration = 0;
      stutterDurationSum = 0;
    }

    requestAnimationFrame(() => {
      render();
    });
  }

  requestAnimationFrame(() => {
    render();
  });
})();

3, Refresh the page to view real-time indicators

a. After completing the configuration, refresh the webpage, and you can instantly view the corresponding page's FPS and jank rate indicators at the top of the page.

Screenshots for front-end loading speed

Screenshots for front-end loading speed