Origin

Lottie is a popular animation library that somewhat fills the gap in animation for front-end (and also client-side) left by the Flash era. After Lottie appeared, the design and development of animations began to have a clearer division of labor: designers can use Adobe After Effects (hereinafter referred to as AE) to design animations, and then use the BodyMovin plugin to export a JSON protocol. The front-end reads the data in the JSON and plays the animation through a lightweight runtime.

However, the official runtime provided by Lottie, Lottie-web, also has some limitations:

1. Initially based on SVG, the performance of DOM operations is poor;
2. In Canvas 2D mode, it is difficult to combine multiple Lotties on the same canvas;
3. It lacks some advanced features in AE, such as 3D transformations.

When developing the 2D functionality of the Galacean Engine, we also encountered the need to play Lottie animations in a WebGL context. Our idea is to position Lottie as an ecosystem component similar to Spine, meaning that the algorithm and rendering are decoupled, and each game engine can implement its own proprietary component renderer.

Understanding the Protocol

Data Structure

Since Lottie's JSON protocol is exported after being created by AE, its data structure is consistent with the layer structure in AE. Here, thanks to the community, someone has already organized a detailed structure diagram (see references). As shown below, the overall structure of the protocol is relatively simple:

{
  "v": "5.4.4",	// version：bodymovin的版本；
  "fr": 30, 		// frames per second：帧率；
  "ip": 238,		// in point：第一帧位置；
  "op": 437,		// out point：最后一帧率位置；
  "w": 750,			// width：动画宽度；
  "h": 750,			// h（height）：动画高度；
  "nm": "0",		// name：名称；
  "assets": [		// assets：资源；
	  {}
  ],
  "layers": [		// layers：图层；
 		{}
  ],
}

The most critical fields are assets and layers. To summarize their relationship in one sentence: a layer in layers can reference a resource in assets, and another resource can reference other resources. The first part is easier to understand, for example, setting the layer type ty to 2 (indicating a layer containing an image) and setting refId to "0" can reference the image resource with id "0" in assets:

{
  "assets": [
    {
      "id": "0",
      "w": 556,
      "h": 308,
      "u": "",
      "p": "data:image/png;base64...",
      "e": 1
    }  
  ],
  "layers": [
    {
      "ddd": 0,
      "ind": 1,
      "ty": 2,
      "nm": "00000.png",
      "cl": "png",
      "refId": "0",
      "sr": 1,
      "ks": {},
      "ao": 0,
      "ip": 0,
      "op": 1,
      "st": 0,
      "bm": 0
    }
  ]
}

The latter part, "another resource can reference other resources," is relatively harder to understand. Here, we need to mention an important concept in AE: Composition. We can understand a composition as a reusable collection of layers. A composition can reference an image resource or another composition. In the following code, layers references the composition with id "2", and this composition has a layer that references the composition with id "3":

{
  "assets": [
    {
      "id": "2",
      "layers": [
        {
          "refId": "0"
        },
        {
          "refId": "1",
        },
        {
          "refId": "3",			// 引用 ”3“ 合成
        },
      ]
    },
    {
      "id": "3",
      "layers": [...]
    }
  ],
  "layers": [
    {
      "refId": "2",					// 引用 ”2“ 合成
    }
  ]
}

At this point, readers may have guessed: Layers expand into a tree structure.

Layer Properties

After understanding the basic data structure, let's understand the properties of each layer:

{
  "ind": 1,				// index：决定绘制顺序，类似 css 里的 z-index
  "ty": 2,				// type：图层类型
  "nm": "1",			// name：图层名称
  "refId": "0",		// referenced asset id：引用的资源
  "sr": 1,				// stretch：时间缩放
  "ks": {},				// keyframes：关键帧变换数据（下面详解）
  "ao": 0,				// auto orient：图层是否沿着运动路径切线方向旋转
  "ip": 407,			// in point：第一帧位置；
  "op": 437,			// out point：最后一帧率位置；
  "st": -72,			// start time：开始时间
}

Among the layer properties, ind determines the drawing order, which is very important because the Z value of 2D elements in 3D space is likely to be the same, and the value of ind is needed for sorting.

Additionally, ty represents the layer type. Lottie layers are categorized into types such as image layers (image), shape layers (shape), solid layers (solid), pre-comp layers (preComp), and text layers (text). The image layer, which uses a bitmap as the texture source, is the most commonly used layer and can meet over 90% of scenarios based on our business practice. Therefore, although the Galacean engine currently only supports image layers, it is sufficient. Implementing shape layers requires real-time triangulation (Delaunay) of vector graphics, which is not very performant, so it has not been implemented yet. In fact, the best practice in Lottie web also suggests minimizing the use of vector and mask operations that are performance-intensive.

So, with the Galacean engine only supporting rendering image layers, we only need to focus on two types:

• Value 2: Image layer;
• Value 0: Composition;

Next, let's talk about the most critical attribute ks keyframe data.

Keyframe Data

For a layer to animate, it needs to perform some transformations (Transform) in real-time, including translation, rotation, scaling, and opacity changes. Therefore, ks includes the following attributes:

"ks": {
  "o": {},	// opacity 透明度
  "r": {},	// rotation 旋转
  "p": {},	// position 位移
  "a": {},	// anchor 锚点
  "s": {}		// scale 缩放
},

A small portion of these attribute values are fixed values, such as opacity being 0.5 or position being [0, 100, 0], while most are keyframe data with the following structure:

{
  "i": {							// input 贝塞尔入点
    "x": [ 0.667 ],
    "y": [ 1 ]
  },
  "o": {							// output 贝塞尔出点
    "x": [ 0.333 ],
    "y": [ 0 ]
  },
  "t": 18.229,				// time 关键帧时间点
  "s": [ 100 ],				// startValue 开始值
  "e": [ 0 ]					// endValue 结束值
}

The above data means that at the time point 18.229, an interpolation of a Bezier curve with a start value of 100 and an end value of 0 is performed, where the Bezier curve's in-point is (0.667, 1) and the out-point is (0.333, 0).

Summary

Lottie's data structure is a tree structure, where each node on the tree is a layer, and each layer has some transformation attributes. The changes in these attributes are calculated through Bezier curve interpolation.

Program Design and Implementation

Next, to interface with the Galacean engine, we can design the program into three main parts:

• A class for loading and managing Lottie animations (red part);
• A class for interfacing with the engine's Lottie layer elements (blue part);
• A class for calculating Lottie layer attributes (green part);

Loading and Parsing Protocol

Since the Galacean Engine is a component-based graphics engine, Lottie is integrated as an ecosystem capability in the form of a component. The API design is as follows:

// 添加 Lottie 动画组件
const lottie = lottieEntity.addComponent(LottieAnimation);
 
// 设置 lottie 数据源
lottie.source = ...;
 
// 播放动画
lottie.play();

The Galacean Engine provides a Script script component base class to extend custom functionality. Therefore, we can extend it to implement the Lottie animation component class:

import { Script } from "@galacean/engine";
 
class LottieAnimation extends Script {
  set source (value: LottieResource) {
  }
  
  play() {
  }
  
  pause() {
  }
}

The Lottie component has only three APIs: play to play, pause to pause, and to set the data source. First, let's look at the design of this data source class LottieResource.

The Galacean Engine's resource memory management uses reference counting. New engine resources implement resource GC by extending the EngineObject class:

import { Engine, EngineObject } from "@galacean/engine";
 
class LottieResource extends EngineObject {
  constructor(engine: Engine, res, atlas?) {
  }
  
  private _buildTree(layers, compsMap) {
    const layersMap = {};
 
		for (let i = 0, l = layers.length; i < l; i++) {
			const layer = layers[i];
			layersMap[layer.ind] = layer;
		}
    
    for (let i = 0, l = layers.length; i < l; i++) {
			const layer = layers[i];
			const { refId, parent } = layer;
 
			if (parent) {
				if (!layersMap[parent].layers) {
					layersMap[parent].layers = [];
				}
 
				layersMap[parent].layers.push(layer);
			}
 
			if (refId && compsMap[refId]) {
        // 克隆合成
				layer.layers = [...compsMap[refId].layers];
        
				// 递归解析和构建树结构
				this._buildTree(layer.layers, compsMap);
			}
		}
 
  }
}

The main function of this class is to convert the Lottie JSON protocol into a tree structure. As mentioned in the "Data Structure" section above, due to the existence of "composition," it needs to recursively parse and build the tree structure. Note that since compositions are reusable layer collections, they must be cloned before being attached to a tree node.

To unify the engine's resource loading interface, we actually encapsulate the process of adding the LottieAnimation component into the LottieLoader class, so the user experience is as follows:

import { LottieAnimation } from "@galacean/engine-lottie";
 
engine.resourceManager.load({
  urls: [
   // ...
  ],
  type: 'lottie'
}).then((lottieEntity) => {
  root.addChild(lottieEntity);
  const lottie = lottieEntity.getComponent(LottieAnimation);
  lottie.play();
});

Creating Layer Elements

With this LottieResource resource, our next task is to recursively create Lottie layer elements and update each layer's attributes and render them in the LottieAnimation's onUpdate function.

import { Script } from "@galacean/engine";
 
class LottieAnimation extends Script {
  private _createElements(value: LottieResource) {
    // create elements
  }
  
  onUpdate () {
  	// update elements
  }
}

Since only image layers are supported at the moment, we only need to create two types of elements: CompLottieElement and SpriteLottieElement:

switch (layer.ty) {
  case 0:
    element = new CompLottieElement(layer, this.engine, childEntity, layer.id);
    break;
 
  case 2:
    element = new SpriteLottieElement(layer, this._resource.atlas, this.entity, childEntity);
    break;

Among them, CompLottieElement is an empty Entity that functions as a parent node and only contains transformation components; SpriteLottieElement is an Entity that contains a SpriteRenderer component:

class SpriteLottieElement extends BaseLottieElement {
  sprite: Sprite;
  spriteRenderer: SpriteRenderer;
 
  constructor(layer, atlas: SpriteAtlas, entity: Entity) {
    super(layer);
    this.sprite = atlas.getSprite(layer.refId);
    const { atlasRegion, texture } = this.sprite;
 
    const spriteEntity = new Entity(entity.engine, layer.nm);
    const spriteRenderer = spriteEntity.addComponent(SpriteRenderer);
 
    spriteRenderer.sprite = this.sprite;
  }
}
 
### Updating Layer Elements
Animation playback has some conventional logic, so we designed the following APIs:
 
- isLooping: Whether to loop playback
- repeats: Number of playback repetitions
- isAlternate: Whether to alternate playback
- speed: Playback speed
- direction: Playback direction
 
We implement these APIs in the `onUpdate` function of `LottieAnimation` (this function is inherited from the engine's `Script` base class), which essentially calculates the value of `this._frame` based on different conditions:
```typescript
onUpdate(deltaTime: number): void {
  const time = this.direction * this.speed * deltaTime;
 
  // 累计帧数
  this._frame += time / this._resource.timePerFrame;
 
  // 超出结束帧
  if (this._spill()) {
    const { duration } = this._resource;
    this._resetElements();
 
    // 重复播放或循环播放
    if (this.repeats > 0 || this.isLooping) {
      if (this.repeats > 0) {
        --this.repeats;
      }
 
      // 交替播放
      if (this.isAlternate) {
        this.direction *= -1;
        this._frame = Tools.codomainBounce(this._frame, 0, duration);
      }
      else {
        this.direction = 1;
        this._frame = Tools.euclideanModulo(this._frame, duration);
      }
    }
    else {
      this._frame = Tools.clamp(this._frame, 0, duration);
    }
  }
 
  // 更新每个图层元素
  this._updateElements(this._resource.inPoint + this._frame);
}

Next, update each element, i.e., update the values of a series of transformation properties of the element. First, we design a TransformFrames class to manage all the properties, which correspond one-to-one with the properties in ks in the Lottie protocol:

class TransformFrames {
  p;			// 位置
  x;			// 位置 x 分量
  y;			// 位置 y 分量
  z;			// 位置 z 分量
  a;			// 锚点
  s;			// 缩放	
  r;			// 2d 旋转
  or;			// 3d 旋转
  rx;			// 3d 旋转 x 轴分量
  ry;			// 3d 旋转 y 轴分量
  rz;			// 3d 旋转 z 轴分量
  o;			// 透明度
}

Since some transformation properties in AE can be set as a whole or designed individually, the properties exported by Lottie may be either whole or component-wise. For example, p is a collection of [x, y, z], and or is a collection of [rx, ry, rz]. The keyframe data for these properties are divided into four categories:

• ValueProperty: A constant of a single number;

• MultiDimensionalProperty: A vector constant;

• KeyframedValueProperty: Keyframe data of a single number;

• KeyframedMultidimensionalProperty: Keyframe data of a vector;

The first two types of properties are constants and do not require computation; the latter two types with the Keyframed prefix require interpolation calculations. If it is a normal Bezier curve, the interpolation calculation is relatively simple:

// 生成贝塞尔插值函数
const bezier = generateBezierEasing(o.x, o.y, i.x, i.y);
 
// 通过传入归一化的时间获得百分比
const percent = bezier((frameName - t)/(next_t - t));
 
// 在起始值和结束值之间获得当前百分比的值
const value = s + (e - s) * percent；

A cubic Bezier curve can be defined by two control points, and generateBezierEasing can be implemented using the bezier-easing library. Note that generating the curve is inefficient, so it should be cached for reuse after being generated once.

At this point, we consider updating properties to be relatively simple, but if you closely observe the KeyframedMultidimensionalProperty type of properties, you will find that it may have tangent properties ti and to. When these two properties exist, bezier-easing does not provide a ready-made method to generate the curve, so we have to use a more brute-force approach, i.e., sampling points on the curve:

// curveSegments 是曲线要分割的段数，k 表示第几段
const percent: number = k / (curveSegments - 1);
 
// 计算点的坐标值
const value = Math.pow(1 - percent, 3) * s +
      3 * Math.pow(1 - percent, 2) * percent * (s + to) +
      3 * (1 - percent) * Math.pow(percent, 2) * (e + ti) +
      Math.pow(percent, 3) * e;

We can use the Pythagorean theorem to calculate the length between two points and obtain the total length of the curve by accumulating the lengths of each segment:

partialLength += Math.pow(point.x - lastPoint.x, 2);
partialLength += Math.pow(point.y - lastPoint.y, 2);
partialLength = Math.sqrt(ptDistance);
 
segmentLength += partialLength;

When updating properties, we first get the length of the curve occupied by the current time, then infer between which two points on the curve the current point is located and perform linear interpolation:

const distanceInLine = segmentLength * percent;
 
for (let i = lastPoint, l = points.length; i < l; i++) {
  const point = points[i];
  const nextPoint = points[i + 1];
  const segmentPercent = (distanceInLine - addedLength) / partialLength;
  const value = point + (nextPoint - point) * segmentPercent;
  
  lastPoint = i;
}

Summary

Based on understanding the Lottie protocol and familiarizing ourselves with the Galacean engine API design, we designed the Lottie resource as LottieResource to enable garbage collection of engine resources. We also implemented a unified resource loading API through LottieLoader and designed the LottieAnimation component API for users. The update implementation of the transformation properties of layer elements is challenging, and using Bezier curves for interpolation is also the essence of Lottie animation.

Some Additional Capabilities

In addition to the above functionalities, considering performance and ease of use, we have also implemented some new capabilities.

Generating Atlas

Galacean Engine is a WebGL-based graphics engine, so it naturally has the ability to batch render. Therefore, we wrote a small tool (https://www.npmjs.com/package/@galacean/tools-atlas-lottie) that can merge image resources in the Lottie protocol into an atlas. This way, rendering a Lottie component only consumes one Drawcall. The left side of the image below shows the rendering effect, and the right side shows the merged atlas used.

If you use atlas resources, you only need to pass the atlas link in the second item of the urls field when passing parameters:

engine.resourceManager.load({
  urls: [
    "xxx.json",
    "xxx.atlas"	// 使用了图集则需要传入 atlas 链接，如果不使用
  ],
  type: 'lottie'
});

Animation Slicing

We often encounter the need for multiple animation segments in business scenarios. At this time, we do not want designers to provide us with multiple Lottie animations, which would increase the application size and memory usage. Therefore, the front end will choose to slice the animation, that is, play from one frame to another frame.

First, we add a private field lolitaAnimations in the Lottie JSON to define which segments are available:

"lolitaAnimations": [
  {
    "name": "beforePlay",
    "start": 0,
    "end": 71
  },
  {
    "name": "onPlay",
    "start": 72,
    "end": 143
  },
  {
    "name": "afterPlay",
    "start": 144,
    "end": 203
  }
]

Then we check in the onUpdate function of LottieAnimation whether to play the full animation or a segment animation, for example:

if (clip) {
  this._frame = Tools.euclideanModulo(this._frame, clip.end - clip.start);
}
else {
  this._frame = Tools.euclideanModulo(this._frame, duration);
}

In this way, we can easily handle the need to play animation segments.

Asynchronous Syntax

We also often encounter the need to listen for the end of an animation. For this, we designed some syntactic sugar to make the program look more elegant:

  const lottie = lottieEntity.getComponent(LottieAnimation);
  await lottie.play();

This is relatively simple. Design the play method as an asynchronous function that returns a Promise, and then execute the resolve method of the Promise when the last frame of the onUpdate function finishes playing. No further details are provided.

References

• body-movin JSON illustration: https://www.processon.com/view/link/5c2ece6ae4b08a768398b06d
• bezier-easing: https://github.com/gre/bezier-easing
• Elegantly control Lottie playback with async/await: https://zhuanlan.zhihu.com/p/101109715