Controlling the camera is a fundamental and common requirement in 3D scenes, such as orbiting around objects for showcase projects or moving the camera in a first-person view for game projects. Common camera controls include orbitControl, which moves the camera around a scene anchor point, and freeControl, which rotates the camera around its own anchor point. In an editor, in addition to the above two controls that manipulate the camera through keyboard and mouse, a visual camera control - navigation control is essential. As a fixed way of viewing navigation, it can (1) intuitively display the direction of the camera in the scene, (2) easily switch to fixed views, such as top view, side view, and other three views, and (3) operate the camera with a single key click, greatly reducing the usability threshold for ordinary users. In the following text, we will break down the three main functions of the navigation control.

Multiple Cameras

The first step in implementation is to consider how to display the navigation control on the screen. According to the concept, the navigation control and the main screen belong to the same scene, with synchronized camera information and independent operation. Similar to a minimap or equipment overview in games. It is generally implemented using the concept of multiple cameras. That is, adding another camera in the scene, placing the navigation control and the culling mask of this camera (used to selectively render rendering components in the scene) on the same layer.

Synchronize Camera Direction

In the concept, the navigation control should be able to synchronize the direction of the scene camera and control the camera.

The first step is how to synchronize the direction of the scene camera? It is natural to think of assigning the camera's rotationQuaternion to the navigation control. However, after implementing this, it will be found to be completely different from the expectation. Let's rethink, what exactly do the three axes of the navigation control display? Imagine a coordinate axis representing XYZ directions at the origin of the scene. Moving the scene camera allows you to observe the relative changes of this coordinate axis, which is what the conventional navigation control should display, rather than the direction of the scene camera itself.

Therefore, the worldMatrix of the scene camera and the navigation control should be inversely related. The scene camera, due to its camera properties, has a viewMatrix of its own, which is inversely related to the worldMatrix of the scene camera entity, i.e., the same as the worldMatrix of the control.

The position of the navigation control is fixed and does not change. In matrix language, this means that the last column representing position changes in homogeneous coordinates, $t_x, t_y, t_z$, can be set to zero.

tempMat.copyFrom(sceneCamera.viewMatrix);
const { elements: ele } = tempMat;
// ignore translate
ele[12] = ele[13] = ele[14] = 0;
gizmoEntity.transform.worldMatrix = tempMat;

Switching to Three Views

Another main function of the navigation control is to switch the scene camera to the corresponding three-view perspective. It is worth noting that, by definition, a three-view perspective does not only involve rotating the camera itself to look up or sideways, but also moving around the object to view it from different angles. Imagine a sphere centered on the object of observation; what we need is to control the scene camera to move on this sphere to the specified position and orientation.

So how do we obtain the worldMatrix of the scene camera corresponding to a specific three-view perspective? Generally, a matrix has a lookAt method, and the obtained lookAtMatrix is the camera's viewMatrix:

Matrix.lookAt(eye, target, upVec, viewMat);

How do we obtain the various components target, eye, upVec?

• target is the object being looked at. In the scene, what factors will change the target of the scene camera? Besides actively changing it, we also need to consider whether the scene camera has the orbitControl component. The orbitControl has three operations: orbit, zoom, and pan. While orbit and zoom change the camera position, they both revolve around the same center. On the other hand, pan will change the target of lookAt while moving the camera position. Therefore, navigation controls need to consider two cases: whether the same camera has the orbitControl component or not. If it has orbitControl, then call its target; if not, the user needs to specify the target.

• eye is the desired position of the scene camera. After determining the specific orientation of the three views, it is calculated by combining target, forwardVector, and radius.

• upVector is generally (0, 1, 0). It is recommended to use (0, 0, 1) or (0, 0, -1) for top and bottom views.

// target
// 场景相机有 orbitControl 控件
const target = orbitControl.target 
// 场景相机无 orbitControl 控件, 指定位置
const target = new Vector3(...);
 
// eye
Vector3.subtract(sceneCameraEntity.transform.worldPosition, target, tempVect);
const radius = tempVect.length();
// 根据需要三视图获得方向，以右视图为例
const directVec = new Vector3(0,radius,0)
Vector3.add(directVec, target, eye);
 
// up direction
const upVec = new Vector3(0,1,0)
 
// 获得场景相机实体worldMatrix
Matrix.lookAt(eye, target, upVec, tempMat);
 
tempMat.invert()
sceneCameraEntity.transform.worldMatrix = tempMat;

Drag Camera Control

We change the orientation of the scene camera by dragging the mouse on the screen. Mouse movement is two-dimensional, while the camera changes in three-dimensional space. How can we control a three-dimensional scene through two-dimensional interaction? Imagine the camera moving on a spherical surface. One approach is to drag the surface of the sphere to change the camera orientation. This approach is intuitive but has clear drawbacks. Since it operates in a two-dimensional screen space, it is difficult to rotate the sphere more than 180 degrees to reach the back side. Additionally, it involves complex sphere calculations.

Our approach is to decompose dragging into two directions: horizontal and vertical. The difference in the x-direction of screen space corresponds to the angle rotated around the vertical axis, while the difference in the y-direction corresponds to the angle rotated around the horizontal axis. In world space, the vertical axis is the y-axis direction, and the horizontal axis is parallel to the scene camera. This correspondence achieves the effect of dragging the camera and allows for controlling the rotation speed by increasing the coefficient.

// 分解为屏幕空间中x、y方向上每帧移动差值
let x = deltaPointer.x 
let y = deltaPointer.y 
 
// 计算得出横轴
horizontalAxis.copyFrom(sceneCameraEntity.transform.worldForward);
Vector3.cross(horizontalAxis, upVec, horizontalAxis);
 
// 获得对应旋转角度
Quaternion.rotationAxisAngle(horizontalAxis, y, tempQuat);
Quaternion.rotationYawPitchRoll(x, 0, 0, tempQuat2);
Quaternion.multiply(tempQuat, tempQuat2, tempQuat);

Summary

In business scenarios, this navigation component has a high degree of visualization and no usage threshold, making it a user-operated axis to control the scene. In implementation, it can provide inspiration for other components that need to be displayed on the main screen, have synchronized information but independent operations. In terms of calculations, it covers the most commonly used properties of the camera such as lookAt Matrix, viewMatrix, and the special properties of the camera entity, helping engine users better understand the camera.

Source Code: https://github.com/galacean/engine-toolkit
Navigation Example: https://galacean.antgroup.com/#/examples/latest/gizmo
Multiple Camera Example: https://galacean.antgroup.com/#/examples/latest/multi-camera