Motion Tracker - 运动跟踪器

kdenlive2304_effects-motion_tracker

Description

Motion tracking is the process of locating a moving object across time. Kdenlive uses OpenCV (Open Source Computer Vision Library)[1] for motion detection. The results of this effect can be used in other effects by copying the keyframe data generated by the Motion Tracker as position keyframes in the Transform effect, for example.

Parameters

Parameter

Value

Description

Tracker algorithm

Selection

Sets the algorithm used for motion tracking

Keyframes spacing

Integer

Determines how many keyframes can be skipped when analyzing

帧形状

Selection

Set the shape of the frame

Shape width

Integer

Set the thickness of the frame shape

形状颜色

Picker

Set the color of the frame shape. Also determines the color for Blur type Opaque Fill.

Blur

Integer

Set the amount of blur for Blur type Median Blur and Gaussian Blur

模糊类型

Selection

Select what to do with the framed section

The following selection items are available:

Tracker algorithm

KCF

Kernelized Correlation Filters (default)

CSRT

Channel and Spatial Reliability Tracking

MOSSE

Minimum Output Sum of Squared Error

MIL

Multiple Instance Learning

MedianFlow

DaSiam

The DaSiamRPN visual tracking algorithm relies on deep-learning models to provide extremely accurate results. Please see note below for installation instructions.

Nano

Nano tracker is a lightweight model and gives good results and is fast.

小技巧

You may need to experiment with different tracking algorithms to produce good results for your specific use case. See a short comparison of the different tracking algorithms below.

Frame Shape

Rectangle

Default

Ellipse

Arrow

小技巧

Selecting the right shape type can make the motion tracking better.

Blur Type

None

Do nothing (default)

Median Blur

Apply median blur to rectangle

Gaussian Blur

Apply Gaussian blur to rectangle

Pixelate

Pixelate rectangle

Opaque fill

Fill rectangle with shape color

Examples for Blur Type:

kdenlive_effects-motion_tracker_blur_type

Different blur types in action

How to Track a Region of a Video

The basic workflow for tracking a region is as follows:

kdenlive_effects-motion_tracking_face

Tracking the face of the model

  • Apply the effect to a clip

  • Select the desired region[2] to track on the Project Monitor

  • Choose a tracking algorithm

  • Click on the Analyze button

kdenlive2304_effects-motion_tracker_copy_kf

Options menu

  • When the analysis is done you can export the keyframes to the clipboard by clicking on application-menu and choose Copy all keyframes to clipboard. See also Exchanging keyframes.

Tracking algorithms

KCF:

Kernelized Correlation Filters，目标跟踪算法

优点： 精度和速度都优于 MIL，它对于跟踪过程出现故障的报告也优于 MIL。

缺点： 在物体的运动过程中，如果某一段时间内物体被完全遮挡，KCF 无法继续跟踪。

CSRT:

Channel and Spatial Reliability Tracking.

In the Discriminative Correlation Filter with Channel and Spatial Reliability (DCF-CSR), we use the spatial reliability map for adjusting the filter support to the part of the selected region from the frame for tracking. This ensures enlarging and localization of the selected region and improved tracking of the non-rectangular regions or objects. It uses only 2 standard features (HoGs and Colornames). It also operates at a comparatively lower fps (25 fps) but gives higher accuracy for object tracking.

MOSSE:

Minimum Output Sum of Squared Error

MOSSE 在使用单帧初始化时，会使用自适应关联关系，以产生稳定相关过滤器的对象跟踪。 MOSSE 跟踪器能够适应照明、尺寸、姿势和非硬性变形等变化。 MOSSE 还能够检测到基于峰对侧边比率的关联，这使得它能够在物体被遮挡时暂停，并在物体重新出现时恢复跟踪。MOSSE 跟踪器还能够以更高的帧率运行（450 fps 或更高）。

优点： 它与其他复杂的跟踪器一样精确，速度也更快。

缺点： 从性能的角度上来说，它落后于基于深度学习的跟踪算法。

MIL:

Multiple Instance Learning

Pros: 性能很好。它在物体被部分遮挡时依然能够得出不错的跟踪结果。

缺点： 无法可靠地报告跟踪失败；在物体的运动过程中，如果某一段时间内物体被完全遮挡，MIL 无法继续跟踪。

MedianFlow:

优点： 非常好的跟踪故障报告。当物体的运动轨迹可以预测并且没有中断时，非常有效。

缺点： 当物体运动幅度较大时可能出现跟踪失败。

DaSiam:

The DaSiamRPN visual tracking algorithm relies on deep-learning models to provide extremely accurate results.

In order to use the DaSiam algorithm you need to download the AI models

dasiamrpn_kernel_cls1.onnx

dasiamrpn_kernel_r1.onnx

dasiamrpn_model.onnx

and place them in folder for models

Nano:

Nano tracker is a lightweight model and gives good results and is fast.

In order to use the Nano algorithm you need to download the AI models (model size about 1.9 MB)

nanotrack_backbone_sim.onnx

nanotrack_head_sim.onnx

and place them in the folder for models

Folder for models

Linux:

$HOME/.local/share/kdenlive/opencvmodels

Flatpak 版本:

$HOME/.var/app/org.kde.kdenlive/data/kdenlive/opencvmodels

Windows:

%AppData%/kdenlive/opencvmodels

Press Win+R (Windows key and R key simultaneously) and copy %AppData%/kdenlive/. Then create the folder opencvmodels