.. meta:: :description: Kdenlive Video Effects - Motion Tracker :keywords: KDE, Kdenlive, video editor, help, learn, easy, effects, filter, video effects, motion, tracking, motion tracker .. metadata-placeholder :authors: - frdbr (https://userbase.kde.org/User:frdbr) - Bernd Jordan (https://discuss.kde.org/u/berndmj) - Eugen Mohr :license: Creative Commons License SA 4.0 .. |OpenCV| raw:: html OpenCV (Open Source Computer Vision Library) .. |DaSiamRPN| raw:: html DaSiamRPN .. |more_info| raw:: html more info Motion Tracker ============== .. .. versionadded:: 19.04.0 .. figure:: /images/effects_and_compositions/kdenlive2304_effects-motion_tracker.webp :width: 365px :figwidth: 365px :align: left :alt: kdenlive2304_effects-motion_tracker .. sidebar:: |kdenlive-show-video| Motion Tracker :**Status**: Maintained :**Keyframes**: Yes :**Source library**: opencv :**Source filter**: tracker :**Available**: |linux| |appimage| |windows| |apple| :**On Master only**: No :**Known bugs**: No .. rst-class:: clear-both .. rubric:: Description Motion tracking is the process of locating a moving object across time. Kdenlive uses |OpenCV|\ [1]_ for motion detection. The results of this effect can be used in other effects by copying the keyframe data generated by the Motion Tracker as position keyframes in the Transform effect, for example. .. rubric:: Parameters .. list-table:: :header-rows: 1 :width: 100% :widths: 25 10 65 :class: table-wrap * - Parameter - Value - Description * - Tracker algorithm - Selection - Sets the algorithm used for motion tracking * - Keyframes spacing - Integer - Determines how many keyframes can be skipped when analyzing * - Frame shape - Selection - Set the shape of the frame * - Shape width - Integer - Set the thickness of the frame shape * - Shape color - Picker - Set the color of the frame shape. Also determines the color for :guilabel:`Blur type` **Opaque Fill**. * - Blur - Integer - Set the amount of blur for :guilabel:`Blur type` **Median Blur** and **Gaussian Blur** * - Blur type - Selection - Select what to do with the framed section The following selection items are available: :guilabel:`Tracker algorithm` .. list-table:: :width: 100% :widths: 20 80 :class: table-wrap * - KCF - Kernelized Correlation Filters (default) * - CSRT - Channel and Spatial Reliability Tracking * - MOSSE - Minimum Output Sum of Squared Error * - MIL - Multiple Instance Learning * - MedianFlow - * - DaSiam - The |DaSiamRPN| visual tracking algorithm relies on deep-learning models to provide extremely accurate results. Please see note below for installation instructions. * - Nano - Nano tracker is a lightweight model and gives good results and is fast. .. tip:: You may need to experiment with different tracking algorithms to produce good results for your specific use case. See a short comparison of the different `tracking algorithms`_ below. :guilabel:`Frame Shape` .. list-table:: :width: 100% :widths: 20 80 :class: table-wrap * - Rectangle - Default * - Ellipse - * - Arrow - .. tip:: Selecting the right shape type can make the motion tracking better. :guilabel:`Blur Type` .. list-table:: :width: 100% :widths: 20 80 :class: table-wrap * - None - Do nothing (default) * - Median Blur - Apply median blur to rectangle * - Gaussian Blur - Apply Gaussian blur to rectangle * - Pixelate - Pixelate rectangle * - Opaque fill - Fill rectangle with shape color Examples for :guilabel:`Blur Type`: .. figure:: /images/effects_and_compositions/kdenlive_effects-motion_tracker_blur_type.gif :width: 90% :alt: kdenlive_effects-motion_tracker_blur_type Different blur types in action How to Track a Region of a Video -------------------------------- The basic workflow for tracking a region is as follows: .. container:: .. figure:: /images/effects_and_compositions/kdenlive_effects-motion_tracking_face.webp :align: left :width: 350px :alt: kdenlive_effects-motion_tracking_face Tracking the face of the model * Apply the effect to a clip * Select the desired region\ [2]_ to track on the Project Monitor * Choose a tracking algorithm * Click on the :guilabel:`Analyze` button .. container:: clear-both .. figure:: /images/effects_and_compositions/kdenlive2304_effects-motion_tracker_copy_kf.webp :align: left :width: 350px :alt: kdenlive2304_effects-motion_tracker_copy_kf Options menu * When the analysis is done you can export the keyframes to the clipboard by clicking on |application-menu| and choose :guilabel:`Copy all keyframes to clipboard`. See also :ref:`Exchanging keyframes `. .. rst-class:: clear-both _`Tracking algorithms` ---------------------- :KCF: **Kernelized Correlation Filters** **Pros:** Accuracy and speed are both better than MIL and it reports tracking failure better than MIL. **Cons:** Does not recover from full occlusion. :CSRT: **Channel and Spatial Reliability Tracking**. In the Discriminative Correlation Filter with Channel and Spatial Reliability (DCF-CSR), we use the spatial reliability map for adjusting the filter support to the part of the selected region from the frame for tracking. This ensures enlarging and localization of the selected region and improved tracking of the non-rectangular regions or objects. It uses only 2 standard features (HoGs and Colornames). It also operates at a comparatively lower fps (25 fps) but gives higher accuracy for object tracking. :MOSSE: **Minimum Output Sum of Squared Error** MOSSE uses an adaptive correlation for object tracking which produces stable correlation filters when initialized using a single frame. MOSSE tracker is robust to variations in lighting, scale, pose, and non-rigid deformations. It also detects occlusion based upon the peak-to-sidelobe ratio, which enables the tracker to pause and resume where it left off when the object reappears. MOSSE tracker also operates at a higher fps (450 fps and even more). **Pros:** It is as accurate as other complex trackers and much faster. **Cons:** On a performance scale, it lags behind the deep learning based trackers. :MIL: **Multiple Instance Learning** **Pros:** The performance is pretty good. It does a reasonable job under partial occlusion. **Cons:** Tracking failure is not reported reliably. Does not recover from full occlusion. :MedianFlow: **Pros:** Excellent tracking failure reporting. Works very well when the motion is predictable and there is no occlusion. **Cons:** Fails under large motion. :DaSiam: The |DaSiamRPN| visual tracking algorithm relies on deep-learning models to provide extremely accurate results. In order to use the DaSiam algorithm you need to download the AI models :download:`dasiamrpn_kernel_cls1.onnx` :download:`dasiamrpn_kernel_r1.onnx` :download:`dasiamrpn_model.onnx` and place them in `folder for models`_ :Nano: Nano tracker is a lightweight model and gives good results and is fast. In order to use the Nano algorithm you need to download the AI models (model size about 1.9 MB) :download:`nanotrack_backbone_sim.onnx` :download:`nanotrack_head_sim.onnx` and place them in the `folder for models`_ .. .. _folders_for_models: _`Folder for models` ^^^^^^^^^^^^^^^^^^^^ :Linux: :file:`$HOME/.local/share/kdenlive/opencvmodels` :Flatpak: :file:`$HOME/.var/app/org.kde.kdenlive/data/kdenlive/opencvmodels` :Windows: :file:`%AppData%/kdenlive/opencvmodels` Press :kbd:`Win+R` (:kbd:`Windows` key and :kbd:`R` key simultaneously) and copy **%AppData%/kdenlive/**. Then create the folder `opencvmodels` ---- .. [1] If you want to build Kdenlive yourself you need to build MLT with OpenCV support. See here for |more_info|. .. [2] If you see just a red rectangle in the Project Monitor but cannot move or size it, enable Edit Mode by clicking on the |edit-mode| icon in the Project Monitor toolbar .. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Icons used here (remove comment indent to enable them for this document) .. |application-menu| image:: /images/icons/application-menu.svg :width: 22px :class: no-scaled-link