As open source AI models emerge, many developers are incorporating them into the applications we use on Linux. For example, Kdenlive anticipates an interesting novelty, the erasure of background in videos.
Kdenlive is a non-linear video editor and pioneered the integration we mentioned. A couple of years ago it added an automatic subtitle generator.
This is good news, yesterday I was talking about the cloud services which are very useful but a nightmare in terms of privacy. That is why it is ideal to have open source programs that have the same features.
Before continuing, a clarification. We are talking about software in a very preliminary testing stage. so it may have glitches. Also, the functionality requires downloading an Artificial Intelligence model that will require disk space and system resources for processing. If you don't have a powerful graphics card, the procedure could take quite a while.
Kdenlive anticipates an interesting novelty
Lets start by the beginning.
SAM2 (Segment Anything Model 2) is an image segmentation model developed by Meta (Facebook and Instagram) developed for their Artificial Intelligence. It works under the "object segmentation" modality, which consists of isolating a specific object from the rest of an image.
The procedure consists of the following characteristics:
- Object identification: SAM2 is able to identify and “cut out” a wide variety of objects within an image, even if it has not seen them during its previous training.
- Multiple ways of selection of the Objects.
The object selection modes are:
- Clicking About the object: The title says it clearly..
- Drawing a box: By the contour of the object: Very useful for objects with irregular or complex shapes.
- Using text: This is a projected feature. It simply describes the object you want to remove. For example, “The plant in the window.”
- Precise masks: It is similar to the method for deleting objects in Gimp. You simply paint over the object you want to delete.
All this is a description of the model in general, it does not necessarily come implemented in Kdenlive.
How this feature will work
The procedure for deleting the background will be as follows:
- Add the clip with whom the project will be worked on.
- Select the area from the background you want to remove.
- Click on the button Mask.
- Select Setup to set the tool settings. This is only done once.
- Click on Install in the object detection settings.
The developers warn that this will take a long time (5-15 minutes) and that you cannot see the progress of the installation yet. Unless otherwise indicated, the program will install the smaller model.
Once the model is downloaded, we do the following:
- Click en Create new mask.
- Click on the object you want to keep. Wait for the white mask to appear.
- Click in Generate Mask.
This may take a while.
Once the video mask has been created, we can see it in the mask manager dialog window. Then we just have to drag the clip area from the clip monitor to the timeline and select the newly created mask. To finish, click on Apply Mask.
This feature can also be used to apply effects or change color in a specific area.
Before giving you the download links, I repeat the clarification that this is a very preliminary program. In fact, the Windows version will require manual installation permissions. You will also have to notify the antivirus that this is not malicious software.
I haven't had the chance to try it yet, but it's certainly a great step forward. Until now, the only way to do it was using the OpenCV library, which required programming knowledge. In any case, we'll have to wait until April 2025 for a stable version.