While I used different Instagram filters, I had the idea of ​​creating my own small play filter, simulating the screen cleaning. To use the knowledge of computer vision I have. In this article, I would like to share how I created my own filter, what I learned when creating it, and I would also like to share my discoveries.

The filter emulates the cleaning of dirt on the screen of the screen. To do this, I used Mediapipe, an open source cross-boiler for processing data from various modalities, such as audio and video.

Mediapipe's hand tracking

My approach ensures the tracking of the hands and fingers to determine the 21 key point in 3 dimensions. The Mediapipe method provides high -precision tracking of hands and fingers in real time. This solution uses a machine learning conveyor consisting of the following models working in parallel:

Now that we have a general idea of ​​the model, let's deal with the implementation of this filter.

For this implementation, I used Mediapipe and OpenCV packages. Please install these packages using the following commands.

In the code, we first import these packages using:

Now we will define a class that implements the basic parameters necessary for detecting a hand and appropriate key points.

The Handsfinder function will track the hands in the input frames. If the True value is set for the DRAW, the hands orientations will also be applied to the image. The Positionfinder function finds the coordinates of X and Y of all 21 points that I will use in this project.

Now let's create the main function that uses OpenCV to capture the video, after which we have a mask containing dirt over our initial image of the frame. Having read the frame behind the video from the video from the camera, we find hands using the Handtracker function and get the coordinates of the position of the hands orientations. As soon as we get the position of landmarks, we choose 2 diagonal key points on the hand so that they form a rectangle similar to fabric in our hand, which cleanses the dirt from the image mask. Here we set the values ​​in this rectangle as (255, 255, 255) to imitate the behavior of the screen cleaning. Now I used the balanced image of the OpenCV (CV2.Addweighted) to create a disguised image output (i.e., weighing the mask in the image with alpha = 0.5) for each input frame.

Now just call the Main function.

While I was implementing this project, I tried to use circles (instead of rectangles) ineffectively without masks of images, which reduced performance. I also tried to use the lines that led to artifacts. Therefore, I tried to use masks along with the OpenCV rectangle.

I think that this implementation is simple for any beginner who is trying to start working with computer vision. As soon as we take possession of the CV base, we will be able to immerse ourselves deeply into something more complicated. The presented packages have many opportunities for the implementation of very interesting projects.

#machinelearning #artificialintelligence #ai #datascience #programming #Technology #Deeplearning