Dynamic Font Customization Using Real-Time Hand Gesture Recognition with ml5.js and Teachable Machine.

Published by Kushtrim Hamzaj
July 10, 2024

HandPose is a machine-learning model from ml5.js that detects hand positions and landmarks in real-time using a webcam. Built on TensorFlow.js, it identifies 21 key points on a hand, including the fingertips and joints. Additionally, ml5.js enables the creation and training of custom machine learning models directly in the browser using its Neural Network capabilities. In this experiment, we trained the model to recognize specific hand gestures, which then dynamically update the variable font axes and interpolations.

Initialization and Setup

The application begins by initializing several global variables to manage video capture, hand pose detection, font settings, and user interface (UI) elements. Key variables include video for the webcam feed, handpose for the hand pose model, and various UI components such as buttons, sliders, and containers. This setup phase ensures that all necessary elements are ready for the application’s core functionalities.

The full Article is coming soon.