Hand Gesture with Text

Abstract

Hand gesture recognition is an emerging area of human–computer interaction that enables natural and intuitive communication between humans and machines. Gestures play an important role in non-verbal communication and are especially useful for assisting speech-impaired and hearing-impaired individuals. Converting hand gestures into text helps bridge the communication gap and allows users to interact with digital systems without the need for physical input devices such as keyboards or microphones.

This project presents a hand gesture to text conversion system using computer vision and machine learning techniques. The system captures real-time hand gestures through a webcam and processes them using image processing techniques. MediaPipe is used for hand landmark detection, which extracts key points of the hand for accurate gesture representation. These extracted features are then classified using a trained machine learning model to recognize predefined hand gestures. Each recognized gesture is mapped to a corresponding text output.

The proposed system supports multiple static hand gestures and converts them into readable text displayed on the screen in real time. Python is used as the core programming language along with libraries such as OpenCV, MediaPipe, NumPy, and TensorFlow for image processing and model implementation. The system is designed to be user-friendly, efficient, and capable of performing gesture recognition with good accuracy under normal lighting conditions.

Experimental results show that the system successfully recognizes hand gestures and converts them into text with reliable performance. This project demonstrates the practical application of machine learning and computer vision in assistive technology and human–computer interaction. The system can be further enhanced by adding dynamic gestures, voice output, and multilingual text support, making it suitable for real-world communication and accessibility application.

Hand gesture recognition is an important area of human–computer interaction that enables users to communicate with computer systems in a natural and contactless manner. This project focuses on the development of a hand gesture to text conversion system using computer vision and machine learning techniques. The system captures real-time hand gestures through a webcam and processes the visual input using image processing methods. MediaPipe is used to detect and track hand landmarks, which represent the position and movement of fingers and the palm. These extracted features are used to train a machine learning.

Country : India

1 Sakshi Waghmare2 Payal Kamble3 Utkarsha Borse4 Deepika Pangundwale5 Prof. Nita Pawar

  1. Student, Ajeenkya D. Y. Patil School of Engineering, Computer Engineering Diploma, Charholi, Pune, India
  2. Student, Ajeenkya D. Y. Patil School of Engineering, Computer Engineering Diploma, Charholi, Pune, India
  3. Student, Ajeenkya D. Y. Patil School of Engineering, Computer Engineering Diploma, Charholi, Pune, India
  4. Student, Ajeenkya D. Y. Patil School of Engineering, Computer Engineering Diploma, Charholi, Pune, India
  5. Student, Ajeenkya D. Y. Patil School of Engineering, Computer Engineering Diploma, Charholi, Pune, India

IRJIET, Volume 10, Issue 1, January 2026 pp. 168-172

doi.org/10.47001/IRJIET/2026.101021

References

  1. Zhang, Z., et al. “MediaPipe Hands: On-device Real-time Hand Tracking.” arXiv preprint arXiv:2006.10214, 2020.
  2. Simonyan, K., & Zisserman, A. “Very Deep Convolutional Networks for Large-Scale Image Recognition.” arXiv preprint arXiv:1409.1556, 2014.
  3. Abadi, M., et al. “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.” Software available from tensorflow.org, 2015.
  4. OpenCV Library – Bradski, G. “The OpenCV Library.” Dr. Dobb’s Journal of Software Tools, 2000.
  5. Gupta, R., & Sharma, S. “Hand Gesture Recognition Using Convolutional Neural Networks.” International Journal of Computer Applications, vol. 175, no. 3, 2017, pp. 1–7.
  6. Chauhan, S., & Singh, A. “Real-Time Hand Gesture Recognition System for Human-Computer Interaction.” International Journal of Computer Science and Information Technologies, 2018.