Adaptive multimodal communication assist system.
The invention is a system that permits visually and/ or hearing impaired persons to communicate with each other and with people who have normal sight and hearing. It accepts input in one of the multiple modes: audio (speech), video (gestures) or text and relays the output in any one or more of the three modes. If the input and output modes are different, the system processes the input to convert it from one format to another using gesture, speech or text recognition systems (as applicable) that are based on machine learning algorithms. The system permits the user to define their own constructs and dictionaries, thus its use extends beyond standard Indian and International languages. Further, it adapts to the user’s proficiency level, adjusting the input and output rates to the speed at which the user processes the input/ output. Since a novice user may become proficient with time, the system is designed to continuously adapt to the changing proficiency of the user.