KSA Vision SignSpeak: An Integrative Multimodal AI-Powered Communication Platform for Enhanced Interaction Among Deaf and Mute Individuals
KSA Vision SignSpeak is a funded research initiative focused on the development and investigation of artificial-intelligence technologies for improving communication accessibility and inclusion for individuals with hearing and speech disabilities. The project explores multimodal AI, sign-language recognition, computer vision, deep learning, speech-to-text and text-to-speech technologies, and interpretable AI approaches toward more effective assistive communication solutions.
Table Of Content
- Project at a Glance
- Project Overview
- Research Objectives
- 1. Multimodal Communication
- 2. Sign Language Recognition
- 3. Arabic Sign Language
- 4. Speech & Text Accessibility
- 5. Explainable & Trustworthy AI
- 6. Inclusive Assistive Technology
- Research Team
- Publications & Research Outputs
- Paper 01 — ActiveCNN-SL
- Paper 02 — Comprehensive Review
- Paper 03 — Arabic Sign Language + XAI
- Project Impact & Significance
- Funding Acknowledgment
Project at a Glance
| Project Metadata | Details |
|---|---|
| Project Title | KSA Vision SignSpeak |
| Full Title | An Integrative Multimodal AI-Powered Communication Platform for Enhanced Interaction Among Deaf and Mute Individuals |
| Project Type | Funded Research & Innovation Group |
| Funding Organization | King Salman Center for Disability Research |
| Research Group / Grant No. | KSRG-2023-435 |
| Year | 2023 |
| Contract Date | 1 May 2023 |
| Original Duration | 12 months |
| Principal Investigator | Dr. Nadiah A. Baghdadi |
| My Role | Prof. Mostafa A. Elhosseini — Co-Researcher / Project Team Member |
| Research Domain | Artificial Intelligence & Assistive Technologies |
| Primary Focus | Multimodal communication and sign-language technologies |
| Research Outputs | 3 peer-reviewed publications |
The contract was signed in May 2023 and specifies an original 12-month duration, with possible extension subject to approval.
Project Overview
The project investigates how multimodal artificial intelligence can reduce communication barriers and improve accessibility for people with hearing and speech disabilities. Its research direction integrates computer vision, machine learning and deep learning, sign-language recognition, real-time speech/text technologies, and explainable AI to support more natural and accessible interaction.
A particular emphasis is placed on developing robust sign-language recognition systems, studying inclusive AI-assisted communication technologies, and advancing Arabic Sign Language recognition using modern deep-learning architectures and interpretable machine-learning techniques.
Research Objectives
1. Multimodal Communication
Integrate complementary AI modalities to improve communication between people with different hearing and speech abilities.
2. Sign Language Recognition
Develop intelligent models capable of accurately recognizing and interpreting sign-language gestures.
3. Arabic Sign Language
Advance robust Arabic Sign Language recognition technologies suitable for practical assistive applications.
4. Speech & Text Accessibility
Investigate speech-to-text and text-to-speech technologies for accessible real-time communication.
5. Explainable & Trustworthy AI
Incorporate interpretable AI techniques to improve transparency and confidence in automated sign-language recognition.
6. Inclusive Assistive Technology
Support intelligent tools that increase accessibility, autonomy, education, and social participation.
Research Team
Dr. Nadiah A. Baghdadi
Principal Investigator
Prof. Mostafa A. Elhosseini
Co-Researcher / Project Team Member
Dr. Amer Malki
Co-Researcher / Project Team Member
Dr. Mansourah Aljohani
Co-Researcher / Project Team Member
Publications & Research Outputs
Paper 01 — ActiveCNN-SL
Journal: Artificial Intelligence Review
Year: 2024
Volume/Article: 57:162
DOI: 10.1007/s10462-024-10792-5
Published online: 1 June 2024
The study proposes ActiveCNN-SL, combining active/deep learning approaches with ResNet50 and YOLOv8 for sign-language recognition. The reported YOLOv8 performance reached 97.8%, while the ResNet experiment reported very high training and validation accuracy.
Paper 02 — Comprehensive Review
Journal: Artificial Intelligence Review
Year: 2024
Volume/Article: 57:188
DOI: 10.1007/s10462-024-10816-0
Paper 03 — Arabic Sign Language + XAI
Journal: Journal of Disability Research
Year: 2024
Volume: 3
Pages: 1–15
Article ID: e20240092
DOI: 10.57197/JDR-2024-0092
Published online: 2 November 2024
Project Impact & Significance
KSA Vision SignSpeak contributes to the development of inclusive and accessible intelligent communication technologies by combining advances in artificial intelligence, computer vision, deep learning, sign-language recognition, and explainable AI. The project’s published outputs address both the technological foundations of accessible communication and the development of high-performance recognition frameworks for sign language, including Arabic Sign Language.
The research supports the broader goal of reducing communication barriers and increasing participation, accessibility, and autonomy for individuals with hearing and speech disabilities.
Funding Acknowledgment
This research project was funded by the King Salman Center for Disability Research through Research Group No. KSRG-2023-435.
No Comment! Be the first one.