Generating Natural Language from AAC Cards for Children with Autism using a Sequence-to-Sequence System

Abstract

Communication deficits are a common challenge experienced by children with Autism Spectrum Disorder (ASD). Existing Augmentative and Alternative Communication (AAC) applications, such as VICARA, are limited to unidirectional communication—only transmitting raw card sequences to another application (e.g., "I, Eat, Noodle"). This limitation renders the conversation flow non-interactive and prone to miscommunication. This study aims to develop a new version of VICARA, named Talk of the Heart AAC, a bidirectional Android-based application capable of bridging the communication gap between autistic children and their caregivers. The application is designed to translate the constructed card sequences into complete, natural, and semantically logical sentences in Bahasa Indonesia through the implementation of a Sequence-to-Sequence model with a Gated Recurrent Unit architecture augmented by an attention mechanism. To train the model, a synthetic dataset comprising 7,000 data pairs was generated using the Google Gemini API and validated by a speech-language pathology expert. Model evaluation results demonstrated a good performance in sentence translation, yielding a BLEU score of 42.95%, ROUGE-1 94.94%, ROUGE-2 87.12%, and ROUGE-L 93.70%. From the application perspective, non-functional testing produced an average System Usability Scale score of 91.67, categorized as "excellent."