profile

Research Assistant

Center for Computational & Data Sciences (CCDS), Independent University, Bangladesh (IUB)

Research Assistant (Part-Time)

Center for Computational & Data Sciences (CCDS), Independent University, Bangladesh (IUB)

Undergraduate Research Student

Computing for Sustainability and Social Good (C2SG) Lab, Brac University

Bachelor of Science

Brac University

Computer Science and Engineering

DM-Codec: Distilling Multimodal Representations for Speech Tokenization

EMNLP 2025 (Findings)

We propose two novel distillation approaches: (1) a language model (LM)-guided distillation method that incorporates contextual information, and (2) a combined LM and self-supervised speech model (SM)-guided distillation technique that effectively distills multimodal representations (acoustic, semantic, and contextual) into a comprehensive speech tokenizer, termed DM-Codec. The DM-Codec architecture adopts a streamlined encoder-decoder framework with a Residual Vector Quantizer (RVQ) and incorporates the LM and SM during the training process. Experiments show DM-Codec significantly outperforms state-of-the-art speech tokenization models, reducing WER by up to 13.46%, WIL by 9.82%, and improving speech quality by 5.84% and intelligibility by 1.85% on the LibriSpeech benchmark dataset.