Md Mubtasim Ahasan

Hello! My name is Md Mubtasim Ahasan. I am working as a Junior Research Scientist at the Center for Computational & Data Sciences (CCDS), Independent University, Bangladesh. My research is supervised by Prof. Amin Ahsan Ali and Prof. AKM Mahbubur Rahman.

Currently, my research focuses on audio and speech tokenization and synthesis, multimodal learning, representation learning and alignment, and large language model (LLM)-powered agents with tool learning. I am fortunate to collaborate with and receive mentorship from Md Mofijul Islam, Aman Chadha, and Tasnim Mohiuddin.

Previously, my undergraduate research, supervised by Prof. Jannatun Noor at the C2SG Lab focused on computer vision and audio signal processing for medical applications. I completed my Bachelor’s from the Department of Computer Science and Engineering, Brac University.

I am actively seeking research collaborations and Ph.D. opportunities.

news

Oct 19, 2024	Our paper DM-Codec is now available on arXiv. Read it here.
Oct 17, 2024	Promoted to Junior Research Scientist at CCDS, IUB.
Mar 01, 2024	Joined as a Research Assistant (Part-time) at CCDS, IUB.

selected publications

Preprint

DM-Codec: Distilling Multimodal Representations for Speech Tokenization

Md Mubtasim Ahasan, Md Fahim, Tasnim Mohiuddin, A K M Mahbubur Rahman, Aman Chadha, Tariq Iqbal, M Ashraful Amin, Md Mofijul Islam, and Amin Ahsan Ali

2024

Abs HTML PDF Code

Recent advancements in speech-language models have yielded significant improvements in speech tokenization and synthesis. However, effectively mapping the complex, multidimensional attributes of speech into discrete tokens remains challenging. This process demands acoustic, semantic, and contextual information for precise speech representations. Existing speech representations generally fall into two categories: acoustic tokens from audio codecs and semantic tokens from speech self-supervised learning models. Although recent efforts have unified acoustic and semantic tokens for improved performance, they overlook the crucial role of contextual representation in comprehensive speech modeling. Our empirical investigations reveal that the absence of contextual representations results in elevated Word Error Rate (WER) and Word Information Lost (WIL) scores in speech transcriptions. To address these limitations, we propose two novel distillation approaches: (1) a language model (LM)-guided distillation method that incorporates contextual information, and (2) a combined LM and self-supervised speech model (SM)-guided distillation technique that effectively distills multimodal representations (acoustic, semantic, and contextual) into a comprehensive speech tokenizer, termed DM-Codec. The DM-Codec architecture adopts a streamlined encoder-decoder framework with a Residual Vector Quantizer (RVQ) and incorporates the LM and SM during the training process. Experiments show DM-Codec significantly outperforms state-of-the-art speech tokenization models, reducing WER by up to 13.46%, WIL by 9.82%, and improving speech quality by 5.84% and intelligibility by 1.85% on the LibriSpeech benchmark dataset.