LFMMI-based acoustic modeling by using external knowledge

Hosung Park; Yoseb Kang; Minkyu Lim; Donghyun Lee; Junseok Oh; Ji-Hwan Kim

doi:10.7776/ASK.2019.38.5.607

All Issue

2019 Vol.38, Issue 5 Preview Page Next Page

Research Article

LFMMI-based acoustic modeling by using external knowledge External knowledge를 사용한 LFMMI 기반 음향 모델링

30 September 2019. pp. 607-613

PDF XML

Abstract

This paper proposes LF-MMI (Lattice Free Maximum Mutual Information)-based acoustic modeling using external knowledge for speech recognition. Note that an external knowledge refers to text data other than training data used in acoustic model. LF-MMI, objective function for optimization of training DNN (Deep Neural Network), has high performances in discriminative training. In LF-MMI, a phoneme probability as prior probability is used for predicting posterior probability of the DNN-based acoustic model. We propose using external knowledges for training the prior probability model to improve acoustic model based on DNN. It is measured to relative improvement 14 % as compared with the conventional LF-MMI-based model.

Keywords

Speech recognition

Acoustic model

LF-MMI (Lattice Free Maximum Mutual Information)

Phoneme- based language

본 논문은 external knowledge를 사용한 lattice 없는 상호 정보 최대화(Lattice Free Maximum Mutual Information, LF-MMI) 기반 음향 모델링 방법을 제안한다. External knowledge란 음향 모델에서 사용하는 학습 데이터 이외의 문자열 데이터를 말한다. LF-MMI란 심층 신경망(Deep Neural Network, DNN) 학습의 최적화를 위한 목적 함수의 일종으로, 구별 학습에서 높은 성능을 보인다. LF-MMI에는 DNN의 사후 확률을 계산하기 위해 음소의 열을 사전 확률로 갖는다. 본 논문에서는 LF-MMI의 목적식의 사전 확률을 담당하는 음소 모델링에 external knowlege를 사용함으로써 과적합의 가능성을 낮추고, 음향 모델의 성능을 높이는 방법을 제안한다. External memory를 사용하여 사전 확률을 생성한 LF-MMI 모델을 사용했을 때 기존 LF-MMI와 비교하여 14 %의 상대적 성능 개선을 보였다.

키워드

음성인식

음향모델

lattice 없는 상호 정보 최대화

음소 기반 언어 모델

References

B. Juang and L. Rabiner, "Hidden Markov models for speech recognition," Technometrics, 33, 251-272 (1991).

10.1080/00401706.1991.10484833

S. Suwon, J. Rho, S. Kim, J. Lee, and H. Ko, "Text independent speaker verficiation using dominant state information of HMM-UBM," J. Acoust. Soc. Kr. 34, 171-176 (2015).

10.7776/ASK.2015.34.2.171

L. Bahl, P. Brown, P. de Souza, and R. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition," Proc. ICASSP, 11, 49-52, (1986).

S. Kapadia, V. Valtchev, and S. Young, "MMI training for continuous phoneme recognition on the TIMIT database," Proc. ICASSP, 2, 491-494 (1993).

10.1109/ICASSP.1993.319349

D. Yu and L. Deng, Automatic Speech Recognition (Springer London limited, London, 2016), pp. 193-215.

B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling," Proc. ICASSP, 2, 3761-3764 (2009).

10.1109/ICASSP.2009.4960445

D. Povey, V. Peddinti, D. Galvez, P. Ghahremani, V. Manohar, X. Na, Y. Wang, and S. Khudanpur, "Puerly sequence-trained neural networks for ASR based on lattice-free MMI," Proc. Intetspeech, 2751-2755 (2016).

10.21437/Interspeech.2016-595

D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, "The Kaldi speech recognition toolkit," Proc. ASRU. (2011).

H. Hadian, H. Sameti, D. Povey, and S. Khudanpur, "End-to-end speech recognition using lattice-free MMI," Proc. Interspeech, 2345-2349 (2013).

H. Lim, M. Kim, and H. Kim, "Sound event classification using deep neural network based transfer learning," (in Korean), J. Acoust. Soc. Kr. 35, 143- 148 (2016).

10.7776/ASK.2016.35.2.143

K. Lee, J. Jeon, and M. Jung, "Automatic generation of pronunciation variants for Korean continuous speech recognition" (in Korean), J. Acoust. Soc. Kr. 20, 35-43 (2001).

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 38
No :5
Pages :607-613
Received Date : 2019-07-10
Accepted Date : 2019-09-10
DOI :https://doi.org/10.7776/ASK.2019.38.5.607

The Journal of the Acoustical Society of Korea ISSN:1225-4428(Print) 2287-3775(Online) 한국음향학회지

All Issue