All Issue

2019 Vol.38, Issue 4 Preview Page

Research Article

Estimating speech parameters for ultrasonic Doppler signal using LSTM recurrent neural networks

LSTM 순환 신경망을 이용한 초음파 도플러 신호의 음성 패러미터 추정

Hyeong-Kil Joo1∗
,
Ki-Seung Lee1
주 형길1∗
,
이 기승1
1건국대학교 전기전자공학부
∗Corresponding Author
31 July 2019. pp. 433-441
PDF XML Citation
Abstract

In this paper, a method of estimating speech parameters for ultrasonic Doppler signals reflected from the articulatory muscles using LSTM (Long Short Term Memory) RNN (Recurrent Neural Networks) was introduced and compared with the method using MLP (Multi-Layer Perceptrons). LSTM RNN were used to estimate the Fourier transform coefficients of speech signals from the ultrasonic Doppler signals. The log energy value of the Mel frequency band and the Fourier transform coefficients, which were extracted respectively from the ultrasonic Doppler signal and the speech signal, were used as the input and reference for training LSTM RNN. The performance of LSTM RNN and MLP was evaluated and compared by experiments using test data, and the RMSE (Root Mean Squared Error) was used as a measure. The RMSE of each experiment was 0.5810 and 0.7380, respectively. The difference was about 0.1570, so that it confirmed that the performance of the method using the LSTM RNN was better.

Keywords
Silence speech interface
Speech parameters
Ultrasonic Doppler signal
LSTM (Long Short Term Memory) RNN (Recurrent Neural Networks)

본 논문에서는 입 주변에 방사한 초음파 신호가 반사되어 돌아올 때 발생하는 초음파 도플러 신호를 LSTM (Long Short Term Memory) 순환 신경망 (Recurrent Neural Networks, RNN)을 이용해 음성 패러미터를 추정하는 방법을 소개하고 다층 퍼셉트론 (Multi-Layer Perceptrons, MLP) 신경망을 이용한 방법과 성능 비교를 하였다. 본 논문에서는 LSTM 순환 신경망을 이용해 초음파 도플러 신호로부터 음성 신호의 푸리에 변환 계수를 추정하였다. LSTM 순환 신경망을 학습하기 위한 입력 및 기준값으로 초음파 도플러 신호와 음성 신호로부터 각각 추출된 멜 주파수 대역별 에너지 로그값과 푸리에 변환 계수가 사용되었다. 테스트 데이터를 이용한 실험을 통해 LSTM 순환 신경망과 MLP의 성능을 평가, 비교하였고 척도로는 평균 제곱근 오차(Root Mean Squared Error, RMSE)가 사용되었다.각 실험의 RMSE는 각각 0.5810, 0.7380로 나타났다. 약 0.1570 차이로 LSTM 순환 신경망을 이용한 방법의 성능 우세한 것으로 확인되었다.

키워드
무음성 인터페이스
음성 패러미터
초음파 도플러 신호
LSTM (Long Short Term Memory) 순환 신경망
References
1
B. Denby, T. Schultz, K. Honda, T. Hueber, J. M. Gilbert, and J. S. Brumberg, "Silent speech interfaces," Speech Comm. 52, 270-287 (2010).
10.1016/j.specom.2009.08.002
2
K. S. Lee, "Prediction of acoustic feature parameters using myoelectric signals," IEEE Trans. On Biomed. Eng. 51, 1587-1595 (2010).
10.1109/TBME.2010.204145520172775
3
T. Toda and K. Shikano, "NAM-to-Speech conversion with Gaussian Mixture Models," Proc. Interspeech, 1957-1960 (2005).
4
S. Li, J. Q. Wang, M. Niu, T. Liu, and X. J. Jing, "The enhancement of millimeter wave conduct speech based on perceptual weighting," Progress in Electromagnetics Research B, 9, 199-214 (2008).
10.2528/PIERB08063001
5
K. Nakamura, T. Toda, H. Saruwatari, and K. Shikano, "Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech," Speech Comm. 54, 134-146 (2012).
10.1016/j.specom.2011.07.007
6
K. Kalgaonkar and B. Raj, "An acoustic Doppler- based front end for hands free spoken user interaces," Proc. SLT, 158-161 (2006).
10.1109/SLT.2006.326835
7
K. Kalgaonkar and B. Raj, "Acoustic Doppler sonar for gait recognition," Proc. 2007 IEEE Conf. Advanced Video and Signal Based Surveillance, 27-32 (2007).
8
K. Kalgaonkar and B. Raj, "One-handed gesture recognition using ultrasonic Doppler sonar," Proc. ICASSP, 1889-1892 (2009).
10.1109/ICASSP.2009.4959977
9
S. Srinivasan, B. Raj, and T. Ezzat, "Ultrasonic sensing for robust speech recognition," Proc. ICASSP, 5102-5105 (2010).
10.1109/ICASSP.2010.5495039
10
K. Livescu, B. Zhu, and J. Glass, "On the phonetic information in ultrasonic microphone signals," Proc. ICASSP, 4621-4624 (2009).
10.1109/ICASSP.2009.4960660
11
A. R. Toth, B. Raj, K. Kalgaonkar, and T. Ezzat, "Synthesizing speech from Doppler signals," Proc. ICASSP, 4638-4641 (2010).
10.1109/ICASSP.2010.5495552
12
K. S. Lee, "Speech synthesis using acoustic Doppler signal" (in Korean), J. Acoust. Soc. Kr. 35, 134-142 (2016).
10.7776/ASK.2016.35.2.134
13
K. S. Lee, "Automatic speech recognition using acoustic doppler signal" (in Korean), J. Acoust. Soc. Kr. 35, 74-82 (2016).
10.7776/ASK.2016.35.1.074
14
K. S. Lee, "Speech Enhancement using ultrasonic doppler sonar," Speech Comm. 110, 21-32 (2019).
10.1016/j.specom.2019.03.008
15
F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Spartan Books, Washington DC, 1961), pp. 3-585.
10.21236/AD0256582
16
D.E. Rumelhart, G.E. Hilton, and R.J. Williams, "Learning internal representations by error propag- ation," in parallel distributed processing: Explorati- ons in the microstructure of cognitio (MIT press, Cambridge, 1986), pp. 318-362.
10.21236/ADA1644533159828
17
S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation. 9, 1735-1780 (1997).
10.1162/neco.1997.9.8.17359377276
18
F. Gers, N. Schraudolph, and J. Schmidhuber, "Learning Precise Timing with LSTM Recurrent Networks," Journal of Machine Learning Research. 3, 115-143 (2002).
19
Understanding LSTM Networks, http://colah.github.io/ posts/2015-08-Under standing- LSTMs/, 2019.
20
J. Turian, J. Bergstra, and Y. Bengio, "Quadratic features and deep architectures for chunking," Proc. NAACL HLT 2009, 245-248 (2009).
10.3115/1620853.1620921
21
Ptb_Word_lm.py, https://github.com/tensorflow/models/ blob/master/tutorials/rnn/ptb/ptb_word_ lm.py
Information
  • Publisher :The Acoustical Society of Korea
  • Publisher(Ko) :한국음향학회
  • Journal Title :The Journal of the Acoustical Society of Korea
  • Journal Title(Ko) :한국음향학회지
  • Volume : 38
  • No :4
  • Pages :433-441
  • Received Date : 2019-05-15
  • Revised Date : 2019-07-03
  • Accepted Date : 2019-07-11
  • DOI :https://doi.org/10.7776/ASK.2019.38.4.433
Journal Informaiton The Journal of the Acoustical Society of Korea The Journal of the Acoustical Society of Korea
E-Book
  • scopus_JASK
  • ESCI_JASK
  • Google scholar_JASK
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • ccl
Journal Informaiton Journal Informaiton - close