Triplet loss based domain adversarial training for robust wake-up word detection in noisy environments

Hyungjun Lim; Myunghun Jung; Hoirin Kim

doi:10.7776/ASK.2020.39.5.468

All Issue

2020 Vol.39, Issue 5 Preview Page Next Page

Research Article

Triplet loss based domain adversarial training for robust wake-up word detection in noisy environments 잡음 환경에 강인한 기동어 검출을 위한 삼중항 손실 기반 도메인 적대적 훈련

30 September 2020. pp. 468-475

PDF XML

Abstract

A good acoustic word embedding that can well express the characteristics of word plays an important role in wake-up word detection (WWD). However, the representation ability of acoustic word embedding may be weakened due to various types of environmental noise occurred in the place where WWD works, causing performance degradation. In this paper, we proposed triplet loss based Domain Adversarial Training (tDAT) mitigating environmental factors that can affect acoustic word embedding. Through experiments in noisy environments, we verified that the proposed method effectively improves the conventional DAT approach, and checked its scalability by combining with other method proposed for robust WWD.

Keywords

Wake-up word detection

Acoustic word embedding

Domain adversarial training

Triplet loss

단어의 특성을 잘 표현하는 음성 단어 임베딩은 기동어 인식에서 중요한 역할을 한다. 하지만 기동어 인식이 수행되는 환경에서 필연적으로 발생하는 다양한 종류의 잡음으로 인해 음성 단어 임베딩의 표현 능력이 손상될 수 있으며, 인식 성능의 저하를 초래할 수 있다. 본 논문에서는 음성 단어 임베딩에 영향을 줄 수 있는 환경적인 요인을 완화시키는 삼중항 손실 기반의 도메인 적대적 훈련 방식을 제안한다. 잡음 환경에서의 기동어 검출 실험을 통해 제안하는 방식이 기존의 도메인 적대적 훈련 방식을 효과적으로 개선하는 모습을 확인할 수 있었고, 잡음 환경에서의 기동어 검출을 위해 기존에 제안된 다른 방법과의 결합을 통해 제안하는 방식의 확장성을 확인할 수 있었다.

키워드

기동어 검출

음향 단어 임베딩

도메인 적대적 훈련

삼중항 손실

References

Y. Zhang and J. R. Glass "Unsupervised spoken keyword spotting via segmental DTW on Gaussian posteriorgrams," Proc. ASRU. 398-403 (2009).

10.1109/ASRU.2009.5372931

G. Mantena and K. Prahallad, "Use of articulatory bottle-neck features for query-by-example spoken term detection in low resource scenarios," Proc. ICASSP. 7128-7132 (2014).

10.1109/ICASSP.2014.6854983

H. Lim, Y. Kim, Y. Kim, and H. Kim, "CNN-based bottleneck feature for noise robust query-by-example spoken term detection," Proc. APSIPA. 1278-1281 (2017).

10.1109/APSIPA.2017.8282220PMC5698221

G. Chen, C. Parada, and T. N. Sainath, "Query-by- example keyword spotting using long short-term memory networks," Proc. ICASSP. 5236-5240 (2015).

10.1109/ICASSP.2015.717897026496023

S. Settle and K. Livescu, "Discriminative acoustic word embeddings: Recurrent neural network-based approaches," Proc. SLT. 503-510 (2016).

10.1109/SLT.2016.7846310

M. Jung, H. Lim, J. Goo, Y. Jung, and H. Kim, "Additional shared decoder on Siamese multi-view encoders for learning acoustic word embeddings," Proc. ASRU. 629-636 (2019).

10.1109/ASRU46091.2019.9003929

H. Lim, Y. Kim, J. Goo, and H. Kim, "Interlayer selective attention network for robust personalized wake-up word detection," IEEE Signal Process. Lett. 27, 126-130 (2020).

10.1109/LSP.2019.2959902

Y. Ganin, H. Ajakan, H. Larochelle, F. Laviolette, and V. Lempitsky, "Domain-adversarial training of neural networks," J. Mach. Learn. Res. 17, 2096-2030 (2016).

E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, "Adversarial discriminative domain adaptation," Proc. CVPR. 7167-7176 (2017).

10.1109/CVPR.2017.316

Z. Pei, Z. Cao, M. Long, and J. Wang, "Multi-adversarial domain adaptation," Proc. AAAI. 3934-3941 (2018).

R. Wang, M. Utiyama, A. Finch, L. Liu, K. Chen, and E. Sumita, "Sentence selection and weighting for neural machine translation domain adaptation," IEEE/ ACM Trans. Audio, Speech, Lang. Process. 26, 1727- 1741 (2018).

10.1109/TASLP.2018.2837223

A. Tripathi, A. Mohan, S. Anand, and M. Singh, "Adversarial learning of raw speech features for domain invariant speech recognition," Proc. ICASSP. 5959-5963 (2018).

10.1109/ICASSP.2018.8462452

S. Sun, C. F. Yeh, M. Y. Hwang, M. Ostendorf, and L. Xie, "Domain adversarial training for accented speech recognition," Proc. ICASSP. 4854-4858 (2018).

10.1109/ICASSP.2018.8462663PMC6242895

S. Mirsamadi and J. H. Hansen, "Multi-domain adversarial training of neural network acoustic models for distant speech recognition," Speech Commun. 106, 21-30 (2019).

10.1016/j.specom.2018.10.010

Y. Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation," Proc. ICML. 1180- 1189 (2015).

D. B. Paul and J. M. Baker, "The design for the Wall Street Journal-based CSR corpus," Proc. Workshop Speech and Natural Lang. 357-362 (1992).

10.3115/1075527.1075614

D. Dean, S. Sridharan, R. Vogt, and M. Mason, "The QUT-NOISE-TIMIT corpus for the evaluation of voice activity detection algorithms," Proc. Interspeech, 3110- 3113 (2010).

H. G. Hirsch and D. Pearce, "The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions," Proc. ISCA ITRW ASR. 181-188 (2000).

M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Lenvnberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wiche, Y. Yu, and X. Zheng, "Tensor Flow: Large-scale machine learning on heterogeneous systems," Proc. USENIX OSDI. 265-283 (2016).

D. Kingma and J. Ba, "Adam: A method for stochastic optimization," Proc. ICLR. 1-15 (2015).

K. Hajian-Tilaki, "Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation," Caspian J. Intern. Med. 4, 627-635 (2013).

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 39
No :5
Pages :468-475
Received Date : 2020-07-17
Revised Date : 2020-08-28
Accepted Date : 2020-08-29
DOI :https://doi.org/10.7776/ASK.2020.39.5.468

The Journal of the Acoustical Society of Korea ISSN:1225-4428(Print) 2287-3775(Online) 한국음향학회지

All Issue