All Issue

2020 Vol.39, Issue 5 Preview Page

Research Article

Dilated convolution and gated linear unit based sound event detection and tagging algorithm using weak label

약한 레이블을 이용한 확장 합성곱 신경망과 게이트 선형 유닛 기반 음향 이벤트 검출 및 태깅 알고리즘

Chungho Park1
,
Donghyun Kim1
,
Hanseok Ko1∗
박 충호1
,
김 동현1
,
고 한석1∗
1고려대학교 전기전자공학과 지능신호처리 연구실
∗Corresponding Author
30 September 2020. pp. 414-423
PDF XML Citation
Abstract

In this paper, we propose a Dilated Convolution Gate Linear Unit (DCGLU) to mitigate the lack of sparsity and small receptive field problems caused by the segmentation map extraction process in sound event detection with weak labels. In the advent of deep learning framework, segmentation map extraction approaches have shown improved performance in noisy environments. However, these methods are forced to maintain the size of the feature map to extract the segmentation map as the model would be constructed without a pooling operation. As a result, the performance of these methods is deteriorated with a lack of sparsity and a small receptive field. To mitigate these problems, we utilize GLU to control the flow of information and Dilated Convolutional Neural Networks (DCNNs) to increase the receptive field without additional learning parameters. For the performance evaluation, we employ a URBAN-SED and self-organized bird sound dataset. The relevant experiments show that our proposed DCGLU model outperforms over other baselines. In particular, our method is shown to exhibit robustness against nature sound noises with three Signal to Noise Ratio (SNR) levels (20 dB, 10 dB and 0 dB).

Keywords
Audio tagging
Sound event detection
Dilated convolution
Gated linear unit
T-f segmentation map
Weak label

본 논문은 약한 레이블 기반 음향 이벤트 검출을 위한 시간-주파수 영역분할 맵 추출 모델에서 발생하는 희소성 및 수용영역 부족에 관한 문제를 완화 시키기 위해, 확장 게이트 선형 유닛(Dilated Convolution Gated Linear Unit, DCGLU)을 제안한다. 딥러닝 분야에서 음향 이벤트 검출을 위한 영역분할 맵 추출 기반 방법은 잡음 환경에서 좋은 성능을 보여준다. 하지만, 이 방법은 영역분할 맵을 추출하기 위해 특징 맵의 크기를 유지해야 하므로 풀링 연산 없이 모델을 구성하게 된다. 이로 인해 이 방법은 희소성과 수용영역의 부족으로 성능 저하를 보이게 된다. 이런 문제를 완화하기 위해, 본 논문에서는 정보의 흐름을 제어할 수 있는 게이트 선형 유닛과 추가의 파라미터 없이 수용영역을 넓혀 줄 수 있는 확장 합성곱 신경망을 적용하였다. 실험을 위해 사용된 데이터는 URBAN-SED와 자체 제작한 조류 울음소리 데이터이며, 제안하는 DCGLU 모델이 기존 베이스라인 논문들보다 더 좋을 성능을 보였다. 특히, DCGLU 모델이 자연 소리가 섞인 환경인 세 개의 Signal to Noise Ratio(SNR)(20 dB, 10 dB, 0 dB)에서 강인하다는 것을 확인하였다.

키워드
음향 태깅
음향 이벤트 검출
확장 합성곱 신경망
게이트 선형 유닛
시간-주파수 영역분할 맵
약한 레이블
References
1
P. K. Atrey, N. C. Maddage, and M. S. Kankanhlli, "Audio based event detection for multimedia surveillance," Proc. IEEE ICASSP. 5. V-V (2006).
2
J. Maxime, X. Alameda-Pineda, L. Girin, and R. Horaud, "Sound representation and classification benchmark for domestic robots," Proc. IEEE ICRA. 6285- 6292 (2014).
10.1109/ICRA.2014.6907786
3
D. Stowell, M. Wood, Y. Stylianou, and H. Glotin, "Bird detection in audio: a survey and a challenge," Proc. IEEE 26th MLSP. 1-6 (2016).
10.1109/MLSP.2016.773887530241180
4
D. Stowell and M. D. Plumbley, "Audio-only bird classification using unsupervised feature learning," Proc. CLEF. 673-684 (2014).
5
K. Ko, J. Park, D. K. Han, and H. Ko, "Channel and frequency attention module for diverse animal sound classification," IEICE Trans. on Information and Systems, E102-D, 2615-2618 (2019).
10.1587/transinf.2019EDL8128
6
S. Park, M. Elhilali, D. K. Han, and H. Ko, "Amphibian sounds generating network based on adversarial learning," IEEE Signal Processing Letters, 27, 640- 644 (2020).
10.1109/LSP.2020.2988199
7
K. Ko, S. Park, and H. Ko, "Convolutional neural netework based amphibian sound classification using covariance and modulogram" (in Korean), J. Acoust. Soc. Kr. 37, 61-65 (2018).
8
D. Stowell, D. Giannoulis, E. Benetos, M. Lagrange, and M. D. Plumbley, "Detection and classification of acoustic scenes and events," IEEE Trans. Multimedia, 17, 1733-1746 (2015).
10.1109/TMM.2015.2428998
9
G. Parascandolo, H. Huttunen, and T. Virtanen, "Recurrent neural networks for polyphonic sound event detection in real life recordings," Proc. IEEE ICASSP. 6440-6444 (2016).
10.1109/ICASSP.2016.7472917
10
A. Mesaros, T. Heittola, and T. Virtanen, "TUT database for acoustic scene classification and sound event detection," Proc. 24th EUSIPCO. 1128-1132 (2016).
10.1109/EUSIPCO.2016.7760424
11
S.-Y. Chou, J.-S. R. Jang, and Y.-H. Yang, "Frame CNN: A weakly-supervised learning framework for frame-wise acoustic event detetion and classification," DACSE. Tech. Rep., 2017.
12
A. Kumar and B. Raj, "Deep cnn framework for audio event recognition using weak labeled web data," arXiv: 1707.02530 (2017).
13
Y. Xu, Q. Kong, W. Wang, and M. D. Plumbley, "Large-scale weakly supervised audio classification using gated convolutional neural network," Proc. IEEE ICASSP. 121-125 (2018).
10.1109/ICASSP.2018.8461975
14
Q. Kong, Y. Xu, I. Sobieraj, W. Wang, and M. D. Plumbley, "Sound event detection and time-frequency segmentation from weak labelled data," IEEE/ACM Trans. on Audio, Speech, And Lang. Processing, 27, 777-787 (2019).
10.1109/TASLP.2019.2895254
15
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv: 1409.1556 (2014).
16
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proc. IEEE CVPR. 770-778 (2016).
10.1109/CVPR.2016.9026180094
17
Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, "Language modeling with gated convolutional networks," Proc. PMLR. 70, 933-941 (2017).
18
Y. Chen, Q. Guo, X. Liang, J. Wang, and Y. Qian, "Environmental sound classification with dilated convolutions," Applied Acoustics, 148, 123-132 (2019).
10.1016/j.apacoust.2018.12.019
19
J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello, "SCAPER:a library for soundscape synthesis and augmentation," Proc. IEEE WASPAA. 344-348 (2017).
10.1109/WASPAA.2017.8170052
20
A. Kolesnikov and C. H. Lampert, "Seed, expand and constrain: Three principles for weakly-supervised image segmentation," Proc. ECCV. 695-711 (2016).
10.1007/978-3-319-46493-0_42
21
Q. Kong, T. Iqbal, Y. Xu, W. Wang, and M. D. Plumbley, "DCASE 2018 challenge baseline with convolutional neural networks," DACSE. Tech. Rep., 2018.
22
K. Miyazaki, T. Komatsu, T. Hayashi, S. Watanabe, T. Tuda, and K. Takeda, "Weakly-supervised sound event detection with self-attention," Proc. IEEE ICASSP. 66-70 (2020).
10.1109/ICASSP40776.2020.9053609PMC6960455
23
Y. Li, M. Liu, K. Drossos, and T. Virtanen, "Sound event detection via dilated convolutional recurrent neural networks," Proc. IEEE ICASSP. 286-290 (2020).
10.1109/ICASSP40776.2020.9054433
24
D. Kingma and J. Ba, "Adam: a method for stochastic optimization," arXiv:1412.6980 (2015).
25
S. Ioffe and C. Szegedy, "Batch normalization: accelerating deep network training by reducing internal covariate shift," Proc. 32nd ICML. 448-456 (2015).
26
J. A. Hanley and B. J. McNeil, "The meaning and use of the area under a receiver operating characteristic (ROC) curve," Radiology, 431, 29-36 (1982).
10.1148/radiology.143.1.70637477063747
27
R. Girshich, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," Proc. IEEE CVPR. 580- 587 (2014).
10.1109/CVPR.2014.81
Information
  • Publisher :The Acoustical Society of Korea
  • Publisher(Ko) :한국음향학회
  • Journal Title :The Journal of the Acoustical Society of Korea
  • Journal Title(Ko) :한국음향학회지
  • Volume : 39
  • No :5
  • Pages :414-423
  • Received Date : 2020-07-24
  • Accepted Date : 2020-08-27
  • DOI :https://doi.org/10.7776/ASK.2020.39.5.414
Journal Informaiton The Journal of the Acoustical Society of Korea The Journal of the Acoustical Society of Korea
E-Book
  • scopus_JASK
  • ESCI_JASK
  • Google scholar_JASK
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • ccl
Journal Informaiton Journal Informaiton - close