Tempo-oriented music recommendation system based on human activity recognition using accelerometer and gyroscope data

Seung-Su Shin; Gi Yong Lee; Hyoung-Gook Kim

doi:10.7776/ASK.2020.39.4.286

All Issue

2020 Vol.39, Issue 4 Preview Page Next Page

Research Article

Tempo-oriented music recommendation system based on human activity recognition using accelerometer and gyroscope data 가속도계와 자이로스코프 데이터를 사용한 인간 행동 인식 기반의 템포 지향 음악 추천 시스템

31 July 2020. pp. 286-291

PDF XML

Abstract

In this paper, we propose a system that recommends music through tempo-oriented music classification and sensor-based human activity recognition. The proposed method indexes music files using tempo-oriented music classification and recommends suitable music according to the recognized user’s activity. For accurate music classification, a dynamic classification based on a modulation spectrum and a sequence classification based on a Mel-spectrogram are used in combination. In addition, simple accelerometer and gyroscope sensor data of the smartphone are applied to deep spiking neural networks to improve activity recognition performance. Finally, music recommendation is performed through a mapping table considering the relationship between the recognized activity and the indexed music file. The experimental results show that the proposed system is suitable for use in any practical mobile device with a music player.

Keywords

Music recommendation

Human activity recognition

Tempo-oriented music classification

Deep spiking neural network

본 논문에서는 템포 기반의 음악 분류와 센서 기반의 인간 행동 인식을 통한 음악을 추천하는 시스템을 제안한다. 제안하는 방식은 템포 기반의 음악 분류를 통해 음악 파일을 색인하고, 인식된 행동에 따라 적합한 음악을 추천한다. 정확한 음악 분류를 위해 변조 스펙트럼 기반의 동적 분류기와 멜 스펙트로그램 기반의 시퀀스 분류기가 함께 사용된다. 또한, 간단한 스마트폰 가속도계, 자이로스코프 센서 데이터가 심층 스파이킹 신경망에 적용되어 행동 인식 성능을 향상시킨다. 마지막으로 인식된 행동과 색인된 음악 파일의 관계를 고려한 매핑 테이블을 통해 음악 추천이 수행된다. 실험 결과는 제안된 시스템이 음악 플레이어가 있는 실제 모바일 장치에 사용하기에 적합하다는 것을 보여준다.

키워드

음악 추천

인간 행동 인식

템포 지향 음악 분류

심층 스파이킹 신경망

References

M. Schedl, H. Zamani, C. W. Chen, Y. Deldjoo, and M. Elahi, "Current challenges and visions in music recommender systems research," Int. J. Multimedia Inf. Retrieval, 7, 95-116 (2018).

10.1007/s13735-018-0154-2

D. Ayata, Y. Yaslan, and M. E. Kamasak, "Emotion based music recommendation system using wearable physiological sensors," IEEE Trans. Consum. Electron. 64, 196-203 (2018).

10.1109/TCE.2018.2844736

G. T. Elliott and B. Tomlinson, "PersonalSoundtrack: Context-aware playlists that adapt to user pace," Proc. SIGCHI, Montreal, QC, Canada, 736-741 (2006).

10.1145/1125451.1125599

F. R. D. Carpentier and R. F. Potter, "Effects of music on physiological arousal: Explorations into tempo and genre," Media Psychol. 10, 339-363 (2007).

10.1080/15213260701533045

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proc. IEEE Conf. CVPR, Las Vegas, NV, 770-778 (2016).

10.1109/CVPR.2016.9026180094

J. Wang, B. Peng, and X. Zhang, "Using a stacked residual LSTM model for sentiment intensity prediction," Neurocomputing, 322, 93-101 (2018).

10.1016/j.neucom.2018.09.049

R. Grzeszick, J. M. Lenk, F. M. Rueda, G. A. Fink, S. Feldhorst, and M. T. Hompel, "Deep neural network based human activity recognition for the order picking process," Proc. iWOAR, Rostock, Germany, 1-6 (2017).

10.1145/3134230.3134231

W. Severa, C. M. Vineyard, R. Dellana, S. J. Verzi, and J. B. Aimone, "Whetstone: A method for training deep artificial neural networks for binary communication," arXiv:1810.11521v1 (2018).

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 39
No :4
Pages :286-291
Received Date : 2020-05-11
Revised Date : 2020-07-09
Accepted Date : 2020-07-20
DOI :https://doi.org/10.7776/ASK.2020.39.4.286

The Journal of the Acoustical Society of Korea ISSN:1225-4428(Print) 2287-3775(Online) 한국음향학회지

All Issue