WatchLogger: Keyboard Typing Words Recognition Based on Smartwatch

Nowadays more and more people are wearing smart-watches in their daily lives. The various sensors embedded in smartwatches bring the ability to evaluate users' status as well as the risk of privacy issues. For example, if users are typing on key-boards while wearing smartwatches, the attacker could know the typing contents from the sensor data collected by the malicious applications that are installed on the targets' smartwatches. In this paper, we propose WatchLogger, the framework using audio and accelerometer signals to recognize the English words being typed, for demonstrating how to implement the smartwatch-based side-channel attack. Different from the previous studies that focused on the recognition of each key or pair of keys being pressed, WatchLogger aims to perform recognition on the scale of words. To achieve this goal, WatchLogger exploits the audio signals for segmentation and the accelerometer signals for classification. In addition, we propose an ensemble classification model to deal with the problem caused by too many words. At last, we build the dataset WTW-100 with 100 classes of words and 100 samples for each class, and we conduct experiments on the dataset. The experimental results show an accuracy of 98.5 % for keystroke recognition and 91.5 % for word classification, showing a considerable performance of WatchLogger.