The ability to decode raw brain activity opens up many possibilities to develop healthcare technologies that can improve the quality of life of people with neurological disorders and amputees. The aim of this work is to research hand gesture prediction during ‘grasp-and-lift’ (GAL) motions, using signal processing and machine learning techniques with electroencephalogram (EEG) signals. Being able to reliably detect different hand gestures based off EEG signals from wearable sensors would allow amputees and those with neurological disorders to intuitively control a robotic hand prosthesis.
Grasp-and-Lift
The GAL is defined as 6 distinct events, taking place in the same order, as:
- HandStart - reach for small object
- FirstDigitTouch - grasp object with index finger and thumb
- BothStartLoadPhase - lift the object a few centimetres
- LiftOff - hold it in the air for a few seconds
- Replace - place the object back down
- BothReleased - release the object
The EEG Data
The dataset downloaded from Kaggle consists of the EEG recordings for 12 different subjects. Each subject has 10 series of recordings, each of which contains 30 GALs. 32 channels of EEG data are recorded, with a sampling rate of 500 Hz. Further information about the data acquisition can be found at: https://www.nature.com/articles/sdata201447 .
Each sample of the EEG data has a corresponding one-hot encoded events vector, with one column for each type of event. A 1 in a column means the event has occurred within +-150 ms of that sample. The image below shows the first 20 seconds of subject 1’s data from channel 1, and the corresponding events at each sample. The events overlap, making this a multi-label classificaton problem (can be more than one event at a time).
Predictions and Evaluation
The evaluation metric used in the Kaggle competition is the area under the receiver operating characteristic curve (AUROC), averaged across all events. The ROC curve plots the true positive rate vs the false positive rate for different decision probability thresholds (the probability at which the class decision becomes 1). A classifier with a threshold = 0.1 is going to make far more positive (i.e. 1) decisions than a classifier with threshold = 0.9, but at the cost of specificity. Using the AUROC allows for the evaluation of predictions without having to decide on the decision probability threshold. Given this metric, the probability of each GAL event happening at each sample is predicted, not the 1 or 0 class label.
Approach
Scores of 0.97+ have been achieved on the Kaggle test set using deep learning model ensembles. This is a very computationally intensive approach that requires GPUs for realistic training times. This work focuses on less intensive techniques, firstly investigating how preprocessing and feature extraction techniques can improve classification using only logistic regression. Based on the findings, more powerful models will be tested in the future.
A note on testing: the Kaggle submission restriction of 4 per day prevents testing a large number of different algorithms. For this reason, the test set used is the 7th and 8th series of the training data, for which the events are available to compute the AUROC score.