VKM

Abstract

The Voight-Kampff Machine (VKM) is an approach to select a rejection threshold for custom gestures. When users want to customize their gesture set, training data is sparse. So, learning a reliable rejection threshold for malformed and accidental input is a challenging problem. At the same time, high activity (HA) data has no periods of low activity which usually indicate end points of gestures. This leads to a high number of false positives, unless the rejection threshold is tight enough to reject such input. Our approach is a 5-step process for selecting a rejection threshold based on synthetically generated positive and negative gesture samples. In our evaluation we compared VKM to alternative threshold selection techniques and found VKM to be the only approach which consistently achieved high accuracy on 4 high activity datasets: Kinect, Mouse, Vive Position and Vive Quaternion.

• Paper PDF: VKM
• Reference Implementation: VKM Code Repository
• Download Datasets: HA Datasets (302.4 MB)
• DOI: 10.1145/3491102.3502000

Voight-Kampff Machine as a 5-step process

1. Form positive synthetic scores distribution using GPSR [1]
2. Form negative synthetic scores distribution using Mincer [2] (this work)
3. Select rejection threshold using F1-accuracy (score that best separates false positives and false negatives)
4. Inflate threshold to account for future data variability and input device type
5. Reduce threshold through a Monte Carlo simulation based on the number of training samples and observed variability

Gesture Path Stochastic Resampling (GPSR) [1] is a technique to generate synthethic gesture samples, by first stochastically resampling the original trajectory to a fixed number of points, and then normalizing the distances between each point. Mincer [2] is a negative synthetic data generation technique which replaces portions of the original sample's trajectory with sub-trajectory of samples from another class. F1-accuracy minimizes the number of false positives and false negatives by balancing precision and recall.

To evaluate VKM, we created a continuous gesture data processing pipeline, called The Dollar General (TDG). TDG is a collection of device-agnostic techniques as illustrated in the figure below.

Machete [3] proposes regions that might be gestures, Jackknife [4] classifies proposed regions, and VKM rejects input that does not cross the similarity threshold.

Further details can be found in the related publications and in the reference implementation.

Citation

If you end up using our VKM approach in your work, please cite:

@inproceedings{taranta2022_VKM,
    author = {Taranta, Eugene Matthew and Maslych, Mykola and Ghamandi, Ryan and LaViola, Joseph},
    title = {The Voight-Kampff Machine for Automatic Custom Gesture Rejection Threshold Selection},
    year = {2022},
    isbn = {9781450391573},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3491102.3502000},
    doi = {10.1145/3491102.3502000},
    booktitle = {CHI Conference on Human Factors in Computing Systems},
    articleno = {556},
    numpages = {15},
    keywords = {rejection, customization, gesture, recognition},
    location = {New Orleans, LA, USA},
    series = {CHI '22}
}