PalmTouch - An Additional Touch Input Modality
PalmTouch is an additional input modality which differentiates between inputs of fingers and the palm. Similar to Huawei’s Knuckle Sense, HTC’s Edge Sense, and Apple’s Force Touch, it adds a convenient way to perform input in addition to touches with a finger. A study showed that potential users perceived this input technique as natural and intuitive especially for shortcuts and increasing the reachability.
PalmTouch uses deep learning to infer the input source from raw measurements of a capacitive touchscreen. The convolutional neural network trained to recognize palms achieves an accuracy of 99.53% even in realistic scenarios. Since it does not require additional sensing technologies, PalmTouch can be readily deployed to any recent mobile device via a software update. This project was presented at the ACM CHI Conference on Human Factors in Computing Systems (CHI ’18), the premier international conference in the field of Human-Computer Interaction.
Models and Data Set
The project page on GitHub provides the PalmTouch model, the full data set, as well as a number of demo applications to showcase the usefulness of this input method. Instructions to root and flash the required Android kernel onto an LG Nexus 5, and to run PalmTouch are provided as well.
Publications
Huy Viet Le, Thomas Kosch, Patrick Bader, Sven Mayer, and Niels Henze. 2018. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, Montreal QC, Canada.
@inproceedings{le2018palmtouch,
author = {Le, Huy Viet and Kosch, Thomas and Bader, Patrick and Mayer, Sven and Henze, Niels},
title = {PalmTouch: Using the Palm As an Additional Input Modality on Commodity Smartphones},
booktitle = {Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems},
series = {CHI '18},
year = {2018},
isbn = {978-1-4503-5620-6},
location = {Montreal QC, Canada},
pages = {360:1--360:13},
articleno = {360},
numpages = {13},
url = {http://doi.acm.org/10.1145/3173574.3173934},
doi = {10.1145/3173574.3173934},
acmid = {3173934},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {capacitive image, machine learning, palm, smartphone},
}
Huy Viet Le, Sven Mayer, Thomas Kosch, and Niels Henze. 2018. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '18). ACM, Barcelona, Spain.
@inproceedings{le2018demonstrating,
author = {Le, Huy Viet and Mayer, Sven and Kosch, Thomas and Henze, Niels},
title = {Demonstrating PalmTouch: The Palm as An Additional Input Modality on Commodity Smartphones},
booktitle = {Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct},
series = {MobileHCI '18},
year = {2018},
isbn = {978-1-4503-5941-2},
location = {Barcelona, Spain},
numpages = {4},
url = {https://doi.org/10.1145/3236112.3236163},
doi = {10.1145/3236112.3236163},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {palm, capacitive image, machine learning, smartphone},
}