InfiniTouch - Hand Grip and Finger Recognition
InfiniTouch is the first smartphone that enables touch input on the whole device surface and identifies the fingers touching the device with deep learning. The prototype morphs two rooted LG Nexus 5 devices and 37 copper plates as capacitive sensors into one handheld device. Different fingers are identified with a convolutional neural network (CNN) which uses the raw capacitive measurements as input. Fingers are identified with an accuracy of 95.78% while their 3D position can be estimated with an MAE of 0.74cm. This project was presented at the ACM User Interface Software and Technology Symposium (UIST ’18) in October 2018.
Smartphone Prototype
The smartphone prototype is fully reproducible with publicly available components. The PCBs to morph the LG Nexus 5 devices, the 3D models of the handheld device and hardware container, a list of all hardware components, and the model to run the finger identification are all available on the project’s GitHub page.
Press Coverage
This project was covered in online magazines and communities including the Arduino Blog (2018), Hackster.io (2018), and open-electronics.org (2018).
Publications
Huy Viet Le, Sven Mayer, and Niels Henze. 2018. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). ACM, Berlin, Germany.
@inproceedings{le2018infinitouch,
author = {Le, Huy Viet and Mayer, Sven and Henze, Niels},
title = {InfiniTouch: Finger-Aware Interaction on Fully Touch Sensitive Smartphones},
booktitle = {Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology},
series = {UIST '18},
year = {2018},
isbn = {978-1-4503-5948-1},
location = {Berlin, Germany},
pages = {779--792},
numpages = {14},
url = {http://doi.acm.org/10.1145/3242587.3242605},
doi = {10.1145/3242587.3242605},
acmid = {3242605},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {finger-aware interaction, machine learning, touchscreen},
}
Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2017. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, Vienna, Austria.
@inproceedings{le2017smartphone,
author = {Le, Huy Viet and Mayer, Sven and Bader, Patrick and Henze, Niels},
title = {A Smartphone Prototype for Touch Interaction on the Whole Device Surface},
booktitle = {Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services},
series = {MobileHCI '17},
year = {2017},
isbn = {978-1-4503-5075-4},
location = {Vienna, Austria},
pages = {1001--1008},
articleno = {100},
numpages = {8},
url = {http://doi.acm.org/10.1145/3098279.3122143},
doi = {10.1145/3098279.3122143},
acmid = {3122143},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {capacitive, full-touch phone, mobile, prototype},
}