Gesture manipulation of a robotic arm



This project explores human-controlled robotics. We manipulate a robotic arm in real-time using five-finger gestures, with a force-sensitive touch interface.


This project was created for two reasons: First, is to evaluate Sensel’s new force-sensitive touch interface. Second, is a general exploration of a magical sensation; The use of touch gestures to animate a robot.


There are four hand gestures that control the robot arm’s movements:

Force down:



Forward / Backward:

Me too?

To make this project happen, the following items were used:

You can get this project up and running in no time! The code is located here.
First, program the Arduino with the “sketch” code. Next, run the Processing application, and control the robot with five fingers!


Sensel is designing next-generation input devices. I’ve had the pleasure of working with their pressure-sensitive, multi-touch interface in this project. The video below is a promo teaser for their upcoming crowd-funding campaign launch.

Check out their offerings, and stay tuned for their kickstarter: