Gesture-Based Computing

Time-to-Adoption: Four to Five Years

Thanks in part to the Nintendo Wii, the Apple iPhone and the iPad, many people now have some immediate experience with gesture-based computing as a means for interacting with a computer. The proliferation of games and devices that incorporate easy and intuitive gestural interactions will certainly continue, bringing with it a new era of user interface design that moves well beyond the keyboard and mouse. While the full realization of the potential of gesture-based computing remains several years away, especially in education, its significance cannot be underestimated, especially for a new generation of students accustomed to touching, tapping, swiping, jumping, and moving as a means of engaging with information.

Gesture-based computing is changing the ways that we interact with computers, both physically and mechanically. As such, it is at once transformative and disruptive. Researchers and developers are just beginning to gain a sense of the cognitive and cultural dimensions of gesture-based communicating, and the full realization of the potential of gesture-based computing within higher education will require intensive interdisciplinary collaborations and innovative thinking about the very nature of teaching, learning, and communicating.

Relevance for Teaching, Learning, or Creative Inquiry

  • Gestural interfaces allow users to easily perform precise manipulations that can be difficult to manage with a mouse.
  • Gesture-based games like those developed by researchers at Georgia Tech University can help deaf children learn linguistics at a critical time of language development.
  • Large multi-touch displays support collaborative work, allowing multiple users to interact with content simultaneously.

In Practice

For Further Reading

Math That Moves: Schools Embrace the iPad
(Winnie Hu,, 4 January 2011.) This New York Times article gives examples of how different schools are integrating iPads into the classroom. The author looks and the pros, cons, and promise of this new form of computing for impacting education in the K12 space.

Some Thoughts from the Classroom on iPad 2
(Fraser Spiers,, 2 March 2011.) Well known iOS Developer and educator Fraser Spiers shares his insight on why the iPad 2 is a powerful tool for education, citing the maturing iOS and the apps that can be used on the device.

What Publishers Can and Should Learn from "The Elements"
(Mac Slocum, O’Reilly Radar, 12 August 2010.) This interview with Theodore Gray of Wolfram Research explores the development and thinking behind the groundbreaking iPad eBook The Elements, which makes extensive use of gesture-based interfaces to provide a truly interactive experience with the content.