A Camera-Based Gesture Recognition System for Mobile Devices

Diploma Thesis from the year 2007 in the subject Engineering - Computer Engineering, grade: 1, RWTH Aachen University, language: English, abstract: Modern mobile devices come equipped with an increasing set of features and capabilities, which challenge us to create new mobile applications with innovative interfaces. Mobile pervasive games, an emerging application field in ubiquitous computing, are a fruitful ground for the development of new and exciting applications for mobile devices. We present a novel spell-casting interface for mobile devices. This interface was developed in the context of the mobile pervasive city game REXplorer, an informational game for tourists, which was launched in Regensburg, Germany, in July 2007. Our interface is designed to allow users to "cast spells" by waving the REXplorer device in the air. A motion estimation algorithm processes image frames obtained from the device's on-board camera. The resulting motion data is then forwarded to a gesture recognizer. This enables us to determine which gesture (or "spell") was entered by the user. We use findings that we obtained in a theoretical analysis of motion estimation to adapt a motion estimation algorithm for use on our target device. We go on to present a simple, yet effective, approach to gesture recognition, based on finite state machines. To evaluate our interface, we conducted a controlled laboratory study and an evaluation of field data, which was collected during actual REXplorer play sessions. In addition to verifying our gesture recognition algorithm's acceptable gesture recognition rate, we demonstrated, by means of a comparative study, that the choice of on-screen feedback has a significant effect on the gesture recognition rate and thus on the user's ability to effectively use our interface. We conclude this thesis with a discussion of avenues of future research. Furthermore, we propose a strategy towards the creation of a framework for the a priori performance evaluatio