On the usability of multimodal interaction for mobile access to information services

The dynamics of our everyday life and the ubiquitous presence of information services create increasing demands for mobile access. Advanced mobile devices are now equipped with a sizeable screen, which opens the door to novel ways of interaction besides speech. This thesis investigates the advantages of multimodal interfaces that combine speech with pen interaction as compared to more traditional unimodal interfaces that only support either of those. Four user evaluations with a train timetable information service demonstrate that multimodal interaction is a promising alternative to unimodal interaction for mobile access to information services. The usability of a multimodal interface was shown to be affected by the extent of control a user has, the strategies offered to correct errors in an efficient and reliable way and the amount of experience that users have with multimodal interaction. The study underlines the importance of theories, standards and evaluation instruments for the successful deployment of future multimodal interfaces.