Interacting Naturally in Virtual Environments - 1 views
-
Sven A. Miller on 27 Nov 10Current methods for controlling one's avatar in a virtual environment interacting with intelligent virtual agents (IVAs) are unnatural, typically requiring a complex set of keyboard commands for controlling your avatar, and dialog menus for interacting with IVAs. Recent advances in markerless body and motion tracking, speech and gesture recognition technologies, coupled with intelligent agent/behavior modeling and speech synthesis technologies, now make it possible to naturally control one's avatar through the movement of one's body and to interact with IVAs through speech and gesture. These capabilities are now just beginning to emerge in the arena of computer gaming, and offer great promise for military training. In this paper we describe our recent work integrating motion capture, gesture recognition, speech recognition, natural language understanding, and intelligent agent/behavior modeling technologies to produce more natural mechanisms for avatar control as well as IVAs that are able to understand relatively unconstrained speech and recognize human movement and gesture. We illustrate these capabilities within the domain of roadside security checkpoint training, where trainees are able to gesture (e.g., wave forward, stop, point to a location) and speak to IVAs (drivers and passengers) in the scene.
-
Sven A. Miller on 27 Nov 10This shall be one of our main inspiration within the framework of our experiment for LEARNING WITH MEDIA, during which we performed a session of "Start the Party" for "Playstation Move" on the Playstation 3, in comparison to our previous experiment with "Wii Sports"…