Acceleration and deceleration at constant speed: systematic modulation of motion perception by kinematic sonification

Abstract

Many domains of human behavior are based on multisensory representations. Knowledge about the principles of multisensory integration is useful to configure real‐time movement information for the online support of perceptuomotor processes (motor perception, control, and learning). A powerful method for generating real‐time information is movement sonification. Remarkable evidence exists on movement‐acoustic real‐time information being effective in behavioral domains (music training, motor rehabilitation, sports). Here, we investigate whether and how biological motion perception can be enhanced, substituted, or modulated by kinematic sonification, with a focus on pitch coding. We work with gross motor cyclic movements and investigate the effectiveness of pitch scaling and consistent transposition on audiovisual motor perception accuracy (Experiment A). Beyond that, a new kind of audiovisual stimulus with inconsistent pitch transposition is used to produce a directed modulation of the integrated audiovisual percept (Experiment B). Results from Experiment A indicate pitch being powerful for mediating kinematic information to enhance motor perception and substituting information between perceptual modalities, even exceeding visual performance. Beyond these findings, results from Experiment B indicate that visual estimations of movement velocity can be enhanced or reduced auditorily. Movement sonification used for reshaping intermodal adjustments should be a powerful new tool for subconsciously shaping human movement patterns in the future.