A model for multimodal representation and inference
pp. 375-386
in: Ray Paton, Irene Neilson (eds), Visual representations and interpretations, Berlin, Springer, 1999Abstract
In this paper some applications of a theory for representation and inference in multimodal scenarios is presented. The theory is focused on the relation between natural language and graphical expressions. First, a brief introduction to the representational structures of the multimodal system is presented. Then, a number of multimodal inferences supported by the system are illustrated. These examples show how the multimodal system of representation can support the definition and use of graphical languages, perceptual inferences for problem-solving and interpretation of multimodal messages. Finally, the intuitive notion of modality underlying this research is discussed.