Intersemiotic translation: meaning-making in film and musical art

Keywords: emotion, culture, film adaptation, intersubjectivity, multisemioticity, music, semiotics, translational transformation


Our research is based on two blocks of material (film adaptation and film-related soundtrack). It aims to analyse intersemiotic translation of a literary work into a film which involves procedures of intralingual translation; look into the semiotic resources of cinematic discourse – visual, sound and light effects, non-verbal means of communication in cinema, symbolism in films, color, etc.; compare written descriptions of associations, emotions and sensations provided by amateur Ukrainian and professional music and film review English subjects to reveal the mechanisms of interpreting the multimodal texts. We assume that the use of transformations in the process of adaptation brings about changes in the verbal component, compresses the text, adds or omits information, etc.; the audiovisual component, however, compensates the reduction of information that was presented verbally in the literary work (Besedin, 2017). Semiotic units contribute to reconstruction of meaning (Krysanova, 2017, p. 25; Peirce, 2000). We also experimentally approach the intersemiotic translation of the film into the medium of a piece of music. Our hypothesis is that such an intersemiotic translation (Jacobson, 1959, p. 233) of a film into a soundtrack will evoke similar associations in amateur and expert recipients despite their different professional and cultural backgrounds. Ingarden’s idea (1937/1973) of a work of art as an intersubjectively accessible object and Hardy’s theory (1998) that views meaning-making as intersubjective spontaneous nonlinear dynamic proving that affect, intuition and sensations are more powerful than linear rational reasoning are in the core of the research.


Download data is not yet available.


Barrett, L. F., Quigley, K. S., Bliss-Moreau, E., & Aronson, K. R. (2004). Interoceptive sensitivity and self-reports of emotional experience. Social Psychology, 87(5), 684-97. DOI: .
Besedin, A. S. (2017). Tekhnologiya intersemioticheskogo i intrayazykovogo perevoda: ot romana k kinotekst [Intersemiotic and intralingual translation technology: from novel to film text]. Vestnik Volgogradskogo gosudarstvennogo universiteta. Seriya 2. Yazykoznaniye [Linguistics], 16(4), 215-221.
Brazgovskaya, Ye. Ye. (2014). Semiotika za predelami auditorii. Statia pyataya: Verbalizatsiya muzyki kak semioticheskoye priklyucheniye [Semiotics outside the audience. Article Five: Verbalization of music as a semiotic adventure]. Filolog, 26, 8-9.
Brazgovskaya, Ye. Ye. (2015). Vospriyatiye muzyki kak polilingvalnyy akt: Yaroslav Ivashkevich o Frederike Shopene [Perception of music as a multilingual act: Yaroslav Ivashkevich about Frederic Chopin]. Filologiya i iskusstvovedeniye. Retrieved from rticle/n/vospriyatie-muzyki-kak-polilingvalnyy-akt-yaroslav-ivashkevich-o-frederike-shopene.
Bryan, G. A., & Barrett, H. C. (2008). Vocal emotion recognition across disparate cultures. Journal of Cognition and Culture, 8, 135-148.
Budd, M. (1985). Music and the Emotions. The Philosophical Theories. London: Routledge.
Camera Angles: The Art of Manipulation. (n.d.). Retrieved from http://www.elementsofcinema. om/cinematography/camera-angles-and-composition/
Campbell, I. G. (1942). Basal emotional patterns expressible in music. The American Journal of Psychology, 55, 1-17. DOI: .
Chandler, D. (2017). Semiotics for Beginners. Retrieved from ocuments/S4B/sem02.html.
Chrzanowska-Kluczewska, E. (2016). Corporeal Imagination in the Reception of Verbal and Visual Artworks. In E. Chrzanowska-Kluczewska & O. Vorobyova (Eds.). Language – Literature – The Arts: A Cognitive- Semiotic Interface (pp. 71-94). Frankfurt am Main/Bern: Peter Lang.
Clayton da Silva, A. (2017). On Jakobson’s Intersemiotic Translations in Asterix Comics. Traduzione intersemiotica e nuove forme di testualità – Intersemiotic translation and new forms of textuality, 2, 71-81.
Cross, I., & Tolbert, E. (2009). Music and Meaning. In S. Hallam, I. Cross & M. Thaut (Eds.). The Oxford Handbook of Music Psychology (pp. 24-34). New York, NY: Oxford University Press.
Cross, I. (2012). Music as an Emergent Exaptation. In N. Bannan (Ed.). Music, Language, and Evolution (pp. 263-276). New York, NY: Oxford University Press.
Damasio, A. (1999). The Feeling of What Happens. New York: Harcourt.
Damasio, A. (2003). Looking for Spinoza. New York: Harcourt.
Di Paolo, E., & Thompson, E. (2014). The Enactive Approach. In L. Shapiro (Ed.). The Routledge Handbook of Embodied Cognition (pp. 68-78). London: Routledge Press.
Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3-4), 169-200. DOI:
Ellis, R., & Newton, N. (2012). Could Moving Others Be the Link between Emotion and Consciousness? In A. Foolen, U.M. Lüdtke, T.P. Racine & J. Zlatev (Eds.). Moving Ourselves, Moving Others. Motion and Emotion in Intersubjectivity, Consciousness and Language (pp. 57-80). Amsterdam: John Benjamins.
Fillmore, Ch. (1982). Frame Semantics. In The Linguistic Society of Korea (Ed.). Linguistics in the Morning Calm (pp. 111-137). Seoul: Hanshin.
Filmmaking 101: Camera Shot Types. (n.d.). Retrieved from explora/video/tips-and-solutions/filmmaking-101-camera-shot-types.
Fitzgerald, F. Sc. (2018). The Great Gatsby. Retreived from
Fritz, T., Jentschke, S., Gosselin, N., Sammler, D., Peretz, I., Turner, R., et al. (2009). Universal recognition of three basic emotions in music. Current Biology, 19, 573-576.
Gabrielsson, A. (2002). Emotion perceived and emotion felt: same or different? Musicae Scientiae, 5(1), Special Issue 2001-2002, 123-147. DOI:
Gabrielsson, A., & Juslin, P. N (1996). Emotional expression in music performance: between the performer’s intention and the listener’s experience. Psychology of Music, 24(1), 68-91. .
Hardy, Ch. (1998). Networks of Meaning: A Bridge between Mind and Matter. Westport, CT: Praeger.
Ingarden, R. (1937/1973). The Cognition of the Literary Work of Art. Transl. by R. E. Crawley & K.R. Olson. Evanston, IL: Northwestern University Press.
Izard, C. E. (1977). Humans Emotions. New York, NY: Plenum.
Jakobson, R. (1959). On Linguistic Aspects of Translation. In R. Brower (Ed.). On Translation (pp. 232-239). Cambridge, MA: Harvard University Press.
Jakobson, R. (1960). Linguistics and Poetics. In Th. Sebeok (Ed.). Style in Language (pp. 350-77). Cambridge: Massachusetts Institute of Technology Press.
Juslin, P. N. (2013). From Everyday Emotions to Aesthetic Emotions: toward a Unified Theory of Musical Emotions. Physics of Life Reviews, 10(3), 235-66.
Klyuyev, A. S. (2008). Filosofiya muzyki. Izbrannyye stati i materialy [The philosophy of music. Featured Articles and Materials]. SPb: Asterion Publ.
Kostyayev, A. I. (2010). Proyektsii temperaturnoy chuvstvitelnosti Cheloveka kulturnogo [Projections of the temperature sensitivity of the Cultural Man]. M.: Librokom.
Krysanova, T. A. (2017). Semiotychni aspekty kinodyskursu [Semiotic aspects of cinema discourse]. Visnyk KhNU im. V.N. Karazina. Inozemna filologiya, 86, 23-31.
Langacker, R. (1987). Foundations of Cognitive Grammar (Vol. I.). Stanford, CA: Stanford University Press.
Laukka, P., Eerola, T., Thingujam, N.S., Yamasaki, T., & Beller, G. (2013). Universal and culture-specific factors in the recognition and performance of musical affect expressions. Emotion, 13, 434-449.
MacDonald, R., Kreutz, G., & Mitchell, L. (2012). What is music, health, and wellbeing and why is it important? In R. MacDonald, G. Kreutz & L. Mitchell (Eds.). Music, Health, and Wellbeing (pp. 3-11). New York, NY: Oxford University Press.
Matito, F. D. (2019). La deriva hacia la burla en el contexto de las prácticas del teatro barroco español [A drift towards mockery in the context of the practices of Spanish baroque theatеr]. Hispanofila, Vol.185, 23-37. DOI: .
Panksepp, J. (1998). Affective Neuroscience: The Foundations of Human and Animal Emotions. New York: Oxford University Press.
Panksepp, J. (2000). The Neuro-evolutionary Cusp between Emotions and Cognitions: Implications for Understanding Consciousness and the Emergence of Unified Mind Science. Consciousness and Emotion, 1, 17-56.
Peirce, Ch. (1931-1935, 1958). The Collected Papers of C. S. Peirce. Cambridge, MA: Harvard University Press.
Peirce, Ch. (2000). Nachala pragmatizma [The Beginning of Pragmatism]. SPb.: Laboratoriya metafizicheskikh isledovaniy filosovskogo fakulteta Publ.
Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In R. Plutchik, & H. Kellerman (Eds.). Theories of Emotion. Emotion: Theory, Research, and Experience (Vol. 1) (pp. 3-33). New York, NY: Academic Press.
Plutchik, R. (1994). The Psychology and Biology of Emotion. New York, NY: Harper-Collins.
Rosenblatt, A., & Thickstun, J. (1994). Intuition and Consciousness. The Psychoanalytic Quarterly, 63(4), 696-714.
Schank, R., & Abelson, R. (1977). Script, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale, NJ: Laurence Erlbaum Associates.
Sheets-Johnstone, M. (2012). Fundamental and Inherently Interrelated Aspects of Animation. In A. Foolen, U. M. Lüdtke, T. P. Racine & J. Zlatev (Eds.). Moving Ourselves, Moving Others: Motion and Emotion in Intersubjectivity, Consciousness and Language (pp. 29-55). Amsterdam: John Benjamins.
Torop, P. (2008). Translation and semiotics. Σημειωτκή-Sign Systems Studies, 36 (2), 253-258.
Trevarthen, C. (1998). The Concept and Foundations of Infant Intersubjectivity. In St. Braten (Ed.). Intersubjective Communication and Emotion in Early Ontogeny (pp. 15-46). Cambridge, MA: Cambridge University Press.
Varela, F., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.
Wildfeuer, J., & Bateman, J. A. (2017). Conclusion: Film Text Analysis – A New Beginning? In J. Wildfeuer & J. A. Bateman (Eds.). Film Text Analysis: New Perspectives on the Analysis of Filmic Meaning (pp. 187-198). New York: Routledge.
Zlatev, J. (2008). Intersubjectivity: “What Makes Us Human?” In J. Zlatev, T.P. Racine, Ch. Sinha & E. Itkonen (Eds.). The Shared Mind: Perspectives on Intersubjectivity (pp. 1-17). Amsterdam: John Benjamins.
How to Cite
Lukianova, T., & Ilchenko, A. (2019). Intersemiotic translation: meaning-making in film and musical art. Cognition, Communication, Discourse, (19), 78-95.