Multimodality  book cover
1st Edition


Edited By

Sigrid Norris

ISBN 9781138845503
Published December 23, 2015 by Routledge
1778 Pages

FREE Standard Shipping
SAVE $280.00
was $1,400.00
USD $1,120.00

Prices & shipping based on shipping country


Book Description

Multimodality is a fast growing area of inquiry as the interest in modes beyond, but always also including language, such as layout, gesture, gaze, or body posture is increasing. A conglomeration of research from various fields interested in examining language as it is embedded in a vast array of other modes, rather than a coherant field of inquiry, this collection will showcase the major achievements in this multidisciplinary field, through the inclusion of theoretically, methodologically and practically important works Topics will range from language studies to education and classroom discourse, from sociolinguistic inquiries into work place practice to communication, from psycholinguistics and gesture studies to systemic functional grammatical inquiries.

Table of Contents

Volume I
The Early Developments in Multimodal Inquiry up to 1999





Part 1: Some Forerunners of Multimodality: Early Days

1. J. Rüesch and W. Kees, extracts from Nonverbal Communication: Notes on the Visual Perception of Human Relations (University of California Press, 1956), pp. 57–65, 76–86, 89–95, 96–107.

2. M. Argyle and J. Dean, ‘Eye-Contact, Distance and Affiliation’, Sociometry, 1965, 28, 3, 289–304.

3. R. Birdwhilstell, extract from Kinesics and Context: Essays on Body Motion Communication (University of Pennsylvania Press, 1970), pp. 180–91.

4. E. Goffman, ‘Gender Commercials’, in Gender Advertisements (Macmillan, 1979), pp. 24–31.

5. C. Goodwin, ‘Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning’, Sociological Inquiry, 1980, 50, 272–302.

6. D. McNeill, ‘Action, Thought and Language’, Cognition, 1981, 10, 201–8.


Part 2: Early Multimodality: The 1990s

7. G. Kress, ‘Against Arbitrariness: The Social Production of the Sign as a Foundational Issue in Critical Discourse Analysis’, Discourse & Society, 1993, 4, 169–91.

8. C. Forceville, ‘Pictorial Metaphor in Advertisements’, Metaphor and Symbolic Activity, 1994, 9, 1, 1–29.

9. C. Goodwin, ‘Professional Vision’, American Anthropologist, 1994, 96, 3, 606–33.

10. M. O’Toole, ‘Semiotics at Work’, in The Languages of Displayed Art (Leicester University Press, 1994), pp. 3–31.

11. G. Kress and T. van Leeuwen, ‘The Meaning of Composition’, in Reading Images: The Grammar of Visual Design (Routledge, 1996), pp. 181–92.

12. J. Streeck, ‘How to Do Things with Things: Objets Trouvés and Symbolization’, Human Studies, 1996, 19, 365–84.

13. A. Kendon, ‘Gesture’, Annual Review of Anthropology, 1997, 26, 109–28.

14. G. Kress and T. van Leeuwen, ‘Front Pages: (The Critical) Analysis of Newspaper Layout’, in A. Bell and P. Garrett (eds.), Approaches to Media Discourse (Blackwell, 1998), pp. 186–219.

15. R. Martinec, ‘Cohesion in Action’, Semiotica, 1998, 120, 1–2, 161–80.

16. T. Royce, ‘Synergy on the Page: Exploring Intersemiotic Complementarity in Page-Based Multimodal Text’, Japan Association Systemic Functional Linguistics Occasional Papers, 1998, 1, 1, 25–49.

17. R. Scollon, ‘Mediated Action as Social Practice’, in Mediated Discourse as Social Interaction: A Study of News Discourse (Longman, 1998), pp. 3–18.

18. K. L. O’Halloran, ‘Towards a Systemic Functional Analysis of Multisemiotic Mathematics Texts’, Semiotica, 1999, 124, 1/2, 1–29.

19. T. van Leeuwen, The Modality of Sound’ extract in Speech, Music, Sound (Macmillan, 1999), pp. 170–7.


Volume II
Multimodality—The Beginning of a New Area of Research: 2000–5





Part 1: A Call for Multimodal Theory

20. G. Kress, ‘Design and Transformation: New Theories of Meaning’, in B. Cope and M. Kalantzis (eds.), Multiliteracies: Literacy Learning and the Design of Social Futures (Psychology Press, 2000), pp. 153–161.

21. G. Kress, ‘Multimodality’, in B. Cope and M. Kalantzis (eds.), Multiliteracies: Literacy Learning and the Design of Social Futures (Psychology Press, 2000), pp. 182–202.


Part 2: Early Multimodality: 2000 and 2001

22. J. L. Lemke, ‘Across the Scales of Time: Artifacts, Activities, and Meanings in Ecosocial Systems’, Mind, Culture, and Activity, 2000, 7, 4, 273–90.

23. G. Kress and T. van Leeuwen, ‘Introduction’ and ‘Issues for the Mutimodal Agenda’, in Multimodal Discourse: The Modes and Media of Contemporary Communication (Arnold, 2001), pp. 1–23, 111–33.

24. R. Scollon, ‘Action and Text: Towards an Integrated Understanding of the Place of Text in Social (Inter)action, Mediated Discourse Analysis and the Problem of Socail Interaction’, in R. Wodak and M. Meyer (eds.), Methods of Critical Discourse Analysis (Sage, 2001), pp. 139–83.

25. Hartmut Stöckl, ‘Texts with a View: Windows onto the World. Notes on the Textuality of Pictures’, in W. Thiele, A. Neubert and C. Todenhagen (eds.), Text – Varieties – Translation (ZAA Studies, 2001), pp. 81–107.


Part 3: Building a Foundation: 2002–2004

26. C. Jewitt, ‘The Move from Page to Screen: The Multimodal Reshaping of School English’, Journal of Visual Communication, 2002, 1, 2, 171–96.

27. G. Kress and T. van Leeuwen, ‘Colour as a Semiotic Mode: Notes for a Grammar of Colour’, Visual Communication, 2002, 1, 343–68.

28. S. Norris, ‘The Implication of Visual Research for Discourse Analysis: Transcription Beyond Language’, Visual Communication, 2002, 1, 97–121.

29. J. Whalen, M. Whalen, and K. Henderson, ‘Improvisational Choreography in Teleservice Work’, British Journal of Sociology, 2002, 53, 2, 239–58.

30. R. Iedema, ‘Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-semiotic Practice’, Visual Communication, 2003, 1, 3, 29–57.

31. R. Scollon and S. Wong Scollon, ‘Geosemiotics’, in Discourses in Place: Language in the Material World (Routledge, 2003), pp. 1–24.

32. A. Baldry, ‘Phase and Transition, Type and Instance: Patterns in Media Texts as Seen Through a Multimodal Enhancer’, in K. L. O’Halloran (ed.), Multimodal Discourse Analysis (Continuum, 2004), pp. 83–108.

33. A. Hofinger and E. Ventola, ‘Multimodality in Operation: Language and Picture in a Museum’, in E. Ventola, C. Charles, and M. Kaltenbacher (eds.), Perspectives on Multimodality (John Benjamins, 2004), pp. 193–209.

34. S. Norris, ‘Multimodal Discourse Analysis: A Conceptual Framework’, in P. Levine and R. Scollon (eds.), Discourse and Technology: Multimodal Discourse Analysis (Georgetown University Press, 2004), pp. 101–15.

35. M. O’Toole, ‘Opera Ludentes: The Sydney Opera House at Work and Play’, in K. L. O’Halloran (ed.), Multimodal Discourse Analysis (Continuum, 2004), pp. 11–27.


Part 4: Developments: 2004–2005

36. R. Scollon and P. Levine, ‘Multimodal Discourse Analysis as the Confluence of Discourse and Technology’, in P. Levine and R. Scollon (eds.), Discourse and Technology: Multimodal Discourse Analysis (Georgetown University Press, 2004), pp. 1–6.

37. T. Van Leeuwen, ‘Multimodality, Genre and Design’, in S. Norris and R. H. Jones (eds.), Discourse in Action: Introducing Mediated Discourse Analysis (Routledge, 2005), pp. 73–93.


Volume III: Multimodal Inquiry becomes More Established: 2006–11





38. L. Mondada, ‘Participants’ Online Analysis and Multimodal Practices: Projecting the End of the Turn and the Closing of the Sequence’, Discourse Studies, 2006, 8, 1, 117–29.

39. C. D. Maier, C. Kampf and P. Kastberg, ‘Multimodal Analysis: An Integrative Approach for Scientific Visualizing on the Web’, Journal of Technical Writing and Communication, 2007, 37, 4, 453–78.

40. K. L. O’Halloran, ‘Systemic Functional Multimodal Discourse Analysis (SF-MDA) Approach to Mathematics, Grammar and Literacy’, in A. McCabe, M. O’Donnell, and R. Whittaker (eds.), Advances in Language and Education (Continuum, 2007), pp. 75–100.

41. K. Pahl, ‘Creativity in Events and Practices: A Lens for Understanding Children’s Multimodal Texts’, Literacy, 2007, 41, 86–92.

42. C. Jewitt, ‘Multimodality and Literacy in School Classroom’, AERA Review of Research in Education, 2008, 32, 241–67.

43. J. A. Bateman, Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents (Palgrave, 2008), pp. 107–42.

44. R. Flewitt, R. Hampel, M. Hauck and L. Lancaster, ‘What are Multimodal Data and Transcription?’, in C. Jewitt (ed.), The Routledge Handbook of Multimodal Analysis (Routledge, 2009), pp. 40–53.

45. R. H. Jones, ‘Technology and Sites of Display’, in C. Jewitt (ed.), The Routledge Handbook of Multimodal Analysis (Routledge, 2009), pp. 114–26.

46. G. Kress, ‘What is a Mode?’, in C. Jewitt (ed.), The Routledge Handbook of Multimodal Analysis (Routledge, 2009), pp. 54–67.

47. S. Norris, ‘Modal Density and Modal Configurations: Multimodal Actions’, in C. Jewitt (ed.), The Routledge Handbook of Multimodal Analysis (Routledge, 2009), pp. 78–90.

48. M. Stenglin, ‘Space Odyssey: Towards a Social Semiotic Model of Three-dimensional Space’, Visual Communication, 2009, 8, 1, 35–64.

49. T. van Leeuwen, ‘The World According to Playmobil’, Semiotica, 2009, 173, 299–315.

50. G. Abousnnouga and D. Machin, ‘Analysing the Language of War Monuments’, Visual Communication, 2010, 9, 2, 131–49.

51. N. Al Zidjaly, ‘Can Art (Powerpoint) Lead to Social Change? A Mediated Multimodal Inquiry’, Multimodal Communication, 2011, 1, 1, 65–82.

52. E. Djonov and T. van Leeuwen, ‘The Semiotics of Texture: From Tactile to Visual’, Visual Communication, 2011, 10, 4, 541–64.

53. J. Fox and N. Artemeva, ‘The Cinematic Art of Teaching University Mathematics: Chalk Talk as Embodied Practice’, Multimodal Communication, 2011, 1, 1, 83–103.

54. S. Norris, ‘Three Hierarchical Positions of Deictic Gesture in Relation to Spoken Language: A Multimodal Interaction Analysis’, Visual Communication, 2011, 10, 2, 129–47.

55. S. Pietikäinen, P. Lane, H. Salo and S. Laihiala-Kankainen, ‘Frozen Actions in the Arctic Linguistic Landscape: A Nexus Analysis of Language Processes in Visual Space’, International Journal of Multilingualism, 2011, 8, 4, 277–98.


Volume IV: Multimodality Growing Rapidly: 2012–14





56. C. D. Maier, ‘Closer to Nature: A Case Study of the Multifunctional Selection of Moving Images in an Environmental Corporate Video’, Multimodal Communication 1, 3, 2012, 233-250.

57. K. L. O’Halloran, A. Podlasov, A. Chua and Marissa K. L. E., ‘Interactive Software for Multimodal Analysis’, Visual Communication, 2012, 11, 3, 363–81.

58. P. White, ‘Multimodality’s Challenge to Marketing Theory: A Discussion’, Multimodal Communication, 2012, 1, 3, 305–23.

59. J. Wildfeuer, ‘Intersemiosis in Film: Towards a New Organisation of Semiotic Resources in Multimodal Filmic Text’, Multimodal Communication, 2012, 1, 3, 277–304.

60. W. L. Bowcher, J. Yameng Liang and S. Wen, ‘The Multimodal Construal of the Experiential Domain of Recipes in Japanese and Chinese’, Semiotica, 2013, 197, 233–65.

61. E. Djonov and T. van Leeuwen, ‘Between the Grid and Composition: Layout in PowerPoint’s Design and Use’, Semiotica, 2013, 197, 1–34.

62. J. Geenen, ‘Actionary Pertinence: Space to Place in Kitesurfing’, Multimodal Communication, 2013, 2, 2, 123–53.

63. D. Machin, ‘Towards a Social Semiotics of Rhythm in Popular Music’, Semiotica, 2013, 197, 119–40.

64. B. Makboon, ‘The "Chosen One": Depicting Religious Belief Through Gestures’, Multimodal Communication, 2013, 2, 1, 171–94.

65. S. Norris, ‘What is a Mode? Smell, Olfactory Perception, and the Notion of Mode in Multimodal Mediated Theory’, Multimodal Communication, 2013, 2, 2, 155–69.

66. J. Pirini, ‘Analysing Business Coaching: Using Modal Density as a Methodological Tool’, Multimodal Communication, 2013, 2, 2, 195–215.

67. J. Rowsell, G. Kress and B. Street, ‘Visual Optics: Interpreting Body Art, Three Ways’, Visual Communication, 2013, 12, 1, 97–122.

68. G. Ferre, ‘Multimodal Hyperbole’, Multimodal Communication, 2014, 3, 1, 25–50.

69. R. H. Jones, ‘The Multimodal Dimension of Claims in Food Packaging’, Multimodal Communication, 2014, 3, 1, 1–11.

70. B. König and E. Lick, ‘Wine Labels in Austrian Food Retail Stores: A Semiotic Analysis of Multimodal Red Wine Labels’, Semiotica, 2014, 200, 313–34.

71. L. Mondada, ‘Instructions in the Operating Room: How the Surgeon Directs Their Assistant’s Hands’, Discourse Studies, 2014, 16, 2, 131–61.

72. E. Tapio, ‘The Marginalisation of Finely Tuned Semiotic Practices and Misunderstandings in Relation to (Signed) Languages and Deafness’, Multimodal Communication, 2014, 3, 2, 131–42.

73. A. Zemel and T. Koschmann, ‘"Put Your Fingers Right in Here": Learnability and Instructed Experience’, Discourse Studies 16, 2, 2014, 163-183.



View More