List of Tables |
|
ix | |
List of Figures |
|
xi | |
Acknowledgements |
|
xvii | |
Preface |
|
xix | |
1 Introduction: Four Whys and a How |
|
1 | |
|
1.1 Learning to walk: framing issues and analytic focus |
|
|
3 | |
|
|
3 | |
|
|
7 | |
|
|
9 | |
|
|
11 | |
|
1.2 How can we analyse multimodal documents? |
|
|
13 | |
|
1.2.1 An orientation for analysis: empirical linguistics |
|
|
14 | |
|
1.2.2 A framework for empirical analysis: the GeM model |
|
|
15 | |
|
1.2.3 Structure of the book |
|
|
19 | |
2 Multimodal Documents and their Components |
|
21 | |
|
2.1 Starting points: how to find document parts? |
|
|
24 | |
|
2.2 The page as an object of interpretation |
|
|
27 | |
|
2.2.1 Interpretation within document design |
|
|
28 | |
|
2.2.2 Multimodal linguistics |
|
|
38 | |
|
2.3 The page as object of perception |
|
|
57 | |
|
|
65 | |
|
2.5 The Page as object of production |
|
|
74 | |
|
2.5.1 Describing a page for design |
|
|
75 | |
|
2.5.2 Describing a page for rendering |
|
|
85 | |
|
2.5.3 Producing a page from intentions: automatic document generation |
|
|
91 | |
|
2.6 Combining viewpoints on document parts |
|
|
103 | |
3 The GeM Model: Treating the Multimodal Page as a Multilayered Semiotic Artefact |
|
107 | |
|
3.1 The GeM Model: the base layer |
|
|
110 | |
|
3.2 The GeM presentation layers: the layout base |
|
|
115 | |
|
3.2.1 Layout segmentation: identification of layout units |
|
|
116 | |
|
3.2.2 Realisation information |
|
|
117 | |
|
|
121 | |
|
3.3 A more complicated example of layout analysis |
|
|
129 | |
|
3.3.1 The parts of the Louvre |
|
|
130 | |
|
3.3.2 The layout of the Louvre |
|
|
134 | |
|
|
142 | |
4 The Rhetorical Organisation of Multimodal Documents |
|
143 | |
|
4.1 Rhetoric and multimodal documents: our starting points |
|
|
144 | |
|
4.2 A brief introduction to Rhetorical Structure Theory |
|
|
146 | |
|
4.2.1 The RST rhetorical relations |
|
|
147 | |
|
4.2.2 The RST rhetorical structure |
|
|
150 | |
|
4.3 The move to multimodal RST: the GeM rhetorical layer |
|
|
151 | |
|
4.3.1 Andre's extension of RST |
|
|
152 | |
|
4.3.2 Problems with traditional multimodal RST |
|
|
155 | |
|
4.3.3 Multimodal relationals: subnuclear elaboration |
|
|
160 | |
|
4.4 Example analyses: rhetorical relations between layout units |
|
|
163 | |
|
4.4.1 Mismatches between layout structure and intended rhetorical structure |
|
|
166 | |
|
4.4.2 Explaining how to use a telephone |
|
|
171 | |
|
|
174 | |
5 Multimodal Documents and Genre |
|
177 | |
|
5.1 Perspectives on genre |
|
|
183 | |
|
5.1.1 Genre as social semiotic |
|
|
184 | |
|
5.1.2 Genre as social action |
|
|
188 | |
|
5.1.3 Genre: the need for fine-grained descriptions |
|
|
194 | |
|
5.2 The move to multimodal genre |
|
|
196 | |
|
5.2.1 Multimodal moves within linguistic and rhetorical approaches to genre |
|
|
197 | |
|
5.2.2 Moving in on genre from the visual |
|
|
201 | |
|
5.2.3 Cybergenres: a brief critique |
|
|
209 | |
|
|
217 | |
|
|
219 | |
|
|
223 | |
|
5.4 The multimodal genre space |
|
|
225 | |
|
5.5 Illustrations of genre: tracking change |
|
|
229 | |
|
5.5.1 Field guides across time |
|
|
229 | |
|
5.5.2 Wildlife fact files across time |
|
|
240 | |
|
5.6 Discussion and conclusion |
|
|
246 | |
6 Building Multimodal Document Corpora: the State of the Art |
|
249 | |
|
6.1 Corpus-based linguistics |
|
|
250 | |
|
6.2 The origin and representation of annotated corpora |
|
|
252 | |
|
6.2.1 Annotated corpora: early days |
|
|
252 | |
|
6.2.2 Applying XML to corpus design |
|
|
254 | |
|
6.2.3 Annotation problems with complex data |
|
|
260 | |
|
6.3 The move to multimodal corpora |
|
|
264 | |
|
6.4 The GeM model as a corpus annotation scheme |
|
|
267 | |
|
6.5 Conclusions and recommendations |
|
|
272 | |
7 Conclusions and Outlook: What Next? |
|
273 | |
Bibliography |
|
279 | |
Author Index |
|
301 | |
Subject Index |
|
307 | |