TY - JOUR
T1 - Text-image complementarity and genre in English as foreign language textbooks
AU - Chang, Peichin
AU - Tsai, Hsin Jung
N1 - Publisher Copyright:
© 2021 Walter de Gruyter GmbH, Berlin/Boston.
PY - 2022/1/1
Y1 - 2022/1/1
N2 - Relating visual images to textual messages may have great potential in facilitating students' reading comprehension. The inevitable and important presence of visuals in textbooks obliges language teachers to exploit all semiotic resources to deepen students' understanding. However, analysis of how images interact with text in textbooks has been rare, and among the efforts it has generally been found that visuals and text often fail to achieve coherence. This study investigates whether and how text and image complement each other ideationally (i.e., the "what") by six sense relations (e.g., synonymy and hyponymy) and interpersonally (i.e., reader engagement) by the Mood system in ninth-grade English as a Foreign Language (EFL) textbooks to reach intersemiotic complementarity (IC). The results revealed that ideational rather than interpersonal IC is more frequent, where many more Participants (i.e., the nominal groups) than Processes (i.e., the verbal groups) in the texts find their visual complements. Ideational IC is particularly high in Information Reports while Recounts generally mark higher percentages of interpersonal IC. To accomplish ideational IC, repetition is most frequent, followed by hyponymy (i.e., general-specific relation) and collocation (i.e., relations that naturally co-occur). Distinct IC patterns also characterize the different editions of textbooks investigated, which may suggest their different potentials in catering to students of varying proficiencies.
AB - Relating visual images to textual messages may have great potential in facilitating students' reading comprehension. The inevitable and important presence of visuals in textbooks obliges language teachers to exploit all semiotic resources to deepen students' understanding. However, analysis of how images interact with text in textbooks has been rare, and among the efforts it has generally been found that visuals and text often fail to achieve coherence. This study investigates whether and how text and image complement each other ideationally (i.e., the "what") by six sense relations (e.g., synonymy and hyponymy) and interpersonally (i.e., reader engagement) by the Mood system in ninth-grade English as a Foreign Language (EFL) textbooks to reach intersemiotic complementarity (IC). The results revealed that ideational rather than interpersonal IC is more frequent, where many more Participants (i.e., the nominal groups) than Processes (i.e., the verbal groups) in the texts find their visual complements. Ideational IC is particularly high in Information Reports while Recounts generally mark higher percentages of interpersonal IC. To accomplish ideational IC, repetition is most frequent, followed by hyponymy (i.e., general-specific relation) and collocation (i.e., relations that naturally co-occur). Distinct IC patterns also characterize the different editions of textbooks investigated, which may suggest their different potentials in catering to students of varying proficiencies.
KW - English textbook
KW - genre
KW - intersemiotic complementarity
KW - multimodality
UR - http://www.scopus.com/inward/record.url?scp=85122635957&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85122635957&partnerID=8YFLogxK
U2 - 10.1515/sem-2020-0103
DO - 10.1515/sem-2020-0103
M3 - Article
AN - SCOPUS:85122635957
SN - 0037-1998
VL - 2022
SP - 53
EP - 80
JO - Semiotica
JF - Semiotica
IS - 244
ER -