Claire Murray
About
No profileSessions
Testing and Evaluation Assessing the validity of essay marking rubrics more
Sat, Jul 9, 11:45-12:10 Asia/Tokyo
As English high school curricula becomes increasingly communication-oriented, it is becoming more necessary to develop university entrance tests which assess students’ ability to produce target language based on communicative goals rather than to translate between languages or select correct answers. A potential problem with these more communication-oriented test questions is they may risk sacrificing reliability for validity; however, the use of rubrics can ensure that both reliability and validity remain high (Jonsson and Svingby, 2007). This presentation looks at the results of a preliminary study to determine if a university entrance exam rubric results in high inter-rater reliability. The study looks at the test scores of three types of markers: 1) those trained to apply the rubric; 2) those who have seen the rubric but have not been trained to apply it; and 3) those who have not seen the rubric. It aims to answer the following questions: Does the rubric achieve a Cohen's kappa value greater than 0.7 for inter-rater reliability 1. between trained markers? 2. between trained and untrained markers? 3. between trained markers and markers who have not seen the rubric? The findings of this study will interest educators involved in test and assessment design.
Vocabulary Facilitating English academic vocabulary learning using fictional graded readers more
Sat, Jul 9, 14:40-15:05 Asia/Tokyo
Given the importance of learning L2 vocabulary in context (Webb, 2008), academic texts would seem to be the most useful way to support learners’ academic vocabulary learning. However, for teachers using word lists such as Coxhead’s (2000) academic word list (AWL), finding authentic materials with sufficient academic vocabulary range and frequency to support course-related academic vocabulary learning can be a challenge. Against this background, the presenter wrote a fictional graded reader series (‘The AWL Readers’) in an attempt to make English academic vocabulary learning more stimulating and (hopefully) more effective for his students. The AWL Readers follow the adventures (and misadventures) of a fictional university student and her unusual friend, and include all 570 AWL words (with spaced repetition). One of the goals in creating the AWL Readers was to add to recent research which is rethinking the assumption that academic vocabulary learning should primarily be facilitated through reading academic (rather than fictional) texts (Krashen, 2010; McQuillan, 2020). This presentation will discuss: (1) how and why the AWL Readers were created; (2) the results of our preliminary study into their effectiveness as a vocabulary learning tool; and (3) their possible usefulness in other teaching and learning contexts.