Can Automated Writing Evaluation Programs Help Students Improve Their English Writing?

Pei-ling Wang

Abstract


This study explores the effect of the automated writing evaluation (AWE) on Taiwanese students writing, and whether student improvement and their perception of the program are related. Instruments included a questionnaire, 735 essays analyzed in Criterion, and a pre/post essay. Two classes of 53 college students participated in the study. Descriptive statistics, paired-samples t-tests, Pearson correlation, effect size, and regression were used to analyze the data. Results showed that students improved significantly in terms of the length of the essay and the scores awarded by the machine and the human raters. However, among the five essays, the first essay is the only one showing a significant level of consistency between student improvement and student attitude, and the correlation declined dramatically after the first essay. To conclude, this study may be of importance in confirming the usefulness of the AWE functions such as recursive revising and instant scoring, as well as in providing teachers with a better understanding of how student beliefs about the Criterion program might relate to their writing performance.

 


Keywords


AWE, Criterion, writing

Full Text:

PDF

References


Brown, H. D. (2001). Teaching by Principles. Addison Wesley Longman.

Chen, C. F. E., & Cheng, W. Y. E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Teaching, 12 (2), 94-112.

Chen, H. J. (2006). Examining the scoring mechanism and feedback quality of My Access. Proceedings of Tamkang University Conference on Second Language Writing. Tamkang University, Taipei.

Chen, H. J., Chiu, T. L., & Liao, P. (2009). Analyzing the grammar feedback of two automated writing evaluation systems: My Access and Criterion. English Teaching and Learning, 33 (2), 1-43.

Cheng, W. Y. (2006). The Use of a Web-based Writing Program in College English Writing Classes in Taiwan— A Case Study of MyAccess. Unpublished Master’s thesis. National Kaohsiung First University of Science and Technology, Taiwan.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Creswell, J. W. (1994). Research design. Thousand Oaks, CA: SAGE Publications.

Dikli, S. (2006). An overview of automated scoring of essays. The Journal of Technology, Learning, and Assessment, 5 (1), 1-36. Retrieved from http://escholarship.bc.edu/ojs/index.php/jtla/article/view/1640

Elbow, P. (1973).Writing without Teachers. New York: Oxford University Press.

Flinn, J. (1986). The role of instruction in revising with computers: Forming a construct for good writing (ED 274963).

Frost, K. L. (2008). The effects of automated essay scoring as a high school classroom intervention. Unpublished Doctoral dissertation. University of Nevada, Las Vegas, USA.

Grimes, D. C. (2008). Middle school use of automated writing evaluation: A multi-site case study. Unpublished Doctoral dissertation. University of California, Irvine, USA.

Grimes, D. C., & Warschauer, M. (2006). Automated essay scoring in the classroom. American Educational Research Association (AERA) Annual Conference, San Francisco, CA, USA.

LinHuang, S. H. (2010).The exploitation of e-writing in an EFL classroom: Potential and challenges. Unpublished Master’s thesis. I-Shou University, Taiwan.

McCurry, D. (2010). Can machine scoring deal with broad and open writing tests as well as human readers? Assessing Writing, 15, 118-129. http://dx.doi.org/10.1016/j.asw.2010.04.002

Moseley, M. H. (2006).Creating recursive writers in middle school: the effect of a writing program on student revision practices. Unpublished Doctoral dissertation. Capella University, USA.

National Commission on Writing (2003). The Neglected “R”: The Need for a Writing Revolution. New York, NY, College Entrance Examination Board.

Otoshi, J. (2005). An analysis of the use of Criterion in a writing classroom in Japan. The JALT CALL Journal, 1(1), 30-38.

Phillips, S. M. (2007). Automated essay scoring: A literature review. TASA Institute, Society for the Advancement of Excellent in Education, 1-70.

Taylor, J. (1996). Computers: Tools of oppression, tools of liberation. A paper presented at the annual meeting of the Conference on College Composition and Communication, Milwaukee (ED 434350).

Tsai, P. Y. (2010). Students’ biggest writing problem- they never write! Retrieved from http://mag.udn.com/mag/campus/storypage.jsp?f_MAIN_ID=13&f_SUB_ID=1259&f_ART_ID=290730

Wang, Y. J. (2011). Exploring the effect of using automated writing evaluation in Taiwanese EFL students’ writing. Unpublished Master’s thesis. I-Shou University, Taiwan.

Wang, J., & Brown, M. S. (2007). Automated essay scoring versus human scoring: A comparative study. The Journal of Technology, Learning, and Assessment, 6(2), 1-28.

Warschauer, M., & Grimes, D. C. (2008). Automated writing assessment in the classroom. Pedagogies, 3, 22-36.

Williamson, M. M., & Pence, P. (1989). Word processing and student writers. In B. K. Britton & S. M. Glynn (Eds.) (1989). Computer writing environments: Theory, research, and design. (pp.93-127). Hillsdale, NJ: Lawrence Erlbaum.

Yang, N. D. (2004). Using My Access in EFL writing. Proceedings of the 2004 International Conference and Workshop on TEFL & Applied Linguistics (pp. 550-564). Taipei, Taiwan: Ming Chuan University.

Yu, Y. T., & Yeh, Y. L. (2003). Computerized feedback and bilingual concordance for EFL college students’ writing. Proceedings of the 2003 International Conference on English Teaching and Learning in the Republic of China (pp. 35-48). Taipei, Taiwan: Crane.

Zamel, V. (1982).Writing: the process of discovering meaning. TESOL Quarterly, 16, 195-209. http://dx.doi.org/10.2307/3586792




DOI: https://doi.org/10.7575/ijalel.v.2n.1p.6

Refbacks

  • There are currently no refbacks.


Bookmark and Share


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2012-2023 (CC-BY) Australian International Academic Centre PTY.LTD. 

International Journal of Applied Linguistics and English Literature

To make sure that you can receive messages from us, please add the journal emails into your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', check your 'bulk mail' or 'junk mail' folders.