What does PISA Assess in Reading Literacy? Misconceptions and Misuses

Tahir Tağa

Abstract


In the 21st century, international interaction in social, economic, cultural, and educational fields has increased. Consequently, international standards have become essential in national education policies, reforms, and practices. As an international assessment, PISA has started to function as a prominent tool in this regard. However, the impact of PISA differs across the participating countries, depending on how the concept, methodology, and practices are handled. One of the domains where this difference is seen is reading literacy. Although this domain expresses a broader and richer phenomenon, the content and scope of the concept are not accurately understood, and its influences vary across the participating countries. In Türkiye, reading literacy is mainly considered and discussed in the scope of Turkish language lessons. This perspective, which focuses on the Turkish language lessons, leads to misunderstanding of the issue and the inaccurate change and transformation of the curriculum and content. Attempting to succeed in this domain through test language lessons deepens the problems instead of solving them. In this article, misconceptions and misuses about reading literacy are explained based on a literature review, and it is pointed out that reading literacy should be addressed in a broader context, including the curricula of other subjects, rather than test language lessons.

Keywords


PISA, Reading, Reading Literacy, Test Language Lesson, International Assessment

Full Text:

PDF

References


Aktaş, E. (2022). 2018-2021 LGS Türkçe sorularının PISA okuma yeterlik düzeylerine göre değerlendirilmesi [Evaluation of 2018-2021 LGS Turkish questions according to PISA reading proficiency levels]. Uluslararası Türkçe Edebiyat Kültür Eğitim (TEKE) Dergisi, 11(1), 258–276. Retrieved from https://dergipark.org.tr/tr/pub/teke/issue/68990/1091168

Arffman, I. (2010). Equivalence of translations in international reading literacy studies. Scandinavian Journal of Educational Research, 54(1), 37–59. https://doi.org/10.1080/00313830903488460

Aşıcı, M., Baysal, N., & Şahenk Erkan, S. S. (2012). Türkiyede yapılan 2009 PISA ve seviye belirleme sınavındaki (SBS) okuma becerileri sorularının karşılaştırılması [A comparison of the reading comprehension questions in PISA 2009 and national level determination exam in Turkey]. Journal of Research in Education and Teaching, 1(2), 210–217.

Batur, Z., & Ulutaş, M. (2013). PISA ile Türkçe öğretim programındaki okuduğunu anlama kazanımlarının örtüşme düzeylerinin incelenmesi [An analysis of the correspondence levels between Turkish reading proficiency objectives and the PISA scale]. The Journal of Academic Social Science Studies, 6(3), 1549–1562. https://doi.org/10.9761/jasss_651

Benzer, A. (2019). Türkçe ders kitaplarının PISA okuma yeterlik düzeyleri ile imtihanı [Challenge PISA reading proficiency levels of Turkish textbooks]. Research in Reading and Writing Instruction, 7(2), 96–109. https://doi.org/10.35233/oyea.659740

Bodin, A. (2007). What does PISA really assess? What does it not? A French view. In S. T. Hopmann, G. Brinek, & M. Retzl (Eds.), PISA according to PISA: Does PISA keep what it promises? (pp. 21–56). LIT VERLAG.

Bozkurt, B. Ü., Uzun, G. L., & Lee, Y. (2015). Korece ve Türkçe ders kitaplarındaki metin sonu sorularının karşılaştırılması: PISA 2009 sonuçlarına dönük bir tartışma [A comparison of reading comprehension questions in Korean and Turkish textbooks: A discussion on PISA 2009 results]. International Journal of Language Academy, 3(9), 295–313. https://doi.org/10.18033/ijla.327

Breakspear, S. (2012). The policy impact of PISA: An exploration of the normative effects of İnternational benchmarking in school system performance. In OECD Education Working Papers (No. 71). OECD Publishing. https://doi.org/https://doi.org/10.1787/5k9fdfqffr28-en

Breakspear, S. (2014). How does PISA shape education policy making? Why how we measure learning determines what counts in education. Centre for Strategic Education.

Carroll, M., & Benton, T. (2018). The link between subject choices and achievement at GCSE and performance in PISA 2015. Cambridge Assessment. Retrieved from https://www.cambridgeassessment.org.uk/Images/517876-the-link-between-subject-choices-and-achievement-at-gcse-and-performance-in-pisa-2015.pdf

Ceyhan, E. (2019). PİSA 2012 okuma becerileri ölçeğinin, uygulama dili doğrultusunda belirlenen ülkeler arasında ölçme değişmezliğinin incelenmesi [Assessing measurement invariance of PISA 2012 reading literacy scale among the countries determined in accordance with the language of application]. (Master’s thesis). Retrieved from Turkey National Thesis Center. (Thesis No. 572502)

Common Core State Standards Initiative. (2010). Common Core State Standards for English Language arts & literacy in history/social studies, science, and technical subjects. National Governors Association Center for Best Practices & Council of Chief State School Officers. Retrieved from http://www.corestandards.org/wp-content/uploads/ELA_Standards.pdf

Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing. Retrieved from www.coe.int/lang-cefr

Dilekçi, A. (2022). Evaluation of Turkey’s PISA reading literacy scores. International Journal of Education and Literacy Studies, 10(1), 138–146. https://doi.org/10.7575/aiac.ijels.v.10n.1p.138

Dohn, N. B. (2007). Knowledge and skills for PISA? Assessing the assessment. Journal of Philosophy of Education, 4(1), 1–16. https://doi.org/10.1111/j.1467-9752.2007.00542.x

Eivers, E. (2010). PISA: Issues in implementation and interpretation. The Irish Journal of Education, 38(2010), 94–118. https://www.jstor.org/stable/20789130

Ertem, H. Y. (2021). Examination of Turkey’s PISA 2018 reading literacy scores within student-level and school-level variables. Participatory Educational Research, 8(1), 248–264. https://doi.org/10.17275/per.21.14.8.1

Figazzolo, L. (2009). PISA: Is testing dangerous? Retrieved from https://www.ei-ie.org/en/item/20648:pisa-is-testing-dangerous

Froese-Germain, B. (2010). The OECD, PISA and the impacts on educational policy. Canadian Teachers Foundation. Retrieved from http://files.eric.ed.gov/fulltext/ED532562.pdf

Gorur, R. (2016). Seeing like PISA: A cautionary tale about the performativity of international assessments. European Educational Research Journal, 15(5), 598–616. https://doi.org/10.1177/1474904116658299

Gorur, R., & Wu, M. (2015). Leaning too far? PISA, policy and Australia’s ‘top five’ ambitions. Discourse: Studies in the Cultural Politics of Education, 36(5), 647–664. https://doi.org/10.1080/01596306.2014.930020

Grek, S. (2009). Governing by numbers: the PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669

Grisay, A., de Jong, J. H. A. L., Gebhardt, E., Berezner, A., & Halleux-Monseur, B. (2007). Translation equivalence across PISA countries. Journal of Applied Measurement, 8(3), 249–266. http://www.ncbi.nlm.nih.gov/pubmed/17804893

Grisay, A., & Monseur, C. (2007). Measuring the equivalence of item difficulty in the various versions of an international test. Studies in Educational Evaluation, 33(1), 69–86. https://doi.org/10.1016/j.stueduc.2007.01.006

Gür, B. S., Çelik, Z., & Özoğlu, M. (2012). Policy options for Turkey: a critique of the interpretation and utilization of PISA results in Turkey. Journal of Education Policy, 27(1), 1–21. https://doi.org/10.1080/02680939.2011.595509

Jakupec, V., & Meier, B. (2015). PISA–Shocks, after shocks and misconceptions. Leibniz Online, 17, 1–11. https://hdl.handle.net/10536/DRO/DU:30070169

Karakoç Alatlı, B., & Çokluk Bökeoğlu, Ö. (2018). Uluslararası öğrenci değerlendirme programı (PISA-2012) okuryazarlık testlerinin ölçme değişmezliğinin incelenmesi [Investigation of measurement invariance of literacy tests in the programme for international student assessment (PISA-2012)]. Elementary Education Online, 17(2), 1096–1115. https://doi.org/10.17051/ilkonline.2018.419357

Kilpin, K. (2020, February 20). PISA’s warning: Teachers need to teach information literacy explicitly if we are to reverse the decline. Retrieved from https://informationliteracyspaces.wordpress.com/2020/02/24/pisas-warning-teachers-need-to-teach-information-literacy-explicitly-if-we-are-to-reverse-the-decline

Kılıç Depren, S., & Depren, Ö. (2022). Cross-cultural comparisons of the factors influencing the high reading achievement in Turkey and China: Evidence from PISA 2018. The Asia-Pacific Education Researcher, 31(4), 427–437. https://doi.org/10.1007/s40299-021-00584-8

Koç, E. S. (2021). Türkçe dersi öğretim programları ile okuma becerileri öğretim programının uluslararası okuma yeterlikleri bağlamında incelenmesi [An analysis of the Turkish curriculum and the reading skills curriculum in context of international reading competencies]. Millî Eğitim, 230, 169–193. https://doi.org/10.37669/milliegitim.702775

Matsushita, K. (2014). The taming of PISA literacy: global functional literacy and national educational content. Kyoikugaku Kenkyu, 81(2), 150–163. https://doi.org/10.11555/kyoiku.81.2_150

MEB. (2019). Türkçe dersi öğretim programı (İlkokul ve ortaokul 1, 2, 3, 4, 5, 6, 7 ve 8. sınıflar) [Turkish Language Lesson Curriculum (Primary and Secondary School Grades 1, 2, 3, 4, 5, 6, 7 and 8]. Millî Eğitim Bakanlığı.

Meyer, H. D., & Zahedi, K. (2014). Open letter to Andreas Schleicher, OECD, Paris. Policy Futures in Education, 12(7), 872–877. https://doi.org/10.2304/pfie.2014.12.7.872

Mortimore, P. (2009). Alternative models for analysing and representing countries’ performance in PISA. Education International Research Institute.

Niemann, D., Martens, K., & Teltemann, J. (2017). PISA and its consequences: Shaping education policies through international comparisons. European Journal of Education, 52(2), 175–183. https://doi.org/10.1111/ejed.12220

OECD. (2000). Measuring student knowledge and skills: The PISA 2000 assessment of reading, mathematical and scientific literacy. OECD. https://doi.org/10.1787/9789264181564-en

OECD. (2004). Learning for tomorrow’s world: First results from PISA 2003. OECD Publishing. https://doi.org/10.1787/9789264006416-en

OECD. (2016). PISA 2018 translation and adaptation guidelines. OECD Publishing. https://www.oecd.org/pisa/pisaproducts/PISA-2018-TRANSLATION-AND-ADAPTATION-GUIDELINES.pdf

OECD. (2019). PISA 2018 Assessment and Analytical Framework. OECD Publishing. https://doi.org/10.1787/b25efab8-en

OECD. (2021). 21st-century readers: Developing literacy skills in a digital world. OECD Publishing. https://doi.org/10.1787/a83d84cb-en

OECD. (2023). Frequently asked questions. Retrieved from https://www.oecd.org/pisa/pisafaq

Özmen, D. T. (2014). PISA 2009 okuma testi maddelerinin yanlılığı üzerine bir çalışma [A study on PISA 2009 reading test items in terms of bias]. Educational Sciences and Practice, 13(26), 147–165.

Rindermann, H. (2007). The g-factor of international cognitive ability comparisons: The homogeneity of results in PISA, TIMSS, PIRLS and IQ-tests across nations. European Journal of Personality, 21(5), 667–706. https://doi.org/10.1002/per.634

Sahlberg, P. (2018). FinnishED leadership: Four big, inexpensive ideas to transform education. Corwin.

Schleicher, A. (2007). Can competencies assessed by PISA be considered the fundamental school knowledge 15-year-olds should possess? Journal of Educational Change, 8(4), 349–357. https://doi.org/10.1007/s10833-007-9042-x

Schleicher, A. (2013). Attacks on PISA are entirely unjustified. Retrieved from Tes Magazine. https://www.tes.com/magazine/archive/attacks-pisa-are-entirely-unjustified-0

Schleicher, A. (2017). Seeing education through the prism of PISA. European Journal of Education, 52(2), 124–130. https://doi.org/10.1111/ejed.12209

Seifert, S. (2021). Is reading comprehension taken for granted? An analysis of Austrian textbooks in fourth and sixth grade. Technology, Knowledge and Learning, 26(2), 383–405. https://doi.org/10.1007/s10758-021-09490-w

Shanahan, T., & Shanahan, C. (2008). Teaching disciplinary literacy to adolescents: Rethinking content-area literacy. Harvard Educational Review, 78(1), 40–59. https://doi.org/10.17763/haer.78.1.v62444321p602101

Sjøberg, S. (2015). PISA and global educational governance – A critique of the project, its uses and implications. EURASIA Journal of Mathematics, Science and Technology Education, 11(1), 111–127. https://doi.org/10.12973/eurasia.2015.1310a

Sjøberg, S., & Jenkins, E. (2022). PISA: a political project and a research agenda. Studies in Science Education, 58(1), 1–14. https://doi.org/10.1080/03057267.2020.1824473

Snow, C. (2002). Reading for understanding: Toward a research and development program in reading comprehension. RAND Education.

Takayama, K. (2018). How to mess with PISA: Learning from Japanese kokugo curriculum experts. Curriculum Inquiry, 48(2), 220–237. https://doi.org/10.1080/03626784.2018.1435975

UNESCO. (2005). EFA global monitoring report 2006: Literacy for life. United Nations Educational, Scientific and Cultural Organization.

Wuttke, J. (2007). Uncertainty and bias in PISA. In S. T. Hopmann, G. Brinek, & M. Retzl (Eds.), PISA according to PISA: Does PISA keep what it promises (pp. 241–263). LIT VERLAG.

Yelken, E. (2016). Eğitimi uluslararası sınavlarla yeniden düzenlemek: Bağlamı, kapsamı ve dönüştürücü gücüyle PISA [Reshaping education by international assessments: The context, scope and transformative impact of PISA]. (Doctoral dissertation). Retrieved from Turkey National Thesis Center. (Thesis No. 431270)

Yıldız, D. (2021). Turkish and Turkish language and literature teachers’ views’ on the reading skills and Turkey’s performance in PISA: A focus group interview. Journal of Qualitative Research in Education, 27, 208–231. https://doi.org/10.14689/enad.27.10

Zhao, Y. (2020). Two decades of havoc: A synthesis of criticism against PISA. Journal of Educational Change, 21(2), 245–266. https://doi.org/10.1007/s10833-019-09367-x




DOI: https://doi.org/10.7575/aiac.ijels.v.11n.4p.57

Refbacks

  • There are currently no refbacks.


Bookmark and Share


License URL: https://creativecommons.org/licenses/by/4.0/

2013-2023 (CC-BY) Australian International Academic Centre PTY.LTD.

International Journal of Education and Literacy Studies  

You may require to add the 'aiac.org.au' domain to your e-mail 'safe list’ If you do not receive e-mail in your 'inbox'. Otherwise, you may check your 'Spam mail' or 'junk mail' folders.