What does PISA Assess in Reading Literacy? Misconceptions and Misuses

In the 21 st century, international interaction in social, economic, cultural, and educational fields has increased. Consequently, international standards have become essential in national education policies, reforms, and practices. As an international assessment, PISA has started to function as a prominent tool in this regard. However, the impact of PISA differs across the participating countries, depending on how the concept, methodology, and practices are handled. One of the domains where this difference is seen is reading literacy. Although this domain expresses a broader and richer phenomenon, the content and scope of the concept are not accurately understood, and its influences vary across the participating countries. In Türkiye, reading literacy is mainly considered and discussed in the scope of Turkish language lessons. This perspective, which focuses on the Turkish language lessons, leads to misunderstanding of the issue and the inaccurate change and transformation of the curriculum and content. Attempting to succeed in this domain through test language lessons deepens the problems instead of solving them. In this article, misconceptions and misuses about reading literacy are explained based on a literature review, and it is pointed out that reading literacy should be addressed in a broader context, including the curricula of other subjects, rather than test language lessons.


INTRODUCTION
Programme for International Student Assessment has left behind two decades as an international study assessing the extent to which 15-year-old students have the knowledge and skills they need in real life.The assessment, which was first held in 2000 and repeated every three years, mainly focuses on students' reading, math, and science skills.In addition, from time to time, innovative domains such as creative problem solving, collaborative problem solving, global competence, creative thinking, and financial literacy are evaluated in certain participating countries.It also collects information about students' home backgrounds, learning approaches, and environments by applying questionnaires to reveal the reasons for success or failure in measured cognitive skills (OECD, 2019).
According to the explanations regarding the assessment, PISA does not aim to create any curriculum or to meet on common ground (OECD, 2023).Instead, it aims to provide countries with the opportunity to compare their education systems with those of other countries.Because in today's world, international standards have become important, as well as national standards, in the evaluation of success in education.At this point, PISA, which stands out as a powerful tool for international comparison, has accelerated and facilitated developments in education by providing a different perspective on the opportunities and problems of education (Schleicher, 2017).In this respect, the OECD has gradually increased its influence and become an expert organization in education (Niemann et al., 2017).On the other hand, by becoming a global yardstick (Breakspear, 2014) in education, it has started to shape the content of curricula in some countries, contrary to its purpose.
Although it is controversial, PISA research results have been taken into consideration by all stakeholders in education and have sometimes played a guiding and sometimes a determining role in the educational policies of countries (Breakspear, 2012;Froese-Germain, 2010;Grek, 2009;Gür et al., 2012;Sjøberg & Jenkins, 2022;Yelken, 2016).When the results are announced, the success or failure of countries through the rankings on league tables are evaluated and discussed on various media platforms, reports are written about the results, and they are the subject of research in academic circles.In the related studies, conflicting views are put forward on PISA's purposes, methods, and results.While some see PISA as an essential tool for countries to compare their education systems with those of other countries and to direct their educational policies and reforms accordingly, some think negatively about the purposes, methods, results,

ABSTRACT
In the 21 st century, international interaction in social, economic, cultural, and educational fields has increased.Consequently, international standards have become essential in national education policies, reforms, and practices.As an international assessment, PISA has started to function as a prominent tool in this regard.However, the impact of PISA differs across the participating countries, depending on how the concept, methodology, and practices are handled.One of the domains where this difference is seen is reading literacy.Although this domain expresses a broader and richer phenomenon, the content and scope of the concept are not accurately understood, and its influences vary across the participating countries.In Türkiye, reading literacy is mainly considered and discussed in the scope of Turkish language lessons.This perspective, which focuses on the Turkish language lessons, leads to misunderstanding of the issue and the inaccurate change and transformation of the curriculum and content.Attempting to succeed in this domain through test language lessons deepens the problems instead of solving them.In this article, misconceptions and misuses about reading literacy are explained based on a literature review, and it is pointed out that reading literacy should be addressed in a broader context, including the curricula of other subjects, rather than test language lessons.and effects of the assessment (Sahlberg, 2018).Of course, PISA makes contributions that cannot be ignored regarding comparing education systems, identifying effective educational practices, and providing a data-based perspective in updating education systems with an innovative approach.However, in parallel with its increasing influence, a severe criticism literature has also emerged regarding PISA's concept, methodology, and implementation.These criticisms are expressed under the headings of cultural differences, translations, sampling, disregard national curricula, distortion of educational policies, lack of sufficient involvement of teachers, use of a cross-sectional design, modeling of data, effects of league tables on national school systems, dominance and secrecy (Mortimore, 2009).Led by Meyer and Zahedi (2014), academics and teachers worldwide wrote an open letter to Andreas Schleicher, the education director of the OECD and PISA, outlining their concerns and criticisms on these issues and offering solutions.
One of the prominent criticisms of PISA is that the primary motivation and emphasis of the assessment is political/economic concerns (Figazzolo, 2009;Sjøberg & Jenkins, 2022).PISA mainly measures skills related to the labor market, and its results do not provide any data on students' social, cultural, and personal development.Jakupec and Meier (2015) summarize this situation as a transition from general educational values to the acquisition of basic skills, while Gorur (2016) argues that this approach, which he describes as utilitarian simplification, may not sufficiently bring about the hoped-for changes in education, even if it provides short-term benefits; on the contrary, the shift in emphasis from educational value to practical skills leads to narrowing of the curriculum.
Another major criticism of the PISA is that the test results are inconvenient for international comparison.Opinions about this are primarily justified by translation and socio-cultural differences.PISA uses parallel forms translated and adapted to the test language for the comparability of the results.For this purpose, a strict translation procedure is applied.Tests and questionnaires prepared in English and French are translated into the test language by two translators and merged by a third person.In the translation, it is expected to avoid biases that would harm international comparability.For this, it is requested to avoid making the text, graphics, and tables easier and more difficult to understand, changing the difficulty level of the questions, ambiguities that may distort the data collected in the questionnaires, and avoiding national contextualization that would substantially change the data collected (OECD, 2016).
However, the problems arising from translation have not able to be eliminated entirely.Therefore, some research reveals that the tests are problematic in terms of translation equivalence (Arffman, 2010;Ceyhan, 2019;Eivers, 2010;Grisay & Monseur, 2007;Karakoç Alatlı & Çokluk Bökeoğlu, 2018;Özmen, 2014;Wuttke, 2007;Zhao, 2020).Even a simple comparison of parallel forms makes this clear.For example, according to Eivers (2010), the German version of the test is 15% longer than the original English version of the math items.In addition, words with a lower frequency were used in the German translation than in English.4 science units are 11% longer in Spanish, 11% of entire booklets are longer in Irish 2006, and 17% longer in German than in English.The data on the volume of the texts and the frequency of the preferred words indicate that translation-based criticism is too significant to be neglected.
Ignoring cultural differences between participating countries is also an essential subject of criticism.PISA aims to measure how prepared 15-year-olds are for future challenges and participation in society (OECD, 2000(OECD, , 2019)).This aim is based on the assumption that future challenges are already known and that young people from different countries and cultures join a society with similar or the same characteristics.However, while challenges and society in many countries are similar in some respects, this assumption does not reflect reality (Sjøberg, 2015).The cluster analysis on item difficulty indices shows that the patterns vary according to geographic, cultural, socio-economic, and linguistic characteristics (Grisay et al., 2007).Eivers (2010) states that there are cultural differences in the quality and equivalence of translations, the Anglophone bias that forms the basis of PISA, how students respond to different types of questions, and the importance students attach to assessment.It also points to the difficulty of creating neutral items, as the reallife context differs across the participating countries.Dohn (2007) argues that PISA does not measure students' knowledge and skills for life but rather knowledge and skills in assessment situations.
Undoubtedly, PISA has been dynamic since the day it was introduced.The concept, methodology, and implementation of the test have been revised occasionally to eliminate the problems criticized or updated according to social, cultural, economic, and technological changes.However, Schleicher (2013) argues against the ongoing criticism, stating that it is inevitable that any assessment of human skills will involve several uncertainties.He also recognizes that, due to methodological limitations, PISA does not cover all the competencies that young people should have (Schleicher, 2007).In this respect, the results should be interpreted with a more modest, careful, and broad approach, taking into account some of the limitations of the PISA assessment.Also, the weaknesses of the test should not be ignored in the changes to be made in the education system based on this assessment.However, since the PISA test method and content are seen as standard, the participating countries tend to use the data uncritically.This lead to a biased or inaccurate change in countries' educational policies and practices.
Depending on their social, cultural, and economic characteristics, the education systems and programs of the participating countries naturally differ.Recently, these differences have been ignored in education policies and reforms, and with the effect of PISA, the curricula have begun to resemble each other in the participating countries (Zhao, 2020).One of the domains where this situation is seen is reading literacy.Although the way it is handled and its practices differ across the participating countries, there is a tendency in some countries to associate reading literacy with test language lessons.Due to this misassociation, test language lessons are seen through the prism of PISA, and accordingly, changes are made in the curriculum, content, and practices of test language lessons based on PISA.This article includes a critique of this misconception and misuse caused by seeing like PISA (Gorur, 2016) rather than a critique of PISA.For this, firstly, how PISA defines reading literacy, how it is measured, and what it measures will be described based on the literature review.Then, how reading literacy is handled in Türkiye will be described, and the criticisms related to this will be explained.

HOW DOES PISA DEFINE READING LITERACY?
Reading, which basically means looking at the letters and signs that make up a text and decoding and understanding or vocalizing them, has been one of the most important means of obtaining information since the invention of writing.Parallel to the spread of written culture, the meaning attributed to reading and the functions expected from reading has also diversified.In this respect, PISA envisions reading as the basis of full participation in contemporary society's economic, political, social, and cultural life.It focuses "on reading to learn rather than learning to read" (OECD, 2004, p. 25).Therefore, reading in PISA is considered a broader and richer phenomenon than the traditional definition.Due to this reading structure, reading literacy has been preferred instead of reading and reading skills.It was stated that this preference was to convey to the non-expert audience what the PISA exams measure more accurately (OECD, 2019).
The concept of literacy, as used in PISA (OECD, 2019) "refers to students' capacity to apply knowledge and skills, and to analyze, reason and communicate effectively as they identify, interpret and solve problems in a variety of situations" (p.13).It also emphasizes "the functional knowledge and skills that allow one to participate fully in society" (p.14).In this respect, PISA uses the term reading literacy to refer to "the active, purposeful and functional application of reading in a range of situations and for various purposes" (p.28).Reading literacy includes "a wide range of cognitive and linguistic competencies, from basic decoding to knowledge of words, grammar and the larger linguistic and textual structures needed for comprehension… integration of meaning with one's knowledge about the world and metacognitive competencies" (p.28).
In PISA 2000, reading literacy was first defined as "understanding, using, and reflecting on written texts, in order to achieve one's goals, to develop one's knowledge and potential, and to participate in society".However, this definition has changed gradually due to the changing nature of reading with the developments in society, economy, culture, and technology.In PISA 2009, engagement was added to this definition, and in PISA 2018, evaluating was added, and the term written was removed from this definition.Evaluating and engagement were added to the definition as they were necessary for complete reading literacy.The removal of the term written from the definition is based on technological developments in the last two decades and the new reading tools and forms that have emerged.Reading has significantly shifted from printed to digital texts in the last two decades.Today, books, magazines, newspapers, etc., are read in printed versions and digital environments through computers or phones.Thus, digital literacy skills have become a requirement for active participation in society.In this respect, printed, handwritten and screen-based formats are also accepted as texts in PISA.With this understanding, any material containing graphics, except for aural language artefacts, is considered text (OECD, 2019).

HOW DOES PISA MEASURE READING LITERACY?
Reading literacy is one of the essential areas in the PISA assessment, along with mathematics and science.Reading literacy is recognized as an essential requirement for success in other subject areas and participation in adult life (OECD, 2019).The focus domain of the first PISA cycle in 2000 was reading, and it was addressed as a focus domain twice in the following exam cycles (2009 and 2018).
The purpose and tools of measurement are shaped according to the definition and details of the phenomenon that is the subject of measurement.In this respect, reading literacy in PISA is measured in a structure created according to the definition and details that have been revised over time.
According to these, reading literacy in PISA is measured based on a different content and practice than conventional reading exams depending on the broad and rich content of the concept.PISA mainly assesses students' ability to "extrapolate inferences from what they have learned and apply their knowledge in new situations" (OECD, 2019, p. 12).
The PISA reading literacy assessment is based on the RAND Group's (Snow, 2002) classification, considering three main factors: reader, text, and task.In the reading process, reader characteristics such as motivation, prior knowledge, and cognitive abilities; text characteristics such as form, genre, and language; and task characteristics such as time, practical constraints, purpose, and complexity are effective.Within the framework of these factors, readers resort to a series of literacy processes to find and extract information and create meaning in a text.Cognitive assessment is measured through various texts and tasks, and the reader factors such as motivation are evaluated through questionnaires (OECD, 2019).
Participation in society, which is PISA's primary purpose and emphasis, is a determining factor in the questions.As a natural consequence, reading is measured with tasks that include real-life contexts.For this purpose, four scenarios are used in PISA based on the classification of the Common European Framework for Languages (Council of Europe, 2020) to illustrate real-life situations: personal, public, educational, and professional.The texts for these scenarios are taken mainly from real and everyday sources such as newspapers, magazines, websites, and manuals.
Texts and their classification have changed in PISA cycles.In the 2018 framework, texts are addressed in four dimensions.They are characterized as single and multiple in source and static and dynamic in structure.Text formats are classified as continuous texts (newspaper reports, articles, novels, short stories, reviews, letters), non-continuous texts (lists, tables, graphs, diagrams, advertisements, catalogs, indexes, and forms), and mixed texts (including continuous and non-continuous texts).Text types are classified as description (travel diary, operations in the technical guide), narration (novel, story, newspaper report), exposition (essay, encyclopedia article), argumentation (letter to the editor, movie-book review), instruction (recipes, user manuals) and transaction (e-mail and SMS) text (OECD, 2019).
Response formats are diversified, considering that students in participating countries are unfamiliar with a particular response form.Multiple-choice and short-answer item response formats are used to measure all cognitive abilities and knowledge defined in the PISA framework.In addition, with the transition to computer-based testing, highlighting and drag-and-drop response formats have also been introduced.The test duration is 2 hours, and 30 minutes is allotted for each domain.Since one of the focus domains is tested in detail in each cycle of PISA, approximately half of the total test time is allocated to the focus domain (OECD, 2019).
The paper-based assessment was used in the first two cycles.Optional computer-based assessments were offered in science in 2006, reading in 2009, and mathematics in 2012.Since 2015, computer-based assessment has been used in all domains in most of the participating countries.With this exam format, reading fluency began to be measured through simple tasks.In addition, PISA has started using adaptive testing methods.In this method, the test progresses with questions of lower or higher difficulty according to the student's answers to previous questions.In this way, it is aimed to assess the skill levels of especially low-performing students more stably and sensitively (OECD, 2019).
PISA evaluates reading literacy results regarding total scores and reading proficiency levels.In PISA 2000, reading proficiency was initially composed of 6 levels (below level 1, 1, 2, 3, 4, and 5).However, the sixth level was added in the 2009 cycle to characterize student performance better.This level identifies high-performing students and has no upper limit.The lowest performance also was renamed 1a and 1b in the 2009 cycle, and 1c was added in the 2018 cycle to indicate the lowest performance.This level also has no lower limit (OECD, 2019).

WHAT LITERACY DOES PISA ASSESS, REALLY?
Literacy, which PISA consciously prefers, is defined basically as the ability to read and write in a language.In the general sense that this definition implies, reading is a concept everyone understands.However, literacy has a complex and dynamic structure and is interpreted and defined differently.The content of these interpretations and definitions is influenced by factors such as academic research, and national context (UNESCO, 2005).On the other hand, this concept has expanded its meaning depending on the changes in society, education, the economy, and technology.As the concept scope expanded, literacy could not express all the facts independently, and literacy domains began to be mentioned.The domains of literacy are diversifying day by day.For example, many types of literacy, such as information, digital, visual, financial, media, ITC, and adult literacy, have emerged to express the knowledge and skills of individuals in a particular domain.
So, which of all literacy does PISA measure?Of course, the answer could be that PISA basically measures literacy in reading, science, and mathematics.However, the question or problem is not so clear and one-dimensional.The complex and ambiguous nature of the purpose and content of the assessment leads to different interpretations.For example, Matsushita (2014) compares functional, cultural, and critical literacies and sees PISA literacy as functional literacy, which is assumed to be globally applicable.Eivers (2010) claims that PISA measures general literacy using three different content areas.Similarly, since each domain measures nested content, there are also comments that it measures a single general ability by referring to the g factor (Bodin, 2007;Rindermann, 2007;Wuttke, 2007).
According to the context of the test itself, PISA measures three different literacies: reading, mathematics, and science.Considering the explanation, "The triennial assessment focuses on the core school subjects of reading, mathematics and science" (OECD, 2019, p. 11), it can be considered that it measures school curricula.However, the assessment of cross-curricular competencies is one of the key features of PISA.Therefore, each domain is structured according to adult life rather than school curriculum (OECD, 2000).Nevertheless, PISA's (OECD, 2019) explanations of mathematical and scientific literacy show that these domains refer to a narrower and more specific domain than reading literacy: National mathematics curricula are typically designed to equip students with knowledge and skills that address these same underlying mathematical phenomena, the outcome is that the range of content arising from organising content this way is closely aligned with that typically found in national mathematics curricula.(p.83) Assesses scientific knowledge using contexts that raised pertinent issues that were often relevant to the science education curricula of participating countries.However, assessment items are not limited to school science contexts.(p.103) These statements make it clear that mathematical and scientific literacy content is based on -but not limited to -national curricula.
However, there is no direct reference to a specific curriculum in the relevant sources for reading literacy.Although it brings to mind test language lessons, there is uncertainty about reading literacy, which expresses more comprehensive content and scope.Because how reading is handled, and practices differ between participating countries.In fact, within the framework of PISA 2018, both these differences and different approaches to reading in the literature were mentioned (OECD, 2019).On the other hand, Eivers (2010) states that it is possible to say that PISA reflects the curriculum and worldview of some countries better than others.This is particularly evident in the reading literacy domain.Takayama (2018), referring to the Reading Expert Group, argues that the Western conception of literacy dominates in PISA.The author also contends that PISA de-territorializes literacy to enable international comparison and abstracts the literacy curriculum from its social, political, and cultural role within a nation-state.
Some uncertainties regarding reading literacy can also be found in PISA's documents.For example, although PISA states that it does not measure a specific curriculum, from the first cycle onwards, many of the student questionnaires asked about the test language lesson to determine school climate.The content and scope of reading were not sufficiently clear from the first implementation.Although the frameworks contain detailed descriptions of what reading skills are, it is not clearly stated what they are not.The rationale for choosing the term reading literacy -perhaps because it was seen to be a misconceptualization -was expressed in the 2009 framework instead of the first cycle.The problems encountered in implementing the questionnaires on reading literacy were only addressed in the 2018 framework.Explanations of these problems provide more enlightening information about the content and scope of reading literacy: When students are 15 years old, reading is no longer taught as a standalone subject in the same way that mathematics and science are.However, reading literacy is still improved by teaching practices, and reading strategies are taught or learned through not only language arts and literature courses in the test language, but also through foreign language courses and social and natural science courses, known in their entirety as "content literacy".While questions about teaching and learning mathematics and science can be, to a great extent, limited to solely mathematics and science lessons, there is clear evidence that rich and valuable information about reading (especially online reading) cannot be obtained solely from test language instruction lessons.Indeed, one of the most striking differences between countries in their reading curriculum is their emphasis on and time dedicated to content literacy, including the teaching reading in other subjects.Consequently, any teacher questionnaire implemented in PISA 2018 investigating the teaching of reading literacy should be administered to a sample of teachers across domains, rather than only to test language teachers.(OECD, 2019, p. 224) Due to this dispersed nature of reading instruction provided directly or indirectly in test language lessons and other disciplines, it is impossible to associate the PISA reading literacy with a specific curriculum, unlike mathematical and scientific literacy.Indeed, PISA considers a range of cross-domain knowledge and skills, including knowledge from fields such as science and mathematics and the effective use of technology, to be necessary for success in reading in today's world (OECD, 2019).Considering PISA's approach to reading and the test content, it can be said that it does not measure a specific curriculum applied in schools but students' general knowledge and skills.In this sense, reading literacy overlaps with PISA's aim of assessing cross-curricular competencies, more than mathematics and science.These competencies are developed in school through test language lessons and other lessons, and out of school through social, cultural and economic opportunities.
A study conducted in the UK (Carroll & Benton, 2018) provides a data-based perspective on the nature of PISA reading.This study examined PISA and General Certificate of Secondary Education (GCSE) results regarding subject-area relationships.Interestingly, a relatively weaker correlation was found between the subjects expected to correlate with PISA reading scores than with subject area subjects: English (r = 0.680) and English Literature (r = 0.637) showed weaker correlations than History (r = 0.696), Highest Science (r = 0.708), Core Science (r = 0.692) and Geography (r = 0.687).This suggests that PISA reading measures different skills than GCSE English assessment.Moreover, it also measures subject area reading.
Because reading is a foundation in all subjects, this correlation between subject area lessons is not surprising.For this reason, in some countries, reading is considered an activity area not only for test language lessons but also for all subjects, and the approach of "Every teacher is a reading teacher" is adopted.In the literature, the terms content area reading, or content literacy refers to students' reading and writing skills for each subject.In the context of this concept, direct and indirect teaching practices are organized in social studies, history, science, mathematics, etc., to develop students' reading and writing skills related to the subject.
For example, in the CCSS, content area reading and writing skills are considered separate activity areas.In addition to English Language Arts, the program includes "Literacy in History/Social Studies, Science, and Technical Subjects" standards (Common Core State Standards Initiative, 2010).In Austria, in addition to the national curriculum for German language classes, the tasks of reading education are also defined in relation to all other subjects.The document states that reading instruction should be cross-curricular and related to all subjects taught at school, not just German language lessons, and that all teachers are responsible for developing reading literacy (Seifert, 2021).
Since PISA aims to measure real-life skills, all kinds of texts that can be encountered in real life are used.These texts are taken from books, magazines, and newspapers.It cannot be said that the texts in these sources represent an isolated domain.Although the texts on economy, culture, art, science, science, and health in a daily newspaper are read by everyone, their comprehension depends on knowledge and experience in these domains.In other words, texts that can be encountered in daily life also demand content area reading skills.When the released questions of PISA are examined, it can be understood that even just the text titles "2000-Flu (ACOL Voluntary Immunization Program), Police (Scientific Police Weapons) New Rules (Technology Creates The Need For New Rules) 2006-Lake Chad, Labour, 2009-Tall Buildings, Democracy in Athens, Metrotransit 2012-Balloon, 2018-Cow's Milk" evoke content area reading.These texts, ranging from law to architecture, history to philosophy, and biology, cannot be read efficiently without basic knowledge about each domain.Moreover, this basic knowledge cannot fit within the limits of a test language lesson.
As a comprehensive concept, reading literacy involves many higher-level skills beyond comprehending the lexical meaning of a text.To reach the proficiency defined in PISA, individuals need to be able to manage a series of related processes skillfully.At this point, the reader is expected to read a text fluently, locate information (accessing and retrieving information within a text and search and select relevant text.),comprehend (represent literal meaning and integrate and generate inferences), evaluate and reflect (assessing quality and credibility, reflecting on content and form, detecting and handling conflict) (OECD, 2019, p. 33).All these reading processes are not independent of the individual's own background, socio-cultural characteristics, and the cultural context in which the text is created.This is because reading skills require the reader to "relate what they read to their own background experience and knowledge" (OECD, 2000, p. 11), "using previous knowledge and a range of text and situational cues that are often socially and culturally derived" (OECD, 2019, p. 27) or "apply their previous knowledge" (OECD, 2019, p. 29).
There are other factors in reading related to students' knowledge and background.Studies consider subject familiarity as a factor affecting reading skills (Schiefele, 2009;Shimoda, 1993).As a reflection of this, PISA also considers engagement with a variety of reading as one of the essential variables of reading skills (OECD, 2019).On the other hand, one of the factors determining reading is vocabulary.Although general vocabulary can be learned and taught through language classes, academic vocabulary can be better learned and taught in the related subject area (Beck et al., 2013).There is also a correlation between ICT literacy and reading performance in PISA.However, where the new skills and processes related to digital reading are taught varies from country to country, and there is uncertainty in this regard (OECD, 2019(OECD, , 2021)).Shanahan and Shanahan (2008) state that reading is generally seen as a set of basic skills that can be widely adapted and applied to all types of texts and reading situations.For this reason, it is common to think that providing general literacy skills in the lessons is sufficient.However, the authors say that reading history, literature, philosophy, and mathematics texts do not require the same processes and skills.Each text demands different knowledge and skills from its readers due to its content and genre.Many literacy skills and texts become highly specialized and require relatively unique actions as student progress through the grades.Content area literacy skills support basic and intermediate literacy but are insufficient.Therefore, there is a need for reading and writing instruction to become increasingly disciplined, in other words, disciplinary literacy.On the other hand, Kilpin (2020) argues that the reading literacy envisaged by PISA should be considered information literacy by considering the cognitive processes.He explains the rationale for this view as follows: These features do carry forward elements of that traditional view of comprehension, but it is the information skills -particularly about locating, handling and evaluating information that makes this PISA report important reading.These are the kete of processes and strategies that support senior secondary and tertiary students to find, organise, process, refine and communicate information, at levels of critical understanding, across a variety of predominantly disciplinary information texts.
In other words, the OECD's concept of comprehension should be more accurately titled Information Literacy Skills (ILS), as they constitute a dynamic cycle of critically challenging, receptive and productive, text-focused work, and are the skills students need to transfer flexibly across their content learning and into their nonschool life and adulthood.(para.4) He states that all teachers should share the responsibility for information literacy in the sense that "Every teacher is a teacher of reading." and that it is an issue that goes beyond test language lessons.Considering the multifaceted and complex nature of reading and the different approaches and practices in the literature, it is clear that reading literacy should be seen as more than an activity area of test language lessons.Reading literacy should be addressed in a broader framework without ignoring the responsibilities and functions of test language lessons.Based on the conceptualizations and discussions in the literature, reading literacy should be addressed and discussed in a different context, such as subject area reading, content literacy, disciplinary reading, and information literacy.

HOW IS READING LITERACY CONSIDERED IN TÜRKIYE?
In the social sciences, concepts, and therefore the phenomena represented by concepts, are often not neutral, unlike in the natural sciences.They are often based on social and cultural background.For this reason, the social and cultural context of the concepts should not be ignored when making comparisons about the subjects related to social sciences.For a healthy evaluation, the meanings attributed to the concepts should be understood and interpreted in the same way by everyone.Although the concept of reading expresses a common phenomenon in its basic form, the meaning attributed to reading and the practices of reading may vary from one society to another.
PISA reading literacy's broad and ambiguous nature leads to different conceptualizations in participating countries.In some countries, reading literacy is sometimes discussed and debated in the context of test language lessons (Kilpin, 2020;Takayama, 2018).Even in PISA's own survey implementation, test language lessons were wrongly emphasized.This is probably why it is emphasized in the PISA 2018 framework that "any teacher questionnaire… investigating the teaching of reading literacy should be administered to a sample of teachers across domains, rather than only to test language teachers" (OECD, 2019, p. 224).
An example of misconceptions and misuses can be seen in Japan.In the 2003 PISA cycle, the decline in reading literacy made improving the domain a top priority in the country's educational reform.A series of studies and practices were quickly implemented to increase the ranking in reading literacy.In the process, PISA reading literacy has significantly influenced the Japanese kokugo curriculum, although it is emphasized that it is quite different in nature from the Japanese concept of dokukai.Textbooks began to include non-continuous texts and emphasized more functional aspects of reading.Many kokugo curriculum experts and educators began to express concerns about the changes in the national literacy curriculum based on PISA (Takayama, 2018).
A conceptualization and practices similar to those in Japan are also experienced in Türkiye.Türkiye, which did not participate in the first cycle in 2000 because it was implementing the TIMMS-R and PIRLS projects, participated in the PISA tests for the first time in 2003.In the cycles in which it participated, it failed to show the desired performance in reading literacy as in other domains.It remained below the OECD country averages in all cycles.This failure created a kind of "PISA shock" effect in Türkiye and formed the justification for many changes in education over time (Gür et al., 2012).
The relatively lower performance in reading literacy than in science and mathematics literacy has made reading skills a higher priority in educational reforms.The low performance in reading literacy has primarily been addressed in relation to Turkish language lessons.This association mainly stems from the uncertainties about reading that PISA aims to measure.At the same time, the Turkish translation of reading literacy also leads to this association.The concept of reading literacy is preferred in PISA instead of reading and reading skills because it expresses a different phenomenon.A different conceptualization approach is also found in other languages.In the literature, conceptualizations such as PISA-gata dokukai (PISA-type reading and comprehension) (Takayama, 2018) can also be encountered.In Türkiye, the term "reading competence" was first used for this concept in the 2003 report, but "reading skills" were preferred in subsequent reports.Although the conceptualizations of "reading literacy" and "reading skills literacy" are occasionally seen in Turkish literature, they have not gained widespread use because they contain a cacophony.These reasons have led to the perception in Türkiye that PISA tests measure reading as a language skill.As a result of this perception, almost all of the studies and practices related to reading literacy focus on the role of Turkish language lessons.Moreover, the Turkish language lessons are seen as almost solely responsible for PISA reading success/failure.This perspective, which puts the Turkish language lessons at the center, leads to an inaccurate understanding of the subject and an incorrect change and transformation of the course content.
As a result of this misconception, Turkish language curriculum (e.g., Batur & Ulutaş, 2013;Koç, 2021), Turkish textbook (e.g., Benzer, 2019;Bozkurt et al., 2015), LGS Turkish questions (e.g., Aktaş, 2022;Aşıcı et al., 2012), Turkish and literature teachers' opinions (e.g., Dilekçi, 2022;Yıldız, 2021) and the effect of various variables on reading literacy (e.g., Ertem, 2021;Kılıç Depren & Depren, 2022) are commonly examined in studies on PISA reading literacy.In these examinations, PISA content and practices are seen as a benchmark.Therefore, the content and practices of PISA reading literacy are never questioned; it is seen as an ideal example for language lessons, and the inadequacy of Turkish language lessons and practices is frequently emphasized.To overcome the failure in reading literacy, the opinions that Turkish curricula should be updated according to PISA reading competencies, the texts used in PISA reading should be increased in Turkish textbooks, and PISA-style questions should be made widespread in Turkish lessons come to the fore.
It is possible to say that these views have been met to a great extent.Turkish curricula have changed five times (2006,2015,2017,2018,2019) in the last two decades under the shadow of PISA.Especially due to the poor performance in 2015, the Turkish curriculum was updated successively in 2017, 2018, and 2019.PISA more or less influenced all these changes.This effect is especially evident in the references to non-continuous texts in the curriculum and textbooks.In 2017, 2018, and 2019 Turkish curricula (grades 1-8), the standard "interprets information presented in graphs, tables and charts/answers questions about the information" is included starting from the third grade.In the following times, graphs, tables, and charts for this standard started to be included in the textbooks.In addition to literary texts, texts that require reading in the subject area were also used.
Depending on updating the curriculum and textbooks, the content of the "Central Examination for Secondary Education Institutions that will take students with exams" underwent a radical change in 2019.Considering the sample questions published by the Ministry of National Education and the content of the test, it is seen that the Turkish questions are structured mainly similar to the PISA reading domain.
In the Turkish section of the tests, analytical reasoning and PISA-style questions are increasingly included, assuming they measure high-level reading skills.Especially in questions on non-continuous texts, content that measures subject area knowledge rather than language skills is included.In addition, reflections of a similar approach can be seen in the "local PISA" test, which was implemented in 2016 within the scope of the Monitoring and Evaluation of Academic Skills Project, and in the Turkish Language Examination in Four Basic Skills.It is common practice to update curricula, textbooks, and examinations according to current needs and possibilities.In this process, both national and international literature and research can be used.In this respect, it cannot be said that the changes in the Turkish curriculum in recent decades are completely unjustified.For, as in the past, there are still problems in Turkish language education in general and reading education in particular.However, trying to solve the chronic and national problems of Turkish language education by looking through the prism of PISA, an international test, leads to a distorted understanding, and Turkish language lesson is forced to fulfill a function that is not appropriate to its mission.
PISA reading literacy have not been discussed sufficiently in Türkiye conceptually until now.Despite all its limitations and flaws, excluding a few studies on measurement invariance, no critique of the concept, methodology, and practices of PISA reading literacy has been put forward.The context of PISA's content and practices is ignored and seen as an ideal example for language education.This perspective leads to ignoring many problems that lead to student failure and solutions to these problems.For example, Türkiye is not in a good position regarding not-reached items.In a table provided by Gorur and Wu (2015), Türkiye ranks 27 th among thirty-four countries in terms of not-reached item rates.The authors explain this mainly in terms of test motivation and point out that better achievement can be possible even by improving motivation instead of many reforms.On the other hand, the concepts and practices of content and disciplinary literacy, which can ensure success in reading, have not been sufficiently discussed in Turkey, and consensus and awareness of these concepts have not been developed.Due to the problems in this area are not defined and described, alternative solutions are not put on the agenda.
The primary purpose of Turkish language lessons in Türkiye, as in every country, is to develop students' listening, speaking, reading, and writing skills.In addition to this, it is aimed to ensure that they directly or indirectly reach language taste and consciousness, develop their worlds of emotion, thought, and imagination, attach importance to national, spiritual, moral, historical, cultural, and social values, strengthen their national feelings and thoughts, and recognize and adopt aesthetic and artistic values.For this, texts with literary and cultural values from widely accepted authors and works are mainly used in textbooks (MEB, 2019).On the contrary, PISA-style texts created by avoiding adapting them to the national context for methodological reasons.It can be observed that this difference is ignored in some changes.
The exam-oriented education system prioritizes the measurable over the valuable.Therefore, teachers and students spend more time on what is useful for the exam.Unfortunately, high-stakes tests negatively affect the Turkish language lesson's general and specific objectives and content.Recently, in Türkiye, in addition to the high-stakes tests, the impact of international tests has begun to be felt in the content and practices of courses implicitly.International exams put a lot of pressure on administrators and experts, even if they have little impact on students and teachers.This situation makes already existing problems in Turkish language lessons even more chronic.
Over the last two decades, all elements of Turkish language teaching, from concepts to curricula, from textbooks to exams, have undergone significant change.Although many factors are influential in these changes, it can easily be seen that the content and results of PISA have played an important role in shaping them.Time will tell whether these changes will provide the desired results and what their possible effects will be.However, the misconception in reading literacy domain undermines the accuracy of these changes.This is a situation that already raises concerns about their possible consequence.

CONCLUDING REMARKS
With the effect of globalization, international communication and interaction are increasing day by day.In this way, each country benefits from the experiences of other countries in social, cultural, and economic issues.International tests, which have become widespread in the last quarter century, have also increased the interaction between countries in education.PISA, which allows international comparisons, has led to many innovations and improvements in education.However, some shortcomings in the concept, methodology, and implementation have led to many negativities due to the lack of consideration of flaws and limitations, ignoring the context of the test itself, and the selective evaluation of the results.
One of these negativities is seen in the reading literacy domain.Considering the definition of reading literacy and the purpose of measurement, PISA accepts all kinds of texts that students may encounter in daily life and need to understand as the object of reading.This is because reading involves understanding and using all types of texts, mainly due to the recent changes in the perception of the text.Undoubtedly, it is not a reasonable approach to present such a wide range of texts only in test language classrooms.Furthermore, it is not the accurate approach to provide students with the knowledge and skills that will enable to understand these texts, only in test language classrooms.Understanding and discussing reading literacy only in the context of test language lessons and trying to succeed in this domain through test language lessons deepens the problems instead of solving them.
To eliminate the misconceptions and misuses, the concept of reading literacy should be included in each curriculum in line with its own objectives and content and should be defined in accordance with the structure of that curriculum.In this way, the danger of preparing course contents and conducting assessment and evaluation in a way that will prevent the Turkish language lesson from reaching its specific goals and objectives should be eliminated.Also, the cost of the ways that PISA prefers to purify assessment from cultural characteristics can be minimized in terms of the Turkish language lesson.This perspective will prevent the conflict between the goal of success in an important international test, which is attempted to be isolated of its cultural context, and the specific aims of the Turkish language lesson, which should have a very strong cultural aspect.In this way, further complication of the problems can be avoided, and an awareness of healthy solutions can be developed.Otherwise, misguided attempts such as changing the content of Turkish lessons for the sake of test success, organizing the structure of Turkish questions in national high-stakes tests according to the international tests in question will not only fail to achieve the expected success, but will also prevent the specific aims of the lesson from being achieved.Therefore, the expectations of the Turkish language lesson have to be in line with the content and limits of this course.
As a result, there is a need for a global conceptual and theoretical discussion of reading literacy, as well as deeper, comprehensive, and profound analyses/evaluations of reading success or failure nationally.It should not be ignored that "taming PISA" (Matsushita, 2014) is necessary not to damage the aims and practices of national language education.Considering the uncertainties in this domain, uncritical use of PISA results should be avoided (Carroll & Benton, 2018).Otherwise, we may be faced with the fact that "the narrow and reductionist practices of 'seeing like PISA' becoming a disaster over time" (Gorur, 2016, p. 612) in language education.In other words, "seeing like PISA" practices may result in a "Pyrrhic victory" regarding national language education policies and practices in the future.