Senior High School English Teachers’ Perceptions on a High-stakes Test (SBMPTN): A Washback Study

The aim of this research is to explore the washback effect of a high-stakes test on teachers’ attitude and teaching methods used. High-stakes tests refer to tests results of which impact the future of the test takers. In this paper, the high-stakes test is the Entrance Exam of Universities (EEU) in the Indonesia context, namely Seleksi Bersama Masuk Perguruan Tinggi Negeri (SBMPTN). The test determines whether students are qualified to be accepted as university students or not. Therefore, SBMPTN is extremely strict and competitive because students all over Indonesia are eager to pass it. The current study explores washback effect of SBMPTN. In order to gain the data of this survey study, an adopted questionnaire was conducted to a number of 30 senior high school teachers. The data analysis using Statistical Package for the Social Sciences (SPSS) was then explained into descriptive statistic encompassing mean score and percentages. The findings revealed that SBMPTN, as a high-stakes test, has a positive washback on Senior High School English teachers’ attitudes and their teaching methods.


INTRODUCTION
Test and language teaching-learning process are inseparable since the test is the most widely used procedure to evaluate the teaching and learning process.Tests are divided into some types based on their use or characteristics, namely standardized and non-standardized test.The non-standardized test includes teacher-made test the purpose of which is to measure learning outcomes of particular teaching material in the classroom.This type of test generally aims to gauge students' learning progress which leads to their learning improvement.On the other hand, standardized tests require particular standard objectives which are conducted through many forms tests (Brown, 2003).Ujian Nasional (National Exam, in translation, henceforth UN) and Seleksi Bersama Masuk Pergururan Tinggi Negeri (University Entrance Exam, in translation, henceforth SBMPTN) are examples of standardized tests in Indonesia.The importance of UN and SBMPTN is high because their results determine whether students are eligible to move to higher levels of education.
Test results can be interpreted in several ways such as in a grade or a test score.Grades can be defined as truly significant indicators such as students' intelligence.Therefore, when tests "carry serious consequences for students or educators," Marchant (2004, p. 2) defined them as high stakes test.In line with it, both UN and SBMPTN are considered high-stakes tests because students' career success can be determined based on these two tests, especially SBMPTN which is the university entrance test.However, UN, recently, might not be considered as high-stakes since it is currently not the sole gatekeeper for a higher level of education (Suriyanto, 2016).In terms of its acceptance rate, only 15.78% of the total number of students who participate in SBMPTN are able to pass the test (Alfiyah, 2014).This is extremely lower in comparison with that of UN which is 99.52% (Meirina, 2014).This indicates that SBMPTN is far more difficult and competitive than UN.SBMPTN tests students' aptitude, Mathematics, Bahasa Indonesia, English, Science, and Social science.Administering any test will have some effects.The effect of a test, either positive (beneficial) or negative (harmful), in teaching and learning is called washback (Taylor, 2005).Considering washback in high-stakes tests is very important because the effect will be significant to the community.
In teaching and learning, the term "washback" is generally termed the impact of testing.Messick (1996, as cited in Tayeb, Aziz, Ismail, & Khan 2014) describes the idea of washback as "the extent to which the introduction and use of a test influence language teachers and learners to do things they would not otherwise do that promote or inhibit language learning" (p.85).Referring to it as the testing influence, Beikmahdavi (2016) states that washback is related to the consequences of test concerning the learners and teachers' behavior in the classroom context which is classified into bipolar concept, as it can result in positive (effective) washback or negative (harmful) washback.Hamp-Lyons (1997, as cited in Weir, 2005) also adds that washback is one form of impact encompassing not only every aspect of instruments but also scoring procedures.On the other hand, Tayeb et al. (2014) argue, "washback is an inevitable trap from which participants have no escape because it gains its influence from the power of tests themselves" (p.86).It is important to have a clear distinction between washback and impact in language testing.While impact covers the language test effects on "individuals, policies, or practices -within a classroom, a school, an educational system or society as whole" (Wall, 1997 as cited in Cheng, Watanabe, Curtis, 2004, p. 4), washback covers the language test consequences on teachers, learners, teaching process (including materials), learning, and results (Hawkey, 2006).In a word, washback is the influence resulted from testing practices in terms of teaching and learning process.Bachman and Palmer (1996) indicate that the effect of test is mainly categorized into two types: micro level and macro level.The micro level is related to the persons directly influenced by a certain test use.This includes the test users, test takers, and teachers.Tayeb et al. (2014) agree that teachers and learners play an important role in the teaching-learning issue as the heart of the whole educational environment.On the contrary, the macro level involves the educational system and society which can be influenced by the consequences of test use.Therefore, Bachman and Palmer (1996) suggest that teachers are supposed to deliberate on the characteristics of the particular testing situation.As pointed out by Bailey (1999), washback is an educational phenomenon that can influence: (1) The 'washback effect' concept which is related to the effect of tests on both learning and teaching; (2) The 'measurement-driven instruction' concept which deals with the idea that testing should drive learning; (3) The 'curriculum alignment' concept which emphasizes on the correlation between the teaching syllabus and testing; and (4) The 'systematic validity'concept which incorporates tests into the educational system.Beikmahdavi (2016) summarizes the washback functions into several points.Firstly, test is considered a well-known means which is involved in making decisions and plans for several determined objectives in teaching and learning context.Secondly, washback could contribute to schools, students, and teachers to have preparation for the test.Thirdly, washback is beneficial to improve the students' competence and knowledge as the result of the test demonstrates the student needs and demands.The fourth function is to place someone in higher education.Lastly, washback can be used to measure and promote the effectiveness of teachers and schools.
The central point of this paper is the washback which is associated with a high-stakes test, as a type of test used to make major decisions about large numbers of individuals.Madaus (1988, as cited in Andrews, Fullilove, & Wong, 2002) refers to high-stakes tests as tests whose results "are seen -rightly or wrongly -by students, teachers, administrators, parents, or the general public, as being used to make important decisions that immediately and directly affect them" (p.87).This type of test obviously brings potential effect towards the language teaching practice.For instance, the entrance exam of universities conducted in Indonesia known as SBMPTN (Seleksi Bersama Masuk Perguruan Tinggi Negeri) has become a great phenomenon, due to its high competitiveness.The tight rules of SBMPTN motivate the students to have extra practices and be well prepared.This also affects the teachers and their teaching methods.Beside teaching methods and content assessment, a test leads to the change of teachers' perception of teaching which can affect their attitudes and motivation as well.

LITERATURE REVIEW
Azadi and Gholami ( 2013) conducted a study on the washback effect of English as Foreign Language (EFL) test on English Language Teaching (ELT) in Iranian high school students.EFL classrooms (n = 30) were randomly observed and a questionnaire was filled up by the teachers from those classes to obtain the feedback on the washback effect of the tests.Their results explained the negative impact in teaching practices as a result of the restrictions of teaching and teachers in Iranian schools to match the needs of the students for matriculation.Azadi and Gholami highlighted that the washback effect on teaching is negative as the English language tests on teaching materials seems to restrict teaching practices employed by the teachers.
Besides, the washback effect of high-stakes tests was inspected by Mahmoudi and Bakar (2013).They studied 6 female pre-university English teachers from two pre-university schools in Iran for their perceptions and attitudes towards Iranian National University Entrance Exam (INUEE).To obtain the data, a validated questionnaire was administered to the teachers and a semi-structured interview was also conducted in order to gain richer information, and to shed more light on the data collected through questionnaire.The results pointed out that the teachers had two perspectives.Four teachers held negative views about INUEE, whereas two other teachers saw this high-stakes test positively.
In addition, Andrews et al. (2002) explored an issue regarding the particular influences of high-stakes tests on the final-year secondary school students' spoken English performance.This study, in particular, focused on the specific context of the Hong Kong Advanced Supplementary (AS) 'Use of English' (UE) oral examination which is a requirement for university entrance.The research findings pointed out that the UE oral affected in various ways from student to student.The research was coducted in two phases.The first phase was to investigate whether the addition of an oral component to the UE exam influenced students' spoken English performance at the end of Secondary 7 or not.Meanwhile, the second phase explored whether the effect of test innovation on students' performance was immediate or delayed.The results indicated that the washback was positive for some students in Phase 1 proven by students' improved performance, whilst the positive washback in Phase 2 was only to superficial learning outcomes such as the ability to conform to the requirements of the exam format, or to produce memorized phrases.Generally, the use of high-stakes test obviously brought influence on the students' spoken output, for instance, adjustment to the exam format.
Meanwhile, Kennedy and Lui (2013) examined the influence of the Beijing Matriculation English Test (BMET) as a high-stakes test on prospective university applicants.They focused on the students and teachers' perception on the function of final-year secondary school English classes in the BMET preparation and English skills improvement.The data obtained from questionnaires and interviews indicated that BMET caused the final-year English class students to prepare for the test, instead of improving their English skill.In a word, BMET was considered to have powerful effects on teaching activities in final-year secondary school English classrooms.
Furthermore, Tayeb et al. (2014) investigated the correlation between teaching and learning factors influenced by the washback effect of the General Secondary English Examination (GSEE).The results of this test determine the learners' academic future in Yemen, and therefore its washback effect is unavoidable.The data collected through interviews and questionnaires showed that the test particularly affected teaching methodology and learning styles.Tayeb et al. (2014) concluded that the washback effect of the exam was the component of the language teaching-learning process.
Finally, another study about washback effect of highstakes tests on teachers was undertaken in Iran by Salehi, Yunus, and Salehi (2012).They investigated teachers' perceptions on Entrance Exam of the University (EEU).The results revealed that little attention was paid to language skills in EEU which implicitly and negatively encouraged the English teachers to teach to the test instead of improving their learners' language skills (Salehi et al., 2012).As this brief review of literature indicates, washback studies in the educational context of Indonesia are lacking.This motivated the present authors to investigate the washback effect of an EEU in Indonesia which is known as SBMPTN.

Objective and Research Questions
This research investigates the washback effect of high-stakes test, which is SBMPTN, on teachers' attitudes and teachers' teaching methods applied.Therefore, the research questions formulated are: 1.How does SBMPTN affect the teachers' attitude?2. What are the washback effects of SBMPTN on the teachers' teaching methods?

METHOD
This study used descriptive quantitative approach involving 30 senior high school English teachers as respondents to the study questionnaire.In accordance with the needs of this survey study, the selected participants were the teachers who teach English in the third grade of senior high schools.It was ensured that the selected participants were teachers who directly faced the influence of the entrance exam of universities (SBMPTN) in their teaching process.Furthermore, for the feasibility of this study, the English teachers were from a widespread region within Java and Sumatra.They filled in a 16-item questionnaire in terms of their perception of SB-MPTN as a test which is addressed to the third grade students of senior high school.Besides, the questionnaire provided the demographic information and teachers' teaching condition currently.The distributed questionnaire was an adopted validated questionnaire from Salehi et al. (2012) which was divided into two parts.The first part discussed the possible effects of SBMPTN on exam practices and teaching syllabus while the second part dealt with the teachers' insights into the implementation of certain teaching methods based on the context of SBMPTN.The data of the questionnaire was analyzed using Statistical Package for the Social Sciences (SPSS) software (version 22).From the teachers' answers, the general perspective of senior high school English teachers' towards the washback effect of the entrance exam of universities could be identified.Each item of the questionnaire was featured with a five-point Likert scale of agreement, in which strongly disagree is rated 1, disagree is rated 2, undecided is rated 3, agree is rated 4, and strongly agree is rated 5. Besides, the Cronbach's alpha for the questionnaire was 0.889 meaning that it had acceptable internal reliability.

RESULTS
The respondents in this research were dominated by female English teachers (20 teachers = 66.67%) in comparison to 10 teachers or 33.33% of male teachers.Most of the teachers were in between 20 -30 years old (26 teachers or 86.67%) while the rest, 4 teachers or 13.33%, aged 31 -40.All the respondents had BA for academic qualification, taught English in the final-year senior high school.18 teachers or 60% used half English and half Indonesian as medium of instruction in their classroom whilst 12 teachers or 40% chose to use English with Indonesian supplementary as explanation.
The first part of the questionnaire aimed to explore the teachers' attitudes to the entrance exam of universities.The questionnaire contained several statements which interpret forms of pressure or extra work due to SBMPTN when teaching took place.The descriptive statistics results are presented in Table 1.The teachers agreed that they greatly focused on the build-up to SBMPTN lesson since the test is deemed to be a high-stakes exam.It can be concluded as its mean score is the highest of all (M = 4.40, SD = 0.49).Besides, it seemed that the English teachers tended to pre-  1, SBMPTN had both positive and negative effect on teachers' teaching attitudes.SBMPTN supported the teaching activities in the classroom since the teachers were encouraged to do more lesson preparation, prepare more materials, revise some of the existing materials, and apply new teaching methods.Moreover, the teachers kept following the teaching syllabus even though SBMPTN contained different material from the one taught in the school.On the other hand, SBMPTN had negative washback either.The teachers admitted that they faced new challenges which refer to difficulties in teaching due to SBMPTN.Those challenges involved setting up new teaching objective and organizing more exam practices.In a word, the researchers concluded that SBMPTN mostly had positive washback on the teachers' attitude in English teaching process.
The second part of the questionnaire explored the teachers' point of views regarding the adoption of teaching method because of SBMPTN.As the Table 2 portrays, the teachers suggested that they encouraged more students' participation in class (M = 4.50, SD = 0.50).According to the Table 2, the teachers indicated that they put more stress in the integrated skills (M = 4.50, SD = 0.50) encompassing four skills such as reading, writing, speaking, and listening.
Besides, communicative approach was also considered to be used in their teaching (M = 4.17, SD = 0.87).Teachers taught in accordance with the test format (M = 3.90, SD = 0.95).Another concern is dealing with the less emphasis on reading comprehension activities (M = 3.83, SD = 0.74) as the test format consists of several reading test.The Table 2 also presents that the teachers did not really prioritize new methods to be applied in their teaching in preparing SBMPTN (M = 3.77, SD = 0.43).In addition, communicative skills and real life language tasks were tended to be neglected in the teachers' ways in teaching.Table 2 shows their low mean scores (M = 3.73 and 3.63 consecutively, SD = 0.94 and 0.96 respectively).Additionally, the mean score of the teachers' perception of adopting teaching methods due to SBMPTN (M = 4.01, SD = 0.56) with minimum and maximum values ranging from 3.13 to 4.75.
From the data above, it is proved that there was negative washback on the teachers' perceptions of adopting teaching methods due to SBMPTN.The teachers were influenced to teach according to the test format by focusing more on reading comprehension activities as the material tested in SBMPTN.Regardless of those influences, SBMPTN generally had positive roles in teaching methods.The teachers' perspectives on SBMPTN mainly pointed out that SB-MPTN made them to encourage more students' participation in class, put more emphasis on the integration skills, use a more communicative approach in teaching, employ new teaching methods, and adopt more real life language tasks.In accordance to the first part of the questionnaire, the research resulted to conclusion that SBMPTN primarily had positive washback on the teachers' way in adopting teaching methods.

DISCUSSION
According to Person (1988, as cited in Cheng et al., 2004), "Public examinations influence the attitudes, behaviors, and motivation of teachers, learners, and parents" (p. 7).As the data above indicated, the senior high school English teachers mostly considered that SBMPTN triggered them to do more lesson preparation for SBMPTN.It was followed with the preparation of additional material of SBMPTN for the students.This indicates that teachers make many efforts to fulfill the students' needs to pass SBMPTN.These findings were in line with the idea stating that perceptions are also influenced leading to changes of the work the participants do, then finally it affects the form of the learning outcomes (Kennedy & Lui, 2013).In addition, high-stakes tests motivate teachers to teach according to the test, instead of the curriculum (Salehi et al., 2012;Azadi & Gholami, 2013;Kennedy & Lui, 2013;Tayeb et al., 2014).In brief, SBMPTN led the teachers to come with the perceptions that SBMPTN was a high-stakes test in which students had to succeed.
In addition, the results of the study revealed that the context of SBMPTN influenced the teachers to give a lot of attention to the integration of the language four skills namely listening, speaking, reading, and writing.It is interesting since the content of SBMPTN does not involve the tasks of speaking, listening, and writing.The tendency to teach the test was considered as positive for the sake of students' test success but negative for national curriculum goal.Cheng et al. (2004) stated that teachers could be affected directly and negatively high-stakes testing as teaching test cannot enhance students' general understanding although it succeeds in improving students' score.Besides, the teachers made an attempt to involve their students more in class activity.The majority of the teachers made a change in the way they taught and learning activities in the classroom.This explained that English teachers expect their students to be well-prepared for the test.The data also revealed that the teachers were keen to apply communicative approach as an effective way to teach.According to the principles of Communicative Approach, successful language learning can be achieved through having communication in real and meaningful contexts.Therefore, the result is seen as positive washback since the teachers taught based on curriculum regardless the existence of SBMPTN with its format which focuses only on reading comprehension.Nevertheless, the results revealed slight negative washback of SBMPTN.It can be seen from the data that revising some of the existing materials, experiencing new challenges in teaching, and arranging new objectives of teaching were reported to be the negative effects of SBMPTN in their teaching.The teachers generally agreed that SBMPTN made them experience several new challenges, as they have to consider the existing teaching material, adjust the teaching objectives to suit SBMPTN, and arrange additional exam practices in preparation for SBMPTN.Importantly, the teachers still decided to teach according to the syllabus of teaching which, to some extent, had different teaching objectives from SBMPTN.In this case, the term 'teaching to the test' is emerged, meaning that the teachers teach a certain material which might not meet the teachers' or the instructional program's values and goals, but refers to a specified test use.It means that the test brings negative influence as the teaching process is mainly dominated by the test effect.SBMPTN made the teachers teach in accordance with the test format which is a multiple-choice test.Therefore, teachers focus on the activities of reading comprehension aiming to help students pass the test as its format mainly deals with reading test where several models of the test are designed to test students' reading skill.Weir (2005) shares the same idea that "teachers may simply not teach certain important skills if they are not in the test" (p.18).This notion occurs since teachers believe that when they teach material which is not related to the test, this is considered to be low-test authenticity.As the data presented, it reveals that SBMPTN leads to negative impact as the test drove the teachers' perceptions of their attitudes toward SBMPTN and their way in adopting teaching methods due to SBMPTN.
The overall findings of this study share contrary findings from previous studies conducted by Salehi et al. (2012) and Azadi and Gholami (2013).This study found that the teachers' attitudes towards SBMPTN mostly resulted in doing more preparation of lesson and materials for students.Besides, the teachers concerned that the students' participation in class was an essential role in creating ef-fective teaching learning process.However, the study of Salehi et al. (2012) pointed out that the teachers mainly found that EEU caused pressure on organizing more exam practices and employing new teaching methods.Meanwhile, Salehi et al. (2012) noted that the teachers tended to put more stress on reading comprehension activities and teach according to the test format.Moreover, they also add that the English teachers in Iran gave little attention to three language skills such as speaking, writing, and listening as those skills are not tested in EEU.Similar finding is also found in a study carried out by Azadi and Gholami (2013) in Iranian schools.Their results explain that the restrictions of teaching and teachers in Iranian schools to meet the needs of the students just for matriculation result in negative impact in teaching practices.Furthermore, Azadi and Gholami (2013) note that the washback effect on teaching is negative as the English language tests on teaching materials seems to restrict teaching practices employed by the teachers.
On the contrary, in general, this research proposed the same idea with Andrews et al.'s (2002) that high-stakes test has positive influence.They encountered that high-stakes test promoted students' performance in spoken English as prerequisite for admission to university.In this study, the high-stakes test caused positive effects either.The teachers' attitudes and teaching method applied were influenced in positive way.However, Andrews et al.'s (2002) research subjects were students whilst this research explored English teachers.The various findings corresponded Bachman and Palmer (2010) who opine that the use of assessment will cause both positive and negative insights into process of preparing for the test, of taking the test, and of anticipating the results.In the end, where and how washback exists and exhibits within a particular educational context does affect whether the washback effect will be positive or negative.

CONCLUSION AND SUGGESTIONS
This study investigated teachers' views related to the washback effect of SBMPTN on the learning and teaching process in the classroom.Besides, SBMPTN refers to high-stakes test since the consequence of the test determines something important for the students which in this context is a university entrance gate.SBMPTN has a washback effect on learners and teachers' learning and teaching.Captivatingly, this study proposes opposite findings to the previous research investigating similar coverage even though the same instrument was employed to obtain the data.
Based on the findings of this research some suggestions can be made.First, broader coverage of research area needs to be considered since Indonesia is a huge archipelagic country.More studies are also needed to investigate the washback effect of the test on the students.Students' perspectives might contribute more as students are ultimate objective of SBMPTN.This would enrich our understanding of washback effect of SBMPTN as a high-stakes test.In the end, in depth investigation is required in order to obtain more accurate and precise data by conducting observation and interview.

Table 1 .
Descriptive statistics results of the teachers' attitudes toward SBMPTN

Table 2 .
The mean scores of teachers' perceptions of adopting teaching methods due to SBMPTN