Application and Effect of Proposal on Dynamic Spoken English Tests in Vocational

[a]Beijing City University， Beijing， China.* Corresponding author.

Sponsored by Beijing City University， China.

Received 9 July 2013； accepted 1 December 2013

Abstract

This study aims to establish a systematic proposal on spoken English tests for vocational English teaching with content， form， difficulty level and scoring criteria characterized by dynamic changes. The proposal is first designed and then applied in four consecutive tests. Its backwash effects on the English classroom teaching is examined. The research subjects are freshman of nonEnglish majors in a vocational college. After the trial program， a survey was conducted among the participants to collect the feedback on the effect of the proposal in terms of its content， form and scoring method， including holistic marking and analytical marking. Results showed that the establishment of the systematic proposal on spoken English tests has a positive backwash effect on vocational English learning and teaching. It is suggested that regular spoken English tests in vocational English teaching be holistically designed.

Key words： Vocational English； Spoken English test proposal； Holistic marking； Analytical marking

GAO Liqun （2013）. Application and Effect of Proposal on Dynamic Spoken English Tests in Vocational College. Studies in Literature and Language， 7（3）， 77-80. Available from： http：//www.cscanada. net/index.php/sll/article/view/j.sll.1923156320130703.3931 DOI： http：///10.3968/j.sll.1923156320130703.3931

INTRODUCTION

With the ever-strengthening emphasis on spoken English combined with the increasing proportion of listening and speaking in English teaching， evaluation system of English has been reformed in vocational schools in recent year. Oral test is adopted gradually. As a new test form for the English teaching in vocational colleges， an in-depth study on refined and systematic spoken test content， form， scoring criteria and organizational methods has not yet been conducted.

Based on the last few years’ test conducting experience in some vocational college and the related study， the present study aims to establish an Oral English Test Proposal （OETP） for one academic year of vocational English with four consecutive spoken tests’ content， form and scoring dynamically changed. After the establishment of OETP， it is applied for one academic year to freshmen of non-English majors in a vocational college and its effect was surveyed after the application.

1. LITERATURE REVIEW

Related research on vocational spoken English test started from 2004. Studies mostly discussed the status of vocational English exam institutions and problems. The explored issues include the improvements in written tests， reform in evaluation methods and to add oral test， such as attempts in designing oral form and content. Many researchers have pointed out the need for vocational English oral exam innovations. However， so far no overall Vocational English spoken test based on consecutive regular spoken tests in one academic year has been made. Moreover， there is no comparative study on the effect of the application of the previously practiced spoken English test and the reformed one.

1.1 Present Situation and Problems Related to Spoken English Test in Vocational College

In a related study， the current status and problems of spoken English test in vocational colleges reflected in related studies are basically the same. Ma （2013） generalized the current status of spoken English test in vocational colleges， which objectively reflects the current problems in the field， that is the spoken test carried out in vocational colleges lacks of reliability and there is inconsistence in the test implementation conditions in each class as well as the institutions are inconsistent， such as the candidates answer time， the examiner professional level， no unified scoring criteria adopted， and the lack of validity of the test as well as the tested spoken language content being seriously out of line with practical work， which leads to the difficulty for the test to exert a positive backwash effect on teaching. The research thus made an attempt to employ the test form of interview between two to one， two to two， and two teachers to one to three test takers and developed a scoring criterion according to the pronunciation， intonation， vocabulary， grammar， fluency and ability to interact socially and other factors.

Wang （2010） analyzed the problems of oral expression of students in spoken tests. He holds that there are four main reasons： the impact of the mother tongue， the lack of training and developing thinking skills， the lack of a solid oral training， especially training at oral discourse level as well as being too stressed when taking tests and thus lack of resilience. He believes that teachers should focus on these aspects in English teaching and readjust the focuses accordingly.

1.2 The Integration of Teaching Content in Regular English Class with Spoken Tests

There are also related discussions on the integration of teaching content in regular English class with Spoken tests in vocational colleges， such as Zhang and Wang（2012）. In these studies， it was pointed out that there are still many problems in the existing vocational English teaching， including the teaching forms and means being too simple； the lack of progressive level in teaching content； low efficiency of classroom teaching. The study made a try in integrating classroom teaching and oral examinations and made proposal to allow students to understand the purpose of the English language teaching， enhance their linguistic awareness and integrate oral tests with classroom teaching. In the study， the methods to get each taught unit specific content combined with regular oral tests are introduced， including individual quiz， picture quiz， quiz of specified topic and so on.

1.3 Backwash Effect of Spoken Testing on Classroom English Teaching in Vocational Colleges

The first researchers to study backwash effect of language tests， namely the impact of testing on language teaching， were Alderson and Wall. They conducted study in this field and made 15 kinds of assumptions backwash effect， involving testing on teaching and learning in content， methods， speed and order， and the extent and depth of attitudes implications （1993）. They also believe that the test itself may have a dual washback effect on language teaching， both positive and negative. Prodromou（1995） proposed the concept of dominant and recessive backwash effects. The former refers to a direct， obvious to purpose for tests； the latter refers to the deep-seated， obvious influence， which impacts teaching and learning everywhere and imperceptibly. In Prodromou’s view， whether backwash effect is dominant or recessive， their overall impact on the teaching and learning are negative.

A small number of studies have examined the domestic situation related to the backwash effect of spoken test on English teaching in vocational colleges， such as Lu（2011）， Zhang （2010）. Lu took South National University for example， introducing the final spoken English test at the academic year 2009 to 2010 in the school. The test adopted the form of dialogue making up with 2 to 3 students forming one group and each conversation length being 3 minutes. All the topics were related to the teaching content in textbook. Each test taker was rated based on his/her performance in terms of pronunciation， fluency， and the coherence of speech content to questions. The researcher holds that the spoken test administrated in the school has a positive backwash effect on both teaching and learning. For teachers， they can discover the strengths and weakness of the teaching measures taken in classroom teaching and problems students have， which helps them to adjust teaching contents and measures. Meanwhile， the spoken test is a reflection of the teaching skills， which provide the school evaluation reference. As teachers teaching the original model has been adjusted， the quality of teaching was improved. Concerning students， their interest in learning was stimulated. Different levels of students were able to participate in classroom exercise， such as oral activities and games.

From the above， it is known that the current domestic research in this field mainly focuses on the generalization of the present situation and problems existing in the classroom English teaching in vocational colleges. Few systematic proposals on the administration of spoken English tests have been made so far. This study aims to establish a systematic proposal on spoken English tests which are carried out in vocational colleges for a whole academic year so as to provide a unified reference for such tests.

2. METHODOLOGY

2.1 Research Questions

To establish a systematic proposal on 4-consecutivetime spoken English tests featured by “dynamic” changes in contents， forms and scoring scheme；

To examine the effect of application of the proposal on English teaching and learning in vocational colleges；

To test the effect of the application of the analytical marking designed in the proposal contrasted to holistic marking.

2.2 Subjects

Subjects are sophomores of non-English major from a vocational school. Two intact classes are randomly selected from the grade with a total number of 105 students.

2.3 Research Methods

2.3.1 Interview and Questionnaire

Questionnaire is employed to collect feedback of nonEnglish major of Grade 2011 in the college on English teaching and learning effectiveness.

Questionnaire and interview are adopted in collecting feedback of the participants in the present study on the effect of the application of the designed proposal in the study.

2.3.2 Experimental Method

Heaton （2000） mentioned the two scoring methods in evaluating spoken test： holistic marking and analytical marking. Two sets of scoring criteria are designed， each adopting holistic marking and analytical marking respectively. Experiment on the comparisons among the same examiner scoring with the two methods in scoring the same test taker and among 3 different examiners who using the same method in scoring the same group of students are carried out to examine the effect of the analystical marking method designed in the spoken tests proposal.

2.3.3 Quantitative Analysis

Data collected from the rating experiment are analyzed with Wilcoxon signed rank test and the Friedman test to test the significant differences among different groups of data.

2.4 Research Process

2.4.1 Carry Out A Preliminary Survey

College Students of non-English majors in Grade 2011 in a vocational college were surveyed with questionnaire to collect feedback of non-English major of Grade 2011 in the college on English teaching and learning effectiveness. Meanwhile the ways of the implementation of four times of regular spoken tests in 2011-2012 academic years were generalized. Based on the result of the survey together with the related studies， an overall impression on the setting of spoken English tests in vocational colleges was formed.

2.4.2 Draft Proposal on Spoken English Tests of 4 Consecutive Times

According to preliminary survey results and related research， the proposal on spoken English tests of 4 consecutive times in an academic year was made， including the test form content， implementation measures and scoring criteria in the 4 consecutive stages of tests. The proposal is featured with form， content， implementation measures， scoring criteria and difficulty level varying with the number of periodic examinations（see Table 1）.

2.4.3 Application of Designed Test Proposal

The designed spoken testing proposal was applied in the 4-consecutive-time spoken English tests on the Grade 2012 of the college.

2.4.4 Examining the Effect of the Application of the Testing Proposal

The effect of the application of the testing proposal were examined from two aspects： one is the backwash effect of the testing content and form proposed； the other is the effect of the analytical marking method designed contrasted to holistic marking.

3. RESULTS AND DISCUSSION

2.1 Backwash Effect of Testing Content and Form on Teaching and Learning

At the end of the trial program， the backwash effect of the testing content and form proposed on teaching and learning is surveyed among 100 students and 3 teachers who participated in the program through interview and questionnaire.

The effect of the testing content and form in the proposal were surveyed in terms of its effect in stimulating students’ learning motivation （class participation； efforts in preparing for tests）， in enhancing poor students’ self-confidence （impact on overall scores of the course） as well as testing difficulty level （degree of difficulty of the questions； preferences to some testing form）. Most students t that think that the application of the proposal in spoken tests inspired there motivation in English learning， and got their self-confidence enhanced （86%）； the most difficult testing forms for the keynote speech and drama performances （62%）； the easiest testing form is sentence reading and make-up dialogue （85%）. Overall findings consistent with the expected effect.

Through interviews， the three teachers participating in the program think that the designed spoken testing proposal gives consideration of students’ proficiency level. The proper percentage of contents in and out of textbook and the progressive process demonstrated in the design have made the achievement in stimulating students’interest and help make them more enthusiastic in taking the tests. Ensuring the validity of the spoken tests， the application of the designed proposal also helps improve the reliability of the tests and thus plays a positive backwash effect on both English teaching and learning.

2.2 Analysis of Data Collected From the Contrastive Experiment of Analytical and Holistic Marking

On the fourth stage of the program， a contrastive experiment of analytical and holistic marking effect was carried out. 3 examiners used the two methods to score the same 20-student group at the same time. The data was processed with the Wilcoxon signed rank test and the Friedman test in SPSS.

Results showed that scores by the three examiners adopting the two different scoring methods demonstrated consistency， with the average rank of them being 2.59， 1.69， 1.72， friedman statistic 9.186 and P value 0.010， Kendall W 0.287 by analytical method and the average rank being 1.53， 2.44， 2.03， friedman statistic 8.612， P value 0.013， and Kendall W = 0.269 by holistic marking respectively. It showed that there is no significant difference among the 3 examiners in scoring， which proves that the criteria they took while rating students’performance in the spoken test were consistent. However， the participant teachers think that analytical marking is more effective in helping them get to know students’problems in oral expression with English， which makes them adjust their focus in teaching.

CONCLUSION

In this study， a systematic spoken English testing proposal was made， applied and tested. Results showed that such a proposal with characteristics of dynamic change in terms of testing content， form， scoring criteria and difficulty level is applicable in the regular classroom teaching of English in vocational colleges for its positive backwash effect on learning and teaching. The spoken testing proposal designed in the study can offer a unified reference for the implementation of oral tests operated in vocational colleges and thus ensure the fairness of the assessment for all students.

Since the number of the subjects in this study is relatively small， and and the experiment lasted only for one year with a lack of communication with peer institutions， further study can be made with an expanded scope of subjects taking a longer experimental time with experience exchange with peer institutions so as to establish a more comprehensive and scientific spoken testing proposal for vocational colleges.

REFERENCES

Alderson， C.， & Wall， D. （1993）. Does backwash exist. Applied Linguistics， 14（2）， 25-36.

Heaton， J. B. （2000）. Writing English language tests. Beijing： Foreign Language Teaching and Research Press.

Lu， F. （2011）. Research on backwash effect of language test on language teaching. Foreign Language Teaching Exchange， 304（2）， 32-33.

Ma， L. （2013）. Necessity of the backwash effect of spoken English test in vocational college. Crazy English （Teacher Edition）， 1， 106-108.

Prodromou， L. （1995）. The backwash effect： From testing to teaching. English Language Teaching， 49（1）， 10-18.

Wang， L. （2010）. Spoken English test and college English teaching. Study of Test， 26，106-108.

Zhang， A. P. ， & Wang， Qin. （2012）. Integration of vocational English spoken tests and classroom teaching. Overseas English， 3（6）， 80-81.

Zhang， Y. （2010）. Backwash effect of language testing on language teaching. Philology Journal， 2， 20-21.