Assessment Resources: Russian and CALL

Philosophy and Rationale

As an instructor of Russian in the U.S., my interaction with assessment will likely be quite different from that of my classmates teaching English. The vast majority of Russian language testing is low-stakes, and generally for achievement and placement purposes. Here are some the tests that I have encountered as a learner of Russian, as well as the contexts in which they appeared: achievement tests in my college Russian courses; ACTFL OPI for placement purposes; ACTFL OPI for achievement purposes (which had no effect on grades); informal assessments of speaking over the telephone for proficiency purposes; and the Test of Russian as a Foreign Language for proficiency purposes (I took the test in order to have documentation of my level of Russian proficiency).

I would venture a guess that the majority of students studying Russian will only be exposed to achievement tests in their classes. Study abroad programs in Russian-speaking countries usually require that participants completed one or two years of college coursework in Russian; generally only study abroad and academic programs for advanced language study require evidence of proficiency for entrance purposes, usually in the form of the ACTFL OPI or a program-specific entrance exam. The Test of Russian as a Foreign Language (TORFL) is intended for foreign students pursuing higher education in Russia and for foreigners seeking Russian citizenship. While many American students may study abroad in a Russian-speaking country, completing a degree in Russia is extremely rare among Americans (I only know of two people who have done this).

I am personally interested in the TORFL because it comprehensively tests the four skills, in addition to grammatical knowledge, at six levels of proficiency. I have included information about the TORFL in this portfolio for that reason; however, I understand that only a small minority of my students will ever come into contact with this test, or any other high-stakes test, for that matter. The ACTFL OPI is the only test that is widely used with nonnative speakers of Russian; in addition to reviewing the test for Assignment I, I have included other resources on its structure and use. The assessments that will most often affect my students are achievement tests that I develop. Thus, my portfolio contains a variety of resources for test development and alternative assessments, such as authentic, portfolio, and self assessment. It also includes basic online assessments of Russian language proficiency and cultural background knowledge for curious students. I have collected these resources through database and web searches, from the course Moodle, and from colleagues; all of the resources available online have been saved on a social bookmarking website called delicious, and I will continue to accumulate assessment-related links on that site.

This semester, I am also enrolled in the CALL and Assessment course, where I have investigated the relationship between technology and language testing. My focus on CALL is apparent in the materials I have chosen to include here: from online applications for creating tests to articles on the use of CALL in assessment. Because of my background in CALL and my knowledge of assessment theory and practice, I anticipate professional involvement in web-based Russian test development in the future. I expect that computer-based testing will continue to expand at a rapid rate and I am prepared to participate in its development. I feel that the projects I have completed for both the CALL and Assessment and the Language Assessment courses as well as the online Russian tests I have found, all of which I have included in this portfolio, will inform any future involvement in this field.

I have also included links to several modules on assessment that I hope can serve as refreshers for myself and for introductions to this area for future colleagues and collaborators. Also to this end, I have incorporated class handouts on statistical analyses and item types in order to evaluate future testing endeavors and to use a wide variety of creative items in the tests I create. When I delve further into a specific kind of assessment, such as testing individual skills or using authentic or portfolio assessment, I have the necessary resources on issues and techniques in those areas as well.

As a language educator, I hope to use portfolio and self assessment in the future. In my literature review for the Second Language Acquisition course, I researched the theoretical underpinnings of the issue of autonomy in language learning, teaching, and assessment, especially as it relates to portfolio and self assessment. In Language Assessment, I became informed about the practical side of implementing these assessment methods through examples from teachers. I intend to make portfolio and self assessment staples of my teaching.

Because I will most often be assessing students in low- and medium-stakes contexts, one of my main purposes is to generate positive washback:  I want my tests to have a positive effect on teaching and learning (Bailey, 1998). I hope to do this by creating assessments that encourage reflection, promote autonomy, strengthen self-efficacy, and that are even fun! I think that by following Swain’s (1984) principles (start from somewhere, concentrate on content, bias for best, and work for washback) I can create such assessments.

I have worked to ensure that I will always start from somewhere by equipping myself with a range of resources and references that will enable me to examine the literature on and see examples of any type of assessment or item I decide to use. I intend to concentrate on content, specifically content that has relevance to learners’ lives. By this, I mean that I hope to incorporate real-life language, authentic tasks, and authentic materials in any assessments I create. In order to bias for best, I plan to provide students with clear instructions, ample time to complete assessments, and, most importantly, the flexibility they need to personalize their responses to items to the greatest extent possible. And again, to work for washback I intend to create tests that will be an integral part of the learning process. Thus, “teaching to the test” will really mean “helpings students to showcase their Russian language skills through a variety of means.” Their ability to do this will ensure their success on assessments.

While I had dealt with some of these issues when creating tests before taking the Language Assessment course, where I mainly focused on concentrating on content, I know that there is much room for improvement. I have included the tests I wrote this past summer, which I hope to improve upon at some point. Although they are far from perfect, they still hold up fairly well under scrutiny. When I wrote them, I was unable to critically examine these testing instruments; I have developed this ability in the Language Assessment course. I am confident that my newfound knowledge of assessment and skills in developing tests will serve me throughout my career as a language teacher.

References

Bailey, K. M. (1998). Learning about language assessment: Dilemmas, decisions, and directions. Boston, MA: Heinle.

Swain, M. (1984). Large-scale communicative language testing: A case study. In S. J. Savignon & M. Berns (Eds.), Initiatives in communicative language teaching (pp. 185-201). Reading, MA: Addison-Wesley.

Web Resources

My Delicious bookmarks related to language testing

Online Assessment Resources

CAL Foreign Language Assessment Directory

Criteria for the Assessment of Foreign Language Instructional Software and Web Sites

Dynamic Assessment: A Vygotskian Approach to Understanding and Promoting L2 Development

Northwest Educational Technology Consortium – Technology Assessment and Evaluation Resources

Practical Ideas on Alternative Assessment for ESL Students

World Language Assessment Program Descriptions

 

 

Online Assessment Tools

ANVILL National Virtual Language Lab

Rubric Generator

Multimedia Interactive Modules for Education and Assessment (MIMEA)

Quia Quiz Creator

 

 

Online Modules about Assessment

CAL – Understanding Assessment

CALPER – Advanced Language Proficiency Assessment

CALPER – Dynamic Assessment

CLEAR Test Development Set

NCLRC – The Essentials of Language Teaching – Assessing Learning

NCLRC Portfolio Assessment in the Foreign Language Classroom Module

 

Online Portfolio Tools

Center for Applied Second Language Studies LinguaFolio Online

Demonstrate – Online Portfolio Assessment System for Languages

 

 

Test of Russian as a Foreign Language

 

Description of the Test of Russian as a Foreign Language (TORFL) – School of Russian and Asian Studies

Zlatoust Publishing – Tests of Russian. Издательство «Златоуст» – Тесты по русскому языку.

 

 

Web-based Tests of Russian

 

Brigham Young WebCAPE (Computer Adaptive Placement Exam)

DIALANG Computer Language Tests (no Russian, but all EU languages)

DLI Online Diagnostic Assessment System

GWU Graduate Russian Tool Examination – Sample Reading Items

Russian Defense Language Proficiency Familiarization Guide

SRAS Online Testing Center for Students of Russian

Stanford University Russian Language Placement Test

Free Online Russian Testlets

http://www.learnrussian.ru/english/russian-test.html

http://www.lidenz.ru/online-test/index.php

http://www.rlcentre.com/quizenc.shtml

http://www.esl-languages.com/en/study-abroad/online-tests/russian-test/index.htm

http://www.kommersant.ru/k-vlast/vlast-test.asp

References

General

Beyer, T. R. (2000). From testing to assessment, from teaching to learning. In O. Kagan & B. Rifkin (Eds.), The learning and teaching of Slavic languages and cultures (pp. 285-294).  Bloomington, IN: Slavica.

Educational Testing Service. (1986). Advanced Russian listening and reading proficiency test. Final project report—Year 2. Office of International Education, Washington, DC.

Educational Testing Service. (1990). Intermediate Russian proficiency test. Final project report. Office of International Education, Washington, DC.

Klee, C. A. (2000). Testing in the foreign language classroom. In O. Kagan & B. Rifkin (Eds.), The learning and teaching of Slavic languages and cultures (pp. 229-236).  Bloomington, IN: Slavica.

 

American Council on the Teaching of Foreign Languages Oral Proficiency Interview

Longan, N. (1995). Novice situation cards: The scripted situation. AATSEEL Newsletter, 37(5), 7-9.

Savignon, S. J. (1985). Evaluation of communicative competence: The ACTFL provisional proficiency guidelines. Modern Language Journal, 69(2), 129-134.

Thompson, I. (1996). Assessing foreign language skills: Data from Russian. Modern Language Journal, 80(1), 47-65.

Assessing Listening

Thompson, I. (1995). Testing listening comprehension. AATSEEL Newsletter, 37(5), 24-31.

Assessing Pragmatics

Billmyer, K., & Varghese, M. (2000). Investigating instrument-based pragmatic variability: Effects of enhancing discourse completion tests. Applied Linguistics 21(4), 517–52.

 

 

Assessing Speaking

National Endowment for the Humanities. (1995). NEH-UT collaborative project for the development and the implementation of oral assessment materials. A manual and practical guide for the classroom testing of spoken Russian using authentic task-specific assessment rubrics. Texas University, Austin. Department of Slavic Languages.

CALL and Assessment

Dooey, P. (2008). Language testing and technology: Problems of transition to a new era. ReCALL 20(1), 21-34.

Zabaleta, F. (2007). Developing a multimedia, computer-based Spanish placement test. CALICO Journal, 24(3), 675-692.

Self-Assessment

Interagency Language Roundtable Self-Assessment – Listening Proficiency

Interagency Language Roundtable Self-Assessment – Reading Proficiency

Interagency Language Roundtable Self-Assessment – Speaking Proficiency

Rivers, W. P. (2001). Autonomy at all costs: An ethnography of metacognitive self-assessment and self-management among experienced language learners. Modern Language Journal, 85(2), 279-290.

 

Portfolio Assessment

Moore, Z., & Bond, N. (2000). The use of the portfolio in a middle school Russian classroom: A case study in attracting and retaining students. In O. Kagan & B. Rifkin (Eds.), The learning and teaching of Slavic languages and cultures (pp. 237-254).  Bloomington, IN: Slavica.

Study Abroad

Brecht, R. D., & Robinson, J. L. (1993). Qualitative analysis of second language acquisition in study abroad: The ACTR/NFLC project. NFLC Occasional Papers.

Ginsberg, R. B. (1992). Language gains during study abroad: An analysis of the ACTR data. National Foreign Language Center Working Papers.

Ginsberg, R. B. (1992). Listening comprehension before and after study abroad. National Foreign Language Center Working Papers.

Test of Russian as a Foreign Language

 

Sample Tests of Russian as a Foreign Language – Second Level

Sample Tests of Russian as a Foreign Language – Third Level

Sample Tests of Russian as a Foreign Language – Fourth Level

National Standards for Russian as a Foreign Language. Elementary Level.

National Standards for Russian as a Foreign Language. Basic Level.