(originally published on March 14 2016 in Tabletmag.com; http://www.tabletmag.com/jewish-news-and-politics/198339/toefl-abuses-english-language)
TOEFL, a Desperately Dull and Profoundly Unfair Exam, Abuses the English Language
I tried to prep a pair of Afghan teens for American boarding school, but instead of teaching them the language they need, I spent all my time explaining cultural references and unfamiliar contexts
By Ann Marlowe
Tablet
United States
TOEFL, a Desperately Dull and Profoundly Unfair Exam, Abuses the English Language
I tried to prep a pair of Afghan teens for American boarding school, but instead of teaching them the language they need, I spent all my time explaining cultural references and unfamiliar contexts
By Ann Marlowe
March 13, 2016 • 10:00 PM
Facebook
Twitter
Pinterest
Google Plus
Pinterest
“Who is Alicia Keys?” Amina’s voice over the Viber connection to Afghanistan was faint but her puzzlement was clear. I didn’t think I was going to have to give lessons on American pop culture when I set out to tutor Amina, 16, and her brother Ahmad, 17, for the Test of English as a Foreign Language (TOEFL). Truth was, I didn’t know much about Alicia Keys either, and that hadn’t interfered with my life so far.
I’d met Amina and Ahmad in their home town of Mazar-i-Sharif, Afghanistan in 2002, when they were 3 and 4 years old. We took to each other from the start. And I kept visiting their family—and staying in their big family compound—at least once a year up until 2011. For most of that time, Afghanistan in general and Mazar in particular seemed to be on an upward path.
Amina and Ahmad both go to the local Afghan-Turk High School—run by the Turkish Gulenist movement and rather peculiar but the best option among the not very impressive selection of local private schools. Everything was going well for Amina and Ahmad until this year, when the Afghan government began what looks like a meltdown. This summer, the girls’ Afghan-Turk school received Taliban death threats, and the Taliban’s success in capturing Kunduz briefly this fall didn’t bode well for Mazar, just 100 miles away. So, this fall Amina and Ahmad’s parents asked me to help them apply to boarding school in the United States.
To apply to American boarding school or college, foreign students must take an English language test, usually instead of the PSAT or SAT. For all the boarding schools we were interested in (and for the vast majority of colleges) the TOEFL is required. While there are TOEFL junior and primary tests for ages 11+ and 8+ respectively, the TOEFL iBT, or Internet-based test, which Amina and Ahmad will take, is suggested for “16+”. They will be competing with students five or 10 years older—they are lucky enough to attend a local TOEFL prep course, and the other students are at the local university. The TOEFL IBT takes four and a half hours and incorporates reading, listening, speaking, and writing sections.
I’d assumed Amina and Ahmad would have a tough time with the TOEFL, as they have little experience with English. Uzbek, a Turkic language, is their home language. Dari—a dialect of Farsi and, along with Pashtu, one of the two official languages of Afghanistan—is what they speak outside the house. Turkish is the second language of their high school. They also study Pashtu in school, as it’s obligatory for Afghan students to study both Dari and Pashtu. But not English. So, I was prepared for an uphill battle to improve Amina and Ahmad’s English. And I adjusted to the fact that the Internet is slow in Mazar, and it takes a long time to download files or refresh a screen.
What I didn’t expect was the esoteric cultural content embedded in the test. In three months of working an hour a day with Ahmad and Amina, seven days a week, I’ve come to see the TOEFL as almost always dull and often profoundly unfair to students from the more remote parts of the developing world.
***
Alicia Keys came up in a practice reading selection on the Empire State Building’s switch to LED lights—an event which wasn’t exactly of earth-shattering importance for me or anyone else I know in New York. “What does Alicia Keys suggest the Empire State building is a symbol of?” the question asked. The explanation involved my explaining to the Afghan siblings why New York is called “The Empire State.” Another question about the lighting of the Empire State Building referred to the size of a Volkswagen Beetle. This also puzzled the young Afghans, as this isn’t a car available in Afghanistan.
It also took a lot of work—and I mean, a lot—to explain the worldview embodied in the test. Two of the 10 questions in the Empire State Building selection focused on the environmental effects of the building’s shift to LED lights, using the phrases “to become more green” and “environmentally conscious,” which make no intuitive sense to non-English speakers from the developing world. For those who live in a largely pre-industrial society like Amina and Ahmad, getting electricity is a much more familiar concern than saving it. (Viber makes our sessions possible even when the city power is down.)
The TOEFL, I came to see, is less a test of proficiency in the English language than it is a test of students’ familiarity with fashionable American media and pop culture references and with current topics like global warming and the greenhouse effect. For instance, the Empire State Building passage includes this:
The LED system has “16.7 million color possibilities, in digital combinations of ripples, sparkles, sweeps and strobes,” says Phil O’Donnell, of Burlington, Mass.-based Philips Color Kinetics that’s responsible for the system and worked with a resident lighting designer. It’s the sum of all possibilities – a huge palette.
While it is possible to imagine the person for whom this kind of breathless virtue-signaling might be a source of near-physical pleasure, it is hard to argue that properly parsing word-combinations like “ripples, sparkles, sweeps and strobes” is crucial for academic success in America. While I was trying to teach Ahmad and Amina grammar, vocabulary, and test smarts—read the first paragraph and the last, then look at the questions to see how many you can answer and what you need to attend to as you read the rest—what I couldn’t teach were dozens of references to things, places, and experiences that Afghans have no acquaintance with.
An internationally famous ballerina, Maria Tallchief, demonstrated that the quality of ballet in North America could equal those of the ballet in Europe.
If you don’t know whether a ballerina is a scientist or a dancer, or that the words “ballerina” and “ballet” are related, how are you supposed to answer the question?
The Educational Testing Service (ETS), the behemoth that runs the TOEFL, insists that cultural context has little to do with the accuracy of the test. ETS’ Manager of Public Relations, Allyson Norton, emailed answers to my questions about cultural context. She responded,
One of the key principles of passage selection is that the passage needs to stand alone, meaning no prior knowledge is needed to understand the passage. … Further, background knowledge of content within test passages does not significantly impact scores.
But is it realistic to think that students who have no background knowledge on any question, like Ahmad and Amina, generally do as well as, say, test takers from a European country?
Norton sent me an internal study that showed that:
reading passages were neither advantageous nor disadvantageous to those who had physical science backgrounds or were familiar with a certain culture, and this holds for both the lower and higher proficiency groups.
The paper that Norton sent me referred to two types of mental processing used in reading comprehension, top down and bottom up. Both are used in most reading. Those who rely too heavily on top down, context-based understanding may make errors. But Norton also sent me a document that proves my main point: a country-by-country breakdown of average scores on the TOEFL’s four parts. There is a clear correlation between countries that are poor, isolated, and less linked to the world economy and countries with low average TOEFL scores. No surprise, Afghanistan ranks near the bottom with an average score of 69 out of a possible 120.
The rankings are fascinating. It’s no surprise that Equatorial Guinea stands dead last at 59, but Saudi Arabia is also at 59. Meanwhile Israel is 93, just two points below English-speaking New Zealand, and the highest scores belong to Austria and English-speaking Northern Ireland tied at 100. So, what are these tests actually measuring?
As the variation in country scores suggests, the common-sense hypothesis that success on the TOEFL is correlated with coming from a richer, more developed, more globalized country is true. The TOEFL tests cultural knowing-ness as much as it tests English skills. Of course, there’s a correlation there, too: If your English is really good, chances are you spend time on English language websites, listen to English language songs, watch English language movies, and so on.
The research also shows that test takers who are applying to high school have the highest average scores, higher than applicants to graduate or professional schools. My hypothesis is that apart from anomalies like Ahmad and Amina, most foreign teens who apply to boarding school in the United States are from rich, sophisticated families who have sent their children to excellent schools where they are immersed in American culture.
There’s also another issue, which has nothing to do with cultural literacy. Many of the TOEFL reading selections are simply deadly dull. Some are poorly written, vague, and confusing; others are clear but concern subject matter like geology that would get very few page views if posted on an online news site. I can understand that graduate-student test takers ought to be ready to tackle tedious material—adults often have to read boring articles for work. But why is this necessary for teenagers applying to high school? Why not pick excerpts from famous speeches by Churchill or Lincoln? How about some easy English poetry? When I asked Amina and Ahmad to read some poems by Robert Frost, they were much more interested than in anything the TOEFL practice tests have put before them.
Ironically, though there is not much to say in favor of the Afghan school system, public or private—they don’t teach evolution, for instance; and brutal physical punishment, though officially prohibited, is not uncommon—Afghan students know well the glories of Persian literature. They study the great Persian (and Pashtu) poets. Ahmad and Amina have memorized poetry in Farsi and Turkish (the most celebrated Afghan poet, Rumi, also wrote in Turkish). It’s sad that nothing they have encountered in prepping for the TOEFL has given them a hint that English is a powerful, economical, and supple language that has produced libraries full of magnificent novels, essays, and poetry.