The Problem With Testing May Be A Failure Of Imagination

The Problem With Testing May Be A Failure Of Imagination

07/06/2013, TeachThought Staff

The Problem With Testing May Be A Failure Of Imagination

by Sara Briggs

What would you guess it takes to be hired at Google? A computer science degree? Academic prowess? The ability to arrive quickly and confidently at the right answer to a question?

Not so long ago, Google famously asked every job candidate for a transcript, G.P.A., and test scores before considering them for a position.

But as you might expect from a fundamentally data-driven company, Google regularly examines its own hiring methods, collecting and analyzing tremendous amounts of information from employees and adjusting its policies accordingly.

In 2011, Google released the results of a study called Project Oxygen, which showed that its old hiring model proved very little about a candidate’s potential for success.

“One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation,” said senior vice president for people operations Laszlo Bock in an interview with Adam Bryant of the New York Times.

“What’s interesting is the proportion of people without any college education at Google has increased over time as well. So we have teams where you have 14 percent of the team made up of people who’ve never gone to college.”

What’s more, the kind of “tests” hiring teams now use in interviews focus on behavior, not knowledge.

For example, managers now ask questions like, “Give me an example of a time when you solved an analytically difficult problem” instead of “How many bowling balls can you fit inside this room?”

“The interesting thing about the behavioral interview is that when you ask somebody to speak to their own experience, and you drill into that, you get two kinds of information,” says Bock. “One is you get to see how they actually interacted in a real-world situation, and the valuable ‘meta’ information you get about the candidate is a sense of what they consider to be difficult.”

But aren’t we all trained in school, from an early age, to be able to demonstrate our competence by arriving at the right answer to a tricky question? Don’t the standards of the classroom prepare us for the standards of the real world?

“Brainteasers are a complete waste of time,” says Bock. “How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.”

Hold on. In every high-performing nation, standardized tests are embedded in the wiring of schools, particularly in high schools. In the developed world, 76 percent of students attend high schools that use them, according to the OECD. Granted, each country uses a slightly different system.

In West Germany, for instance, education is the responsibility of the states, rather than the national government. Standardized tests are not used on a population-wide basis, and the use of standardized tests is largely restricted to counseling centers and similar specialists in the schools.

Neither achievement nor intelligence tests are often used in the schools. The Netherlands created a national curriculum development center in 1975 and has created national examinations, although they are not yet widely used. Achievement tests are used by teachers only, and intelligence test use is similar to that in West Germany. In Sweden, national standardized tests based on objective techniques are used above the primary levels. And the list goes on…

Why would so many societies be using standardized tests if they didn’t predict anything important?

Daniel Koretz, a scholar in educational measurement, has been studying standardized testing models for over two decades.

“Standardized tests as we currently have them are enormously useful,” Koretz says. “Anyone who follows education knows that there are enormous inequities in American schooling. One of the ways we know this is because of standardized tests.

“The problem is not that the testing shouldn’t be used. I wouldn’t argue that testing shouldn’t be used, we’re just using it in the wrong way, putting far too much pressure on increases in scores.”

In the United States, where K-12 standardized testing has become a multi-billion-dollar industry, highly regarded and national institutions and associations such as the National Board on Educational Testing and Public Policy and the American Educational Research Association have been writing for decades about the pitfalls of both standardized testing in general and the over-reliance on standardized testing, especially in using these tests to make high-stakes decisions about things like high school graduation and teacher evaluation.

Finland uses a standardized testing system, but more effectively than most countries. Finnish high school seniors are required to take a standardized, national matriculation exam that determines their chances of attending a Finnish university. The exam stretches out over three grueling weeks and takes about 43 hours. Whereas in the United States, tests have greater consequences for teachers and administrators than for students, in Finland, the students are the ones who feel the pressure.

Tests are hard, and they affect students’ lives. The same is true in South Korea, the other top-performing nation on international tests.

But is creating a better test really the answer? Maybe it’s not the Finnish testing system that makes a difference but the selectivity of its teacher-preparation programs. And even if it is the Finnish testing system, who’s to say that what’s being tested is what students will actually need to succeed in their careers? Even if every student in the developed world aced his or her college entrance exams, would they be able to land a job at Google? McDonald’s, even?

In his book, Creating Innovators: The Making of Young People Who Will Change the World, Harvard education specialist Tony Wagner says many of today’s K-12 and college tracks are not consistently “adding the value and teaching the skills that matter most in the marketplace.”

Because today’s job landscape is changing so fast, and because high-paying, middle-skilled vocations are fewer and father between, it is absolutely imperative for young professionals to be able to solve problems creatively and think critically.

The capacity to innovate, Wagner says, is far more important than academic knowledge.

But how do you measure innovation? How do you set standards for it, define it, teach it?

Can it become part of each school system’s curriculum the way that reading, math, and writing are?

That’s part of the exciting challenge facing us as teachers. The fact that we haven’t found a solution the least bit elegant, creative, or compelling is a telling failure of our collective imagination.

This is based on a post that first appeared on


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: