2015-09-20 – I write lies for a living. No lie. People I’ve said this to say, “No, Steve, what you write is not meant to deceive.” But what I write is very definitely meant to deceive.
I write multiple-choice questions.
That’s not all I write, of course. I write online courses for adults. So there’s the content of the course that I try to keep free of lies. But then there are the quizzes and exams. Being online, we need a simple way to grade questions, so we use multiple choice, usually with four answer options. Most questions I write are in the form:
Which of the following is true?
In the test writing biz we call the questions “items” and we call the lies “distractors.” Oh, what a slippery slope! Three quarters of what I write is lies.
You might be surprised to learn that the hardest part of writing a good multiple-choice question is writing convincing lies (i.e., distractors). If you can easily see through the lies, you’ll answer my question with no trouble. I want you to have at least a little trouble. I want to deceive you.
Quizzes and exams are not the only place where multiple-choice questions are used. They are also used in surveys. Did you ever see a headline like this?
23% of Americans Think Elvis Was the Third President of the U.S.?
Well don’t believe it. The test-takers are not stupid, the test-writer is. Statistics like this come from questions like this:
Who was the third President of the United States?
A. George Washington
B. Thomas Jefferson
C. Barack Obama
D. Elvis Presley
The Elvis distractor is a signal to the test-taker that the test is not serious. So nearly a quarter of them select the ridiculous “lie” and then headline writers have their “proof” that people are stupid.
Multiple-choice questions are used for everything from the U.S. census to college entrance exams. And the reliability of these information-gathering devices all depends on how good the lies are.
Your child’s chance of getting into a good college depends on some test-writer writing believable lies and your child’s ability to separate the lies from the truth.
In all fairness, questions for high-stakes tests are supposed to be vetted so that questions that elicit Elvis as a 19th century president are weeded out. But the truth is that multiple-choice testing is used because it is cheap, not because it is good.
* * *
While we are on the topic of lies, I want to remind everyone that October 1 is Diogenes Day, the day we honor truth telling. The antithesis of April Fool’s Day. If you want to learn more about Diogenes Day, click here.
* * *
You say/I say—