Why aren’t we using our best technology to improve education?
GUEST COLUMN | by Denise Wydra
In 2013, Ron Maggiano left teaching after thirty-three years. He was just 4 years shy of retirement, but he couldn’t take it any more.
Maggiano, who taught history at Springfield High School in Virginia, is no slouch. He won awards for outstanding teaching from both Disney (2005) and the American Historical Association (2006). But despite his passion and expertise, he could no longer “cooperate with the standardized testing regime that is destroying creativity and stifling imagination in the classroom.” He was quoted in The Washington Post —
“Research shows that today’s students need to be prepared to think critically, analyze problems, weigh solutions, and work collaboratively to successfully compete in the modern work environment. These are precisely the type of skills that cannot be measured by a multiple-choice standardized test.”
Educators, parents, law-makers, students, and education advocates all decry the over-reliance on multiple-choice items in standardized testing. And what’s on the tests drives the curriculum: student spend hours memorizing the factoids and that are amenable to this format; their homework and in-class tests are often multiple choice as well.
If we care about education, our teachers and students should be supported by technology that’s at least as good as what we use to drive e-commerce.
But why does it have to be multiple choice?
Multiple-choice items are undoubtedly fast and easy to score. They also have the aura of being “objective,” in the sense that you either get it right or you get it wrong—there’s no fallible human judgment involved.
But test-making companies know how difficult it is to construct a reliable multiple-choice item. For starters, all ambiguities in wording and logic must be eliminated, and the distractors need to be equally plausible. There’s a good chance that many of the multiple-choice items students routinely encounter are confusing or misleading—so whether or not students guess correctly has little to do with what they know.
And what does it tell us that so much “coaching” for multiple-choice tests is actually about gaming the system, placing better bets? What are we really measuring?
Multiple choice to our core
The use of multiple-choice tests in large-scale assessment was originally intended to eliminate bias and to identify promising young recruits during World War I. It was a reasonable technology for its time and purpose.
In the past decade, creators of digital learning products have used this simple format as the engine that drives a wide variety of engaging and creative learning activities. Students can respond to videos, enter virtual worlds, play games, get help from their peers. Because multiple-choice is fast and easy for machines to score, students get immediate feedback, a huge boon.
But this is all built on a technology that’s conceptually over a hundred years old. You see a set of options and you click to place your bet.
We’re seeing exciting advances in education’s use of sophisticated technology. Big data can identify students at risk and highlight actions associated with success. Adaptive learning can provide each student with the optimal content for where she is in her learning path.
Yet none of this is helping to transform the core learning experiences and assessments themselves. Most adaptive products function by steering students to the most appropriate MC-based activity.
Shifting the baseline model
Here’s where education can learn from other sectors. Outside of the classroom, the use of Artificial Intelligence is growing rapidly. Medicine uses it to assist doctors in diagnosis and patient care. Journalism uses it to write new stories, and corporate America uses it to compose earnings reports. Amazon uses it to recommend new items to shoppers. And Apple uses it to help you buy movie tickets—and thus sell more iPhones.
Researchers have been excited about AI’s potential use in education for years. What we lack, though, are practical solutions that can be used to build the next generation of learning activities and assessments.
Imagine a classroom where students could get feedback on their projects as often as they wanted, before or after their check-ins with instructors. Imagine a tutoring system that could help students develop self-reflection and metacognition, not just quiz them with problem sets. Or an e-textbook that could engage in dialogs with students to assess and develop their deep understanding of the material.
Imagine a world where standardized testing actually assessed not only students’ deep conceptual understanding but also their ability to solve new problems and make new connections?
If we care about education, our teachers and students should be supported by technology that’s at least as good as what we use to drive e-commerce. We need to get moving on the use of Artificial Intelligence and other advancements to improve learning experiences, not just churn through multiple choice tests more quickly.
Personally, I’d be willing to give up on Siri’s ability to help me make dinner reservations if it would help students with critical, creative thinking instead.
—
Links:
http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/06/24/life-is-not-a-multiple-choice-test/
http://www.wired.com/2014/06/ai-healthcare/
—
Denise Wydra is currently COO at Cognii, a technology company focused on using artificial intelligence to increase the availability of high-quality learning experiences. Previously she was VP of print and digital product at Macmillan Higher Education and Editorial Director at Bedford/St. Martin’s, a college publisher, where she focused on innovative solutions for teachers and students.
Thanks for your comment, Harry. The great thing is, the ability to give students intelligent feedback on their work and to assess their creative problem-solving is within the capabilities of today’s Artificial Intelligence. It doesn’t have to be a pipedream (or a novel), if we focus on making it a reality.
AI has become both our great hope and a potential scourge. Just look at the warnings from Steven Hawking. I think it’s overstated in both directions. On the scourge side, plenty of intelligent people have settled down to the position I raised a while ago in my ETCJournal.com article. AI taking over the world is vastly overrated. AI as killing machines (like the Daleks of Dr. Who) is a real and present danger.
Similarly, AI as the great tutor machine, as suggested by Prof. Alfred Bork, is so far into the future that we can dismiss it out of hand. In the meantime, AI under the radar is making gains in education, albeit slowly. Because the term AI is a bit fuzzy in the minds of many, I hesitate to write more. It should be more than clever algorithms. I think that clever algorithms can add much to education all by themselves.
Regarding multiple-choice testing, I have the opinion that you can test deeper thinking with multiple-choice testing. It’s very difficult to craft these tests, though. The alternative of long answers runs into the grading problem. The questions are relatively easy to write but much, much more difficult to grade. AI is hardly a white knight riding to the rescue here, not yet anyway. As I write, I see advice on my spelling and grammar. The advice on grammar is frequently incorrect. The spelling advice is rarely off but does make mistakes too.
Truly creative written responses that may be marked down by human graders will be even more likely to be denigrated by machine grading. As soon as you allow responses of more than a few words, the situation becomes overly complex, and the best students will be likely to score below those who are better at following rules and memorizing answers.
Writing multiple-choice questions and the answer choices requires art and skill beyond the abilities of most people hired to write them. Having written thousands of these questions for my business, I can attest to the difficulty. When we get comments about how good and how penetrating our questions are, I feel justified in spending all of those hundreds of extra hours on them. It would have been easy just to pop out the usual pap. That would have saved me enormous amounts of time. It would also have been counter to our mission to improve thinking skills, our primary mission that is above teaching science — the apparent goal of our work.
This is not to suggest that constructive responses are useless. On the contrary, they are a key part of learning to think well. Every one of our lessons requires these, but we do not grade them by machine. There’s no sense even to checking grammar and spelling because students already have these tools at their fingertips. For the present, I see software as playing an assistive role in grading essays and not as the grading engine for automating the process. The computer can highlight areas for human review and thus help speed the process. Computers can even “learn” which essays deserve higher and lower grades and so become an even better assistant, but they will mess up horribly on that occasional truly creative student who thinks beyond the boundaries of the question and comes up with a brilliant response. Hopefully, we can write software that will identify such material as being especially deserving of close human scrutiny.
Computers, like corporations, are not people.
I can imagine the above and much more. However, imagination is for writing novels, not examinations.