Curbing assessment data waste with technology.
GUEST COLUMN | by Eric Ermie
There is a lot of interest in higher education about the use of data in assessments of student performance. Data can reveal much about the performance of students, the integrity of an exam, the strength of a course and the progress of an institution. For those reasons I am a proponent of using data, but I also understand the challenges. Collecting data is only half the battle. Without the proper tools data can be overwhelming, creating an endless maze where it’s easy to get lost and feel overburdened. But, technology can play a vital role in diminishing data waste and helping create efficiencies for faculty members and institutions alike.
The first step is to move faculty members away from using Word documents or text files to write exam questions and simple spreadsheets to track results. A great deal of information can be tied to an exam question if the questions are stored in an online platform designed to support assessment. Building a library of exam questions can also help streamline the creation of exams as faculty members can more easily access question archives and share questions across a department.
During my time at the Ohio State University College of Medicine, we used the ExamSoft assessment platform to build libraries of question items and then assigned specific tags that aligned course objectives to exam questions. This exam “blueprinting” allowed us to more purposefully integrate learning outcomes with instruction and exams. The model also allowed us to dive deeper into understanding student performance and inform academic advising. Instead of giving students a flat letter grade, faculty members could present a full report that provides the opportunity for greater analysis and engagement around a student’s strengths and areas in need of improvement.
Armed with assessment data, faculty members can make the most out of office hours by going through customized reports (provided with just one-click in ExamSoft), highlighting the gaps that exist for students and advising on strategies for improvement and where to focus study time. When the data shows strong performance in certain areas, there is an opportunity for faculty members to offer career guidance and suggest paths based on early aptitude for subject areas. At OSU students had coaches who met with them after each part of the curriculum to review performance and guide them using data.
Reports of assessment data can be generated at multiple levels, so in addition to identifying performance patterns for individual students, faculty members can also spot trends across classes and courses. Within the results from exams, you can see how well students are performing compared to their classmates as well as others taking the same course at the same institution. At OSU, the data added a level of pliancy to classroom instruction as reports enabled faculty members to use results of exams to refine their teaching, focusing in class on topics or subjects where students struggled and minimizing time on topics that were well grasped.
Exam blueprinting with a platform like ExamSoft can greatly improve the way exams are designed and refined over time. Having the ability to track the performance of one particular question or topic allows you to establish a barometer of exam performance. When a particular question is missed by a large percentage of students, the data helps faculty members determine whether students have a knowledge gap or if, simply, the question is flawed.
Technology can also provide a way for faculty members to uphold the rigor of their courses. There is a presumption that policies enacted around greater use of assessment are meant to boil down curriculum to measuring what is easiest. There is great value in subjective assessments, such as essays, presentations, videos, etc., that are typically challenging to systematically score. This month ExamSoft announced the addition of a rubrics platform that allows multiple types of assessment to be synthesized. This feature, which will be widely released in January 2014, enables the creation of rubrics templates, establishes transparency in grading and gives faculty members a more complete view of student performance. The expansion proves how powerful technology can be in making sense of assessment data and closing the feedback loop with students.
VIDEO: RUBRICS SNEAK PEEK
Faculty members invest a great deal of time in creating and grading exams to have so much data go to waste. Technology can help faculty and institutions create better student engagement and improve student performance and the examples above are just a few ways to more effectively use assessment data in higher education. As more faculty members are able to mine data and see its value, the more they can demand specific features in technology and the stronger and more robust platforms will become. The result will be less data waste, higher rates of student performance and better levels of student engagement.
—
Eric Ermie is the Director of Client Solutions for ExamSoft Worldwide, Inc. He spent five years managing assessment for the Ohio State University College of Medicine and has presented his work in assessment at the Association of American Medical Colleges (AAMC), International Association of Medical Student Educators (IAMSE) and the Central Group on Educational Affairs (CGEA). He can be reached at eermie@examsoft.com.
