What to consider when selecting assessment technology.

GUEST COLUMN | by Patrick Owen Chadd

Curriculum, instruction and assessment are all important elements in education. Lately, there has been a huge emphasis on assessment, and with good reason.

Creating appropriate and well thought-out assessments help to determine what objectives have or have not been mastered by the learners, and assessment data helps in performing the evaluation of curriculum, instruction and of the assessment itself.

It isn’t rocket science. If we think of objectives as describing where you are going, the assessment items are the means by which we find out whether we got there. The effective integration and implementation of assessment technology is critical to ensure that these tools can accomplish your assessment goals.

It is really important to figure out the needs of your institution in choosing assessment technologies, and it is important to start any process with an inventory of the assessment technology that your institution already has, how it is used and whether it is adequate or needs updating. In our case, we were very dependent ad nauseum on paper-based methods and spreadsheets. Basically, we desperately needed to update our assessment technology because we didn’t have any in place.

After an inventory audit, a detailed needs analysis must be completed. A needs analysis is the second step in figuring out how you are going to determine what assessment technology or technologies are needed. Needs analyses can be conducted from several different perspectives and using common data collection tools. Input should be collected from students, staff and faculty on assessment technology needs, so that there is a good global perspective on what all the stakeholders’ perceptions and thoughts are in reference to assessment technology at an institution. Some places to start include:

• Interviews with stakeholders to see where gaps or needs exist
• Focus groups that meet to discuss the future of assessment technology
• Technology planning team brainstorming sessions
• Surveys of the stakeholders

The needs assessment isn’t the time to determine exactly what products you should purchase and install, but rather to clarify where gaps exist that can be met with assessment technology. For example, if the results of the evaluation find that stakeholders complain about the time interval for getting grading accomplished for an exam and that stakeholders are spending hours a day grading, processing grades or waiting for grades, then a result of the needs assessment could be that assessment technologies have an ability to provide fast grading turnaround. Once the needs have been established, your institution can now pursue looking at the feasibility of choosing and implementing assessment technology.

Here are some criteria that I use in looking for assessment technology:

Ease of Use. Any technology should be first and foremost, relatively easy to use. If a product is difficult to use or has a steep learning curve; then it will not likely be used. If a product isn’t used, then it isn’t going to accomplish your goals.

Requires Minimal Training. This goes along with ease of use. Rarely is any data collection or analysis technology immediately intuitive, but it should ideally require only a minimal amount of training to get up and running. The faster stakeholders can get up to speed, the faster the technology will be integrated into the environment.

Secure, Stable and Reliable Environment. Data is important. The ability to access and archive data and to have redundant backup of the data in the event of any catastrophe is very important. If there are no back-up contingencies, then the collected data is always at risk of being lost or corrupted. The technology also has to be stable and work reliably. “Buggy” technology is technology that is rapidly abandoned.

Scalable and Flexible Platform. The technology should be scalable and flexible. The goal of most institutions is to increase enrollment and so any technology must be easily scalable and be flexible enough to meet the majority of the needs of stakeholders.

Integrate With Existing Infrastructure. This is a very important point. Our network is very secure, and due to the nature of what we do, those security measures cannot be compromised. Any technology that is introduced into our environment must integrate smoothly into our existing infrastructure or faculty will not implement it.

Robust Reporting and Data Collection. Assessment technology should collect and provide robust data, and it should be able to be filtered and manipulated to provide different perspectives of the data collected.

Support Resources. Any technology that is implemented must often be able to be implemented and supported with existing resources so support of the technology in terms of FTE and by the vendor is always a consideration.

Cost. It’s not a big surprise that cost is always a factor and always plays into the equation. But, remember to account for all costs, including licensing, use, archiving and support.

Hosted/Cloud Environment. Cost hosted and cloud systems are preferred both for security and ease of management. Servers are expensive; require maintenance, special environments and regular replacement.

Support Resources. Any technology needs support and support resources needed by the institution and provided by the vendor are an important element in any technology procurement and implementation.

ROI. Finally, what is the return on investment? The ongoing use of a product must provide a regular return on investment. For budget and staffing justification as well as institutional benefit, a product must continually provide value and a good return on investment.

I have found that by meeting these criteria in selecting any technology, my recommendations are moved forward and ultimately approved.

—

Patrick Chadd, M.Ed., is manager of Academic Systems and Educational Technology at
Rowan University, School of Osteopathic Medicine, in Stratford, New Jersey. Write to: chaddpo@rowan.edu