How higher ed can avoid the IT/analytics ‘lock-in’ with three demands to vendors.

GUEST COLUMN | by Timothy D. Harfield

In a recent Future Trends Forum discussion with Bryan Alexander, George Siemens expressed concern about lock-in: a situation in which technology investments become so integrated with the business practices of an institution that disentanglement becomes all but impossible. Where hyper-rationalized approaches to data-driven decision-making come together with inflexible technological ecosystems characterized by a lack of interoperability, what we end up with is a dystopian future in which colleges and universities are unable to change their investments. “Once you integrate your learning systems with your business practice systems, you don’t change anymore. When was the last time someone changed their

How can institutions invest in technology, while at the same time avoiding lock-in and preserving the agility they need to promote the success of their students under conditions of constant change and uncertainty?

accounting system? You do it once every 20 years and it is hell. The learning analytics systems vendors are integrating with systemic and organizational software, and you are never changing again.” With lock-in, competition ceases, innovation stops, and students lose out. As Jaron Lanier so eloquently remarked in You Are Not A Gadget, “The process of lock-in is like a wave gradually washing over the rulebook of life, culling the ambiguities of flexible thoughts as more and more thought structures are solidified into effectively permanent reality.” With increased lock-in comes an increased inability for technology to adapt to change, and an inability on the part of teachers to use technologies as their pedagogies demand.

In a 2015 blog post entitled “Next Generation Learning Analytics: Or, How Learning Analytics is Passé,” I articulated many of Siemens’ same concerns. In that piece, I observed that what learning analytics looks like for many institutions is a major investment, the success of which hinges upon the widespread adoption of a single learning management system combined with standardized teaching practices that are optimized for data collection. Such an approach to learning analytics — and to learning management in general — is inconsistent with the needs of 21st century learners. A major risk of data-driven approaches to education is that they view student success strictly in terms of graduation and retention. What happens when a student’s career in higher education is so highly managed that they achieve success without having attained the skills necessary to be successful outside of the wall of the classroom? What is a degree if students lack a sense of personal responsibility for their education? What is a diploma if students cannot adapt to the constantly changing demands of what Zygmunt Bauman has termed ‘liquid modernity’?

Obviously, neither George Siemens nor I believe that learning analytics is actually passé. With George, I share a conviction that learning analytics — like most educational technologies — are neither bad nor good in themselves. Analytics have a huge potential to function as a mirror, helping universities brave enough to look to identify barriers to student success that they, themselves, are erecting. As we have seen in the case of Georgia State University, they can be used to identify students in need and improve our ability to reach out and intervene. While at Emory University, I demonstrated how analytics can help teachers to be agile in how they understand and improve instructional design. Alyssa Wise and others have championed a view that, rather than restricting teaching practices, analytics can be used flexibly as an embedded part of a reflective approach to teaching and learning.

The question, then, is this: instead of the dystopian future imagined by Siemens, how do we bring about a utopian future in which analytics and humane education walk hand in hand?

Public institutions and educational technology companies are not enemies. To the contrary, public-private partnerships are essential to the processes of innovation, scale, and diffusion that are so important as American higher education reshapes itself in the face globalization and the information technology revolution. In these relationships, however, it is vital that institutions of higher education take the lead in shaping conversations and establishing expectations. It is incumbent upon teachers and administrators to think carefully about the long-term consequences of their investments, and the extent to which those investments will allow them to be agile in the face of changing national workforce demands, research discoveries about teaching and learning, and ongoing reflection about what it means for a student to be successful. In the absence of this kind of strong leadership from institutions, it is all too easy for technology to drive understandings of educational needs rather than the other way around.

How can institutions invest in technology, while at the same time avoiding lock-in and preserving the agility they need to promote the success of their students under conditions of constant change and uncertainty? How can colleges and universities form strong and honest partnerships with technology companies that address their actual needs rather than be forced to think of their needs in terms of the constraints imposed upon them by existing products and IT investments? When it comes to educational data and analytics products in particular, the only way to avoid lock-in is to be unflinching in the demand for three things:

Transparency. Predictive analytics are not magic. There is no ‘secret sauce.’ The highest quality results are achieved when vendors apply well-established data mining techniques to clean and well-defined institutional data. There is no advantage to using a proprietary technique or predictive model. If anything, a vendor’s refusal to share the model they have produced using your data should be reason for suspicion. If the model can’t be shared, it can’t be independently validated. If you don’t know the model along with its underlying assumptions, you also can’t meaningfully interpret its results. When it comes to educational data, trust needs to be earned and constantly renewed, not expected as a matter of course.

Flexibility. It is important that analytics tools and products complement existing workflows, and allow institutions the flexibility to change their practices when doing so is in the best interests of their students. What this means is that these tools should not demand significant institutional change in order to work.   True, in some cases significant cultural and structural changes may be in the best interest of the institution, and those changes may also increase the usefulness of a particular technology. It’s not a bad thing when cultural and technological change take place in the service of a larger institutional vision. A prime example of this is where a college decides to newly adopt a proactive advisement strategy using some kind of Integrated Planning and Advisement Services system. But when the cart of educational technology begins to lead the horse of institutional culture, values and priorities begin to be misplaced.

What matters most are not the changes themselves, but rather what is driving them. As Kevin Kelley has argued, technology has a way of making demands on us. If we fail to be vigilant, it is easy, or even natural, that we succumb to the pressure to conform to the ‘desires’ of technology. Institutions must constantly reflect about who they are and what they are trying to accomplish. They must be prepared to transform themselves in response to the changing needs of students, and they must demand that their technologies be flexible enough to change along with them.

Interoperability. The demand for interoperability is a tricky one. On the one hand, an institution doesn’t want to be limited in its adoption decisions by pre-existing investments (the biggest ones being its student information and learning management systems). It also shouldn’t feel locked into a sub-optimal learning management system because a critical analytics tool doesn’t work with anything else. On the other hand, particularly in an immature educational technology space like analytics where startups abound, the prospect of supporting every version of every major system is a mammoth effort. Fortunately, international data standards like LTI and Caliper are making interoperability easier now than ever before. With interoperability comes the opportunity to ‘unbundle,’ which means that schools can select the right technologies for their students and easily swap them out as needs change and better products emerge. This is starting to happen now, but the pace of change is slow. It is only as institutions begin to apply pressure on vendors to adhere to common data standards and definitions that we will see an important acceleration in the pace of change.

With George Siemens, I agree that lock-in is bad for innovation, and it is bad for students. In demanding transparency, flexibility, and interoperability, institutions place significant and inconvenient constraints upon their analytics partners, to be sure. But it is only in the presence of these constraints that true innovation can take place. Innovation isn’t easy. Innovation doesn’t happen as a result of iterating on what has already been done. Innovation is not about products. Rather, innovation takes place as a result of grappling with difficult questions, and rallying disparate elements to solve real problems here and now. It is the task of institutions of higher education to understand the needs of their students and to frame conversations about the use of student data in ways that will serve those students best. And it is the responsibility of vendors to listen attentively and respond by building products that embody 21^st century pedagogical values.

—

Timothy D. Harfield, Ph.D., is a scholar, blogger, and project director with particular interests in educational technology, learning analytics, and student success. He has published and presented internationally on a large number of educational topics including analytics, ethics, instructional design, and cultural change. He is currently Senior Product Marketing Manager for Blackboard Analytics. He previously worked at Georgia State University in the Office of Enrollment Management and Student Success.