An in-depth look at a Web-based learning and assessment program like no other

INTERVIEW | by Victor Rivero

Imagine a Web-based learning assistant that provides students with individualized and highly-targeted instruction in a variety of subjects. By assessing a student’s current knowledge in a subject, it would be able to deliver only the topics that the student is ready to learn. It’s real, it’s very smart, and it’s called ALEKS — an acronym for Assessment and LEarning in Knowledge Spaces – named by the developers for its basis in Knowledge Space Theory – a scientific theory that was originated in the early 1980s by cognitive scientists and applied mathematicians at New York University and The Free University of Brussels. It’s a web-based learning and assessment program that uses artificial intelligence (AI) and adaptive questioning to provide individualized instruction to each student. It begins by posing 25-30 open-ended, algorithmically generated problems to assess precisely which topics the student knows, doesn’t know, and which topics the student is ready to learn next. At the conclusion of the individualized initial assessment, ALEKS provides students with a user-friendly, color-coded pie chart that shows which topics the students are ready to learn right now. ALEKS then enables adaptive learning in which the student selects among the topics he is ready to learn. Whenever a student learns a new topic, ALEKS updates the student’s knowledge state and the set of topics he is ready to learn. ALEKS periodically re-assesses the student to verify and solidify content mastery. Research shows this type of “retrieval practice” significantly enhances long-term retention. In essence, “ALEKS is a cycle of learning and assessment,” says Wil Lampros (pictured left), CEO of ALEKS Corporation, the team of more than 150 content writers, editorial and production staff, software engineers, computer programmers, mathematicians, mathematical cognitive scientists, R&D professionals, business professionals and support staff that make ALEKS work. Here, Wil provides a view of a company with tremendous depth on the leading edge of learning technologies.

Victor: What is the history and some background of your company?

Wil: ALEKS began in 1992 as a project at the University of California to use Knowledge Space Theory to build an intelligent learning and assessment system with major funding from the National Science Foundation. Knowledge Space Theory was developed by our Chairman and other scientists and mathematicians to address a critical need: namely, the need to move from assigning each student a simple one-number score (or at best, a few “sub-scores”), to enabling topic level analysis of individual student knowledge.

The ALEKS project was extremely successful and produced a fully functional web-based learning and assessment technology for math in 1996. Shortly thereafter, the University of California licensed the ALEKS technology to ALEKS Corporation, which was founded by the cognitive scientists and software engineers who created it. After I joined the company in 1998, ALEKS Corporation opened for regular business with four employees in 1999. Now we are a company of over 150 employees, and we help over one million students succeed each year in math, science, business and behavioral science.

What we’re most excited about right now is releasing ALEKS course products for Apple’s iPad and other popular tablets.

Victor: How is ALEKS unique from other Web-based learning systems?

Wil: ALEKS is unique because its sophisticated technology uses pinpoint accuracy based on scientific theory and massive data to ensure the student is always working on topics she is ready to learn or needs to review, and it uses algorithmically generated open-response questions, and user-friendly input tools – avoiding multiple-choice. In addition, it is a cycle of learning and assessment that maximizes learning and retention of course material.

Using Knowledge Space Theory and free-response questions, ALEKS is able to provide precise, detailed and comprehensive assessment and highly personalized learning. ALEKS accurately determines, at the atomic level, precisely which topics a student has mastered, has not mastered, and which topics the student is ready to learn next. This information is used to enable students to learn more efficiently, and saves them from boredom and frustration. Students who use ALEKS engage with the subject matter because they are always working on what they are ready to learn.

Each time a student masters an additional topic, ALEKS immediately updates the student’s knowledge state and provides the student with a new list of topics she is ready to learn. The result is a continual re-optimization of the student’s learning path.

Victor: What does “ready to learn” mean when it comes to ALEKS or any other online learning system?

Wil: In ALEKS, the student is always permitted to choose which topic to work on, but only from among the topics she is ready to learn. “Ready to learn” means exactly what it sounds like: “Will the student be able to master that topic if she works on it right now?” We keep statistics on ALEKS’ success in determining whether a student is truly ready to learn a topic in order to evaluate and improve our content and artificial intelligence engine. When ALEKS provides a specific topic to a student as “ready to learn,” on average the student successfully masters that topic well over 90 percent of the time. This is what we mean by “ready to learn,” and this is what makes the program so powerful and efficient. With ALEKS, students always work on what they are ready to learn right now.

Victor: Tell me about who your customers are in the K-12 market and what products you have to serve them.

Wil: Our customers comprise a wide variety of public, private, and parochial schools; urban, suburban and rural; large and small; using ALEKS as “core/basal” and as a supplement. We have customers reporting hundreds, if not thousands, of successful implementation approaches. We also have many homeschool students using ALEKS.

Our principal K-12 course products currently cover elementary school math through PreCalculus, as well as courses that support intervention, state-specific exit exams and end-of-course exams, and AP Chemistry and Statistics. Our courses cover the entire math curriculum for each grade and are correlated to the Common Core Standards, as well as individual state standards.

We also have robust intervention products for middle school and high school that can be used with students who are substantially behind grade level. For example, for the very large number of students who are significantly behind when they reach Algebra 1 in high school, we offer several different approaches, including products highly focused on the core pre-requisites for Algebra 1, a course that comprehensively covers all the Foundations of High School Math, and a two-course Algebra 1 approach with broad, comprehensive coverage of the Common Core Standards.

Teachers are essential to effective student learning. ALEKS does not replace teachers or parents. What we do is provide the best set of tools we can so that each student has completely individualized learning, and the teacher has vast amounts of detailed, formative, instructionally-actionable information.

Victor: How is what ALEKS does in higher education different than in K-12?

Wil: The technology, essential AI design, and much of the content is the same for similar subject areas in our higher education and K-12 course products. However, our product set and the AI for each course are finely tuned to the standard courses in each respective market and the student populations being served.

Victor: How does assessment work in ALEKS? Is it different than in other self-paced online learning programs?

Wil: Assessments in ALEKS are based on Knowledge Space Theory. There are no question banks in ALEKS. The problems used for assessments are drawn directly from the curriculum for the course. Also, remember that ALEKS avoids unreliable multiple-choice questions. Questions in ALEKS are open-response, and algorithmically generated, as unique instances of each of the topics in the course.

Each student begins her ALEKS experience with a brief initial assessment. After the student answers each question, ALEKS updates its artificially intelligent representation of the student, and bases the next question on the student’s answers to all the previous questions. This enables ALEKS to determine which of the vast number of knowledge states the student belongs in, and what she is ready to learn.

Victor: Are all of your offerings in math? What other content areas does ALEKS cover in K-12 and higher education?

Wil: Very roughly half the students using ALEKS are in K-12, and half are in higher education.

In K-12, our course products cover grades 3-12 math. We also have products to help K-12 students in AP Chemistry and AP Statistics, as well as a small number of business courses for high school students.

In higher education, we have all the popular college math classes below Calculus. Over the last few years, we began to make Chemistry courses available. We now have terrific courses for General Chemistry, adoptions of which are growing very rapidly at major universities. We also have excellent offerings to prepare students for General Chemistry. In addition to our math and chemistry offerings, we also have three “flavors” of statistics, as well as Financial Accounting and Business Math.

Many colleges use ALEKS as a key component of one or more of the courses in their math programs. These customers often have dramatically improved student learning outcomes, increasing the pass rates, for example from roughly 50 percent to over 75 percent.

Victor: Tell me how Artificial Intelligence works in ALEKS? How is it different from adaptive learning?

Wil: The science behind ALEKS is a significant departure from standard psychometric theory, which originated in the late nineteenth century and still forms the basis of most testing and adaptive learning systems. As I mentioned, ALEKS uses Knowledge Space Theory, which functions as a form of artificial intelligence, and also enables ALEKS to adapt to each new student response in both learning and assessment. We have materials on our web site explaining the science behind ALEKS, including a great new video called “How ALEKS Works”: http://www.aleks.com/video/how_aleks_works

One way of understanding ALEKS AI is as a scientifically advanced “cousin” of a “knowledge graph.” The idea is that there may be a map, or “graph,” of how concepts are related to each other in a particular context. You hope that you can create a graph that is complete enough to enable you to draw inferences from what you think the student already knows, to other things she might also know or be ready to learn.

In ALEKS, the basic element of the graph is not an individual concept or topic, but a “knowledge state”, that is, the combination of topics that might constitute an actual state of student knowledge in a subject. We use “big data” to build knowledge spaces, which map the relations among the knowledge states, or feasible states of student knowledge. These knowledge spaces enable ALEKS to accurately determine which individual topics the student has already mastered, and which ones she is ready to learn. 

Victor: How do ALEKS’s products prepare students to meet the learning of Common Core? What role does technology play in that support?

Wil: At the student level, ALEKS products cover the entire subject matter of the K-12 course. In other words, we have always attempted to satisfy as high a percentage as possible of all 50 states’ curriculum standards. In our most recent courses, we try to satisfy all of the Common Core Standards. Algebra 1, for example, released during the summer of 2012, satisfies 100 percent of the Common Core.

At the instructor level, we have extremely robust reporting and student group formation tools that measure, down to the atomic level, the precise degree of each student’s mastery of each of the many topics covering each standard. This provides extensive, extremely detailed, formative, instructionally-actionable information at the individual student and topic levels. These tools permit the instructor to determine precisely which students need help with which topics and standards, enabling the teacher to target the most effective and efficient strategies for Common Core mastery.

Victor: What kind of response do you get from educators about the effectiveness of ALEKS in the classroom?

Wil: Many educators report significantly improved learning outcomes on standardized test scores. They also report an increase in the percentage of students who score at proficient or advanced level, and in the percentage of students who improve their proficiency category from one year to the next. For interested readers, we have special pages for success stories and implementation strategies on our web site (www.aleks.com) in both K-12 and Higher Education Math.

To pick a common situation, the average middle school math teacher may have a group of students who are at drastically different grade levels; some may be far below grade level, while others are far above it. However, even students at the same grade level all have widely varying gaps in their prerequisite knowledge. We call this the “Swiss Cheese Effect,” because all students arrive in a class with a different set of holes (or gaps) in their background knowledge.

Because of the program’s ability to determine which topics a student has mastered and precisely which topics the student is ready to learn, ALEKS has the unique ability to find and fill each individual student’s gaps in prerequisite knowledge, enabling the student to progress on a much more solid foundation. Our customers frequently tell us their students tend to do better in their next math class as well.

Educators who use ALEKS in the classroom or lab (such as in a “flipped” or blended learning model) observe that their students are much more likely to be engaged with the material. They report that ADD/ADHD students are often able to concentrate on ALEKS for extended periods of time. Additionally, ALEKS works well to provide an Individualized Education Plan (or “IEP”) for every student, and has seen strong results in special education.

Victor: What kind of tools and resources do you provide to teachers? Is there any professional development needed for educators who want to use ALEKS with their students? If so, did you use technology to deliver that professional development?

Wil: ALEKS is intuitive and easy for instructors to get started with immediately.

We offer flexible professional development options for educators both online and in-person. Our dedicated Implementation & Professional Development team offers complimentary daily online trainings where customers can learn more about best practices, as well as pre-recorded training sessions on a variety of ALEKS features.

We also encourage our customers to share ideas and best practices with each other. We have a robust implementation strategies database where ALEKS customers have shared their tips on getting started and implementing the program in a variety of classroom settings. In addition, we have an active ALEKS Community where customers can ask and answer questions with their colleagues.

Victor: Do homeschoolers use ALEKS? Do you offer them special support or resources?

Wil: Homeschoolers have had great success with ALEKS. All of our independent user customers, many of whom are homeschoolers, have access to the full ALEKS course library. So ALEKS can be used by families with children in grades 3-12, as well as by individual students of all ages who wish to learn a new subject, or review or prepare for tests or college. In fact, a great benefit to homeschoolers with older students is that students can receive college credit for American Council on Education (often called “ACE”) credit-recommended ALEKS courses. ALEKS QuickTables, a math fact mastery program available in all markets, is particularly popular with younger homeschool students.

A Master Account is provided to parents to allow them to monitor their student’s progress in ALEKS. The Master Account contains a variety of reports that detail what their student knows, doesn’t know, and is ready to learn right now. Additionally, an attendance report helps track precisely what each student works on and when. Homeschoolers can also use the ALEKS content, and algorithmically generated instances, to provide highly focused quizzes. We find that homeschool customers are up and running on ALEKS very quickly, and if they need additional support, they can access pre-recorded trainings on the ALEKS website.

Victor: What have been some of ALEKS’s biggest successes? Greatest challenges?

Wil: There appear to be two key factors in some of our most successful customer implementations. Our customers’ greatest successes have occurred when students have used ALEKS for a minimum of three, and preferably four or five hours per week, with meaningful physical or virtual instructor participation in the students’ ALEKS learning experience. We have also seen great success where the teacher has taken particular advantage of the robust tools in the ALEKS learning management system. These tools are truly fabulous for educators who want to target their teaching, create homework and quizzes on high-need areas and find and address individual student and group learning issues.

Our biggest challenges fall into two opposite types of institutional or instructor attitudes: one is educators who assume students will do everything necessary to learn math with ALEKS without encouragement, incentive or requirement, and the other is educators who want to micro-manage every student action. We could call these two views: the “get in shape by buying a gym membership” view and the “instructor knows best what each individual student should be working on” view.

We have many options to help address both of these challenges. Most often, work in ALEKS is made a percentage of a student’s grade or a source of extra credit. We also have technology that facilitates ALEKS being integrated with more traditional teaching styles, such as modules and textbook chapters. In addition, we have tools to permit homework, quizzing, and testing outside the framework of the ALEKS AI, but with all of the advantages of fully automated grading of non-multiple choice questions. For the most active and involved instructors, there are a vast number of precise, formative reports provided in ALEKS to facilitate the most productive learning management.

Victor: What would you say to a school district that might be on the fence about whether or not to use ALEKS?

Wil: ALEKS offers complimentary, no-commitment pilots to serious potential customers. There are no set-up costs or licensing fees of any kind, so there is no cost to schools or districts that sign up for a free trial. We only ask that institutions give ALEKS a serious and committed trial. Once they experience the dramatically improved learning outcomes that ALEKS provides, they become committed ALEKS customers.

Victor: On to a few broad questions: What are your thoughts on the state of education today?

Wil: Education faces a lot of challenges in the United States today. Schools are overwhelmed by a highly diverse, multilingual population with some students who are years ahead and many who are years behind. Class sizes are often too large for the traditional one-teacher model, which means many students are not being engaged. Often, the best students are slowed down while the weaker students struggle.

Even with these challenges, there are many outstanding, committed, hardworking educators who make a significant, positive impact. In addition, students learn a lot from family members, afterschool programs, classmates, mentors and their community. We know people are resilient; many students overcome a multitude of challenges and go on to eventual success.

In higher education, the United States has become a sort of “university to the world,” where many of the most highly trained people in the country and from around the globe come to the U.S. to teach, study, and do research at our universities. Yet, sadly, the average passing rate in most developmental math courses hovers around 50 percent.

In fact, the data show that in both K-12 and in higher education, many students are left behind. We see this every day in the dropout rates at high schools and colleges. Many students arrive at school who are years behind in math and other core subjects, or who haven’t mastered very basic skills. All too often, they give up.

We can do a lot better.

Victor: What is your outlook on the future of education?

Wil: We are seeing some of the same trends in education today that people have been discussing with regard to healthcare. Reliable, high-quality education is much too expensive for most people, and many of our educational practices in the U.S. do not reliably succeed. Costs are increasing, and class sizes are growing – handicapping even the most talented instructors from effectively providing differentiated instruction. Learning outcomes are becoming more uncertain.

However, there is light at the end of the tunnel. We are beginning to see signs in education of the kind of transformation that has so dramatically affected most other areas of life. That is, as more effective educational technologies become available, it may become possible to affordably provide a high-quality education to the vast majority of students. Educational technology can be an extremely effective aid to educators, allowing them to achieve more successful learning outcomes with a higher percentage of the students. At its best, computer software will facilitate the kind of differentiated, one-on-one learning experiences that previously required very small classes or expensive, individual tutoring. In other words, one can imagine a very effective collaboration between experienced educators, and reliable, efficient, and inexpensive technology.

This is the kind of instructor collaboration that ALEKS provides, and the kind of cultural transformation that ALEKS helps to bring about.

Victor: Any final thoughts on education and technology?

Wil: In the design of the first generation of educational technology, the technology was thought of primarily as a device to save educator time via machine grading; little attention was paid to student learning outcomes. These systems tended to be based on multiple-choice questioning, which is far easier to engineer, program and maintain. So-called homework management products are examples of this type of technology, but there are many other examples.

But a new generation of technology is emerging. We are now in a position to provide straightforward, non-multiple-choice answer input and machine grading. With the advent of systems that are able to intelligently adapt to individual student needs, we can now provide a student-centric experience in which each student learns at his or her own pace and is far more likely to proceed to mastery. Educators and government entities are beginning to demand acceptable learning outcomes for all students. If our educational institutions begin to routinely require technology that, by individualizing learning, helps each unique student maximize his or her potential, we can see the potential for a far brighter outcome in education.

We think ALEKS is part of this solution.

—

Victor Rivero tells the story of 21st-century education transformation. Get your story told through case studies, white papers and other materials you can share at trade shows and on your website. Write to: victor@edtechdigest.blog