Combining Activity Data from Multiple Sources to Improve Student Success Predictions

In a study published today, researchers from Blackboard, University of Maryland Baltimore County, and VitalSource describe a joint project designed to test whether a predictive model of student achievement using IMS Caliper Analytics® (Caliper) events would benefit from combining data from multiple learning tools: Blackboard Learn and VitalSource. Their findings were incredibly promising:

1. Early activity in learning tools is a strong predictor of whether a student will pass a class
2. Patterns of activity differ significantly between courses
3. Learning activity data is a more powerful predictor of achievement than demographics and educational background
4. Combining data from multiple learning tools (like Blackboard Learn and VitalSource) improves the accuracy of predictions about student achievement
5. Students with high levels of activity in multiple learning tools can significantly increase their chances of successfully passing a class

Thanks to open learning analytics standards like IMS Caliper, vendors and institutions alike have the ability to more easily combine data from multiple sources in a way that can increase predictive accuracy and, hopefully, be used by institutions to positively impact course completion rates. Not only can combining data from multiple sources significantly increase the accuracy of predictive algorithms, but open standards can be leveraged to significantly reduce the cost of doing so. This joint research from Blackboard, VitalSource, and UMBC is a great proof of concept for the value of combining data through partnership. As adoption of the IMS Caliper standard increases, it is our hope that our work here might serve as a jumping off point for collaborative efforts by others in the future

Increasing the predictive accuracy of student achievement models is not valuable in and of itself. The value that an institution is able to derive from predictive analytics is, instead, a function of three factors:

1. The predictive accuracy of models that are deployed,
2. The cost of generating those predictive models, and
3. The effectiveness of the student support system that makes use of predictive analytics

We know from examples like Concordia University Wisconsin that it is possible for institutions with a mature advising model to increase student retention by as much as 10% using descriptive analytics alone. From our own research, and from the industry in general, we also find that the addition of predictive analytics to an already effective advising program and increase retention rates by an additional 1 – 3%.

Predictive analytics is an area where the perfect can be an enemy of the good.  Given the right conditions, we are able to create predictive models using LMS data alone that are more than 90% accurate, but fetishizing predictive accuracy can easily lead an institution to become distracted from investing time and energy into developing the high impact practices that are necessary to make a difference in student outcomes.

Realistically, any model that reliably predicts student achievement in a way that performs significantly better than chance will have a major impact when its results are used by the right people and institutional processes. Models maybe improved by considering data from sources outside of student information and learning management systems, but until recently the cost of integrating data from additional sources in order to achieve unknown, limited, or diminishing returns has been difficult for many institutions to justify

The widespread adoption of open learning analytics standards like IMS Caliper has the potential to change the accuracy-cost-practice equation.  By significantly decreasing the time and effort required to combine data from multiple sources into a single model, open standards decrease the cost of increased predictive accuracy and increase its relative value, particularly when used by mature advising and student success programs.

Download the full report here and read the press release here. For more information about how Blackboard can help you to use data and analytics to solve your core educational challenges, visit blackboard.com/analytics.

The post Combining Activity Data from Multiple Sources to Improve Student Success Predictions appeared first on Blackboard Blog.

Combining Activity Data from Multiple Sources to Improve Student Success Predictions original post at Blackboard Blog