How can an institution provide meaningful learning data to academic teams at scale?

Andy Ramsden, Director of Technology Enhanced Teaching & Learning, and Learning Analytics, The University of Law

A challenge many educational institutions face is captured in the conversation I had with an Executive Board Member at the University of Law. They asked, "how can we use learning analytics to enhance the design and delivery of our programmes and drive transformational change to improve the learner experience?"

My reply focused on the need to provide an institutional service, an evidence-based approach that enables the Programme Team and Learning Designers to make better decisions on effective module design within Blackboard and other learning tools to improve the student learning experience and outcomes. 

From an institutional perspective, this raises the question, we can provide many incredible insights for people to improve the design and delivery, but what are the requirements for a transformational, sustainable service for all our programs?

This article outlines the University of Law's first tentative steps as we define the service, identify if the service has a positive impact, and think of the likely resource requirements to support this going forwards.

“From my perspective, I believe using clear scheduling is a transformational, scalable service. Therefore, I'd strongly encourage you at your institutions to take your first steps into a very insightful world.”

The service we have tested over the last three months is called "Module Analytics Review Service (MARS). MARS has two offerings; (i) the Annual Module (Programme) Review Report and (ii) The 4-week teaching insight report. The former is a detailed analysis that provides academic teams with insights and ideas for their redesigns.

The Annual Module (Programme) Review is divided into the following phases;

Phase 1: Defining with the academic team

Phase 2: Data Collection

Phase 3: Analysis

Phase 4: Briefing and planning

Phase 1 is crucial for a successful outcome. This is the alignment phase, where we capture what questions they want to be answered, orientate the analytics team to the learning context and student characteristics, and outline their current enhancement plans.

Phase 2 uses a range of data collection methods. This includes running reports against our learning platforms (Blackboard) through the Blackboard Data Service. This is combined with a team member completing "module learning walks" as they review the student experience against a quality rubric aligned with our design principles.

Our planned improvements include the following:

Including data from broader learning tools.

Improving the data queries.

Providing clearer visualizations

identifying how best to integrate the student's voice.

So, what feedback have we had? Is this service impactful? I'd suggest yes. The incredible depth of discussion around what it's showing and how it shifts the debate from anecdotal to evidenced-based, stimulating innovative ideas and simple plans.

Peter Goodchild, National Programme& Student Affairs Director SQE Conversion, "The opportunity MARS presents is great. To discuss with an education technology professional what questions to ask and then get those questions answered with tangible evidence is a fantastic resource. I'd like to use this service for all the programmes for which I am responsible."

Richard Hagget, National Programme Director Online Programmes, "The data helps us to reflect upon and plan the release of information, and consider the ways of presenting information that appear most popular, which helps drive refresh and renew."

Finally, can this model scale? What are the likely resource requirements? Our initial pilots have identified the requirement of the learning analytics team is approximately a total of 40 hours, shared by two team members to deliver all phases.

From my perspective, I believe using clear scheduling is a transformational, scalable service. Therefore, I'd strongly encourage you at your institutions to take your first steps into a very insightful world.

review