Learner success approach projects – phase 1

Learner success approach projects – phase 1

Last updated 30 October 2021
Last updated 30 October 2021

We have partnered with four tertiary education organisations (TEOs) to run learner success pilots.

During 2019-21 we have developed and trialed a Learner Success Framework (the Framework); piloting the approach with four TEOs. While we have refined the framework along the way, we are confident that it is fundamentally the right approach addressing disparities in outcomes of learners in tertiary education. 

Our four TEO partners

The four partner TEOs were engaged to undertake pilot projects that aimed to:

  • determine the suitability and utility of the Framework across different sectors; and
  • provide recommendations and insights into ways of improving and implementing the Framework.

Impacts on learner outcomes and equity will only be measurable once the processes and changes required become more mature and entrenched within the partner TEOs.

Waikato Institute of Technology (Wintec)

Project commenced 30 June 2019 and finished in December 2019.

Total funding - $430,000 plus an additional $100,000 to work with the US organisation Achieving the Dream (ATD) to support TEO coaching activities.

Wintec was the first and longest running of the partner projects. They tested the first four phases of the Framework - from diagnostic to strategy development.  By using human-centered design methodologies, Wintec developed a deep and comprehensive student journey mapping process.

This approach has given Wintec key insights into their learners’ needs and formed the basis of a new operating model for the organisation.  Wintec is in the process of developing culturally situated momentum strategies that leverage data, leadership, people, and partnerships to drive continuous improvement.

Based on their learner success focused approach, preliminary feedback from Wintec shows improvements in retention and course completions for Māori and Pacific learners during the COVID-19 level four lockdown in 2020. The gains have halted and reversed historical declines for these measures.

Te Wānanga o Raukawa (TWoR)

Project commenced 26 July 2019 and finished in July 2020.

Total funding - $375,000

Changing learner demographics and demand necessitated TWoR to undertake a radical shift in the way they design and deliver their education offerings, by moving most of their content into a blended delivery model. We partnered with TWoR to support them in undertaking this business transformation project, which entails them using a TWoR derived change management model which closely aligns with the Framework. This provided us with an unprecedented opportunity to test the framework in a mātauranga Māori context.

The high alignment between their model and the Framework meant that TWoR had already made considerable progress towards developing a holistic understanding of their learners and developing a learner-centred operating model. The Framework has assisted in aligning the learner success practices with their own guiding kaupapa. TWoR is driving organisational and learner change by leveraging technology and continuous reflection and improvement strategies.

TWoR is now partnering with local and global IT industry leaders to look at further opportunities to refine their own operating model and also deliver IT programmes of learning.

University of Waikato (UoW)

Project commenced 30 June 2019 and finished in August 2020.

Total funding - $297,500

UoW’s project focus was to create a robust system of learner insights by building their data analytical capability. This approach enabled them to identify teaching and learning gaps, trends, and patterns to build responsive information technology systems that allow pro-active approaches to strengthen student success. While working through the Framework stages such as undertaking a learner journey mapping project, it was realised that a broader whole of organisation change was required.

This meant the approach evolving from “just an IT project” to more of a holistic change project. While the deep qualitative evidence required for a comprehensive learner needs analysis to determine profile characteristics was lacking, extensive work had been done to develop an artificial intelligence driven data and insights platform to support future student-centered business planning.

Manukau Institute of Technology (MIT)

Project commenced 31 January 2020 and finished in May 2021.

Total funding - $331,200

We partnered with MIT to test the first four phases of the Framework, from diagnostic to strategy development. The project builds on MIT’s successful Hōkūle’a Project which increased the retention rates of Māori and Pacific learners in levels 1 -3 provision, through an early assessment of needs and provision of holistic support (academic, pastoral and cultural). MIT researched specific approaches and interventions used at Georgia State University, such as applying the process of ‘meta-majors’ to reduce confusion in learner subject choice; and developing technology options to support early alerts.

Note: Meta-majors group individual majors under a larger academic umbrella. These programs provide learners with a clear pathway and help them make connections between their studies and different career tracks. Learners also have set schedules depending on their meta-major. A learner interested in health care, for example, may enrol in a meta-major that includes prescribed health-related courses in science, communications, and statistics, opening up career opportunities ranging from nursing to pharmacy.

The intermittent Auckland region lockdowns resulting from COVID-19, as well as internal changes, have caused significant disruptions and delays in MIT’s ability to effectively progress their learner success initiative in 2020. Despite these factors, progress has been made in setting up quantitative and qualitative data collection processes to inform the learner journey mapping and learner needs analyses.

During the COVID-19 level four lockdown period, MIT developed an online version of their learner success approach. The recent transferring of the learner success project to the Deputy Chief Executive Academic to drive the learner success approach is a further strong indication of organisational commitment.

Evaluation of projects

From September 2019 to early 2021, a developmental evaluation was undertaken (in two phases) alongside the four LSF pilots in different contexts. This evaluation was led by the TEC strategy and evaluation team, with support from the Ōritetanga team and TEO partner organisations.

The aim of this evaluation was to provide the TEC with:

  • an understanding of how the learner success framework (LSF) and tools could be implemented, based on its unique iterative development and use,
  • the key findings of the piloting of the learner success framework in different educational contexts; and
  • the key learnings at TEO and system level to consider in scaling LSF approaches.

The current pilot projects were aimed at determining the suitability/utility of the adapted framework to our local context. Impacts on learner outcomes and equity will only be fully measurable once the LS processes and changes required become more mature and entrenched.

Overarching findings and themes

The suitability and utility of the Framework was endorsed

The preliminary overall view based off the evaluative findings is that the Framework has been successfully tested and implemented within the tertiary education context in Aotearoa New Zealand. This was clearly showcased in the enthusiasm with which TEOs engaged with the process and worked through the stages (in whichever permutation they used). The engagements with TEOs were seen as beneficial and useful. They provided a structure for the TEOs to use, adapt, and embed new processes to become more learner focused. These concrete steps along the learner success pathway clearly showcase the perceived utility and applicability of the Framework within the context of Aotearoa New Zealand.

Tertiary education organisations took ownership of learner success approaches

The open and collaborative approach used throughout allowed for all four pilot TEOs to implement the Framework in their own idiosyncratic ways. This allowed TEOs to:

  • adapt the Framework to their own unique contexts; and
  • internalise the approach within which they situate future work on understanding learner success and improving learner outcomes.

Tertiary education organisations shifted their mindset towards different ways of doing business

A significant development already present was the initiation of organisation wide cultural shifts in becoming more learner-centric and have organisational processes tailored around learner needs, not business processes. Most TEOs required a mind shift to a more learner centric way of doing operating on top of developing data systems for creating a holistic understanding of learners and needs. Data driven insights cannot be used effectively if operational contexts do not allow for it to inform strategies, policies, or processes.

While the full effects of the initiatives will likely take some time to be seen, meaningful progress on that journey had already been achieved by all TEOs engaged. These include learner journey mapping, creation of learner data systems and insights systems, as well as business cases for reviewing and changing organisational processes.

Tertiary education organisations understood the large scale work required for large scale change

A common issue encountered was an underestimation of the scale of work required.  The learner success approach is a complex and lengthy process to work through. All pilot TEOs underestimated the intensity of the process, the resources required and the timing needed from end to end.

This was mostly due to underestimating the holistic whole-of-organisation nature of the process. The scope of the organisational change often required is not clear at the outset of the project and only becomes clear once a TEO has completed the diagnostic and problem definition stages of the Framework. Extra guidance or upfront information on the resourcing requirements for effectively engaging in the learner success process would be beneficial.

Flexibility is required or needs to be better emphasised in a future iteration of the Framework

One pilot TEO, however, did report identifying more dimensions than the four used in the diagnostic tool of the Framework. While the extra dimensions did not conflict with those in the Framework, this may reflect differing emphases or organisational processes unique to the TEO. This underscores the need to emphasise flexibility in the Framework when applied to different TEOs or contexts.

Learner success approaches made meaningful progress in challenging times

The extraordinary events of 2020 were felt by all and caused significant disruptions and obstacles to the pilot projects. Regardless of these challenges, all TEOs engaged and continued building their learner success initiatives. Meaningful progress is already evident in creating and implementing the cultural and organisational shifts required for effective systems that will help identify, engage, and support not only high needs learners, but all learners.