Blog - Time well spent: How and why you should measure learning

Blog by Katharina Ehrhart, Policy and Research Manager, FSSC, and Allegra Lumb, Financial Services Consultant, EY

• In the last 12 months, employee learning hours in financial services have increased more than 30%, but more consistent measurement of learning is needed to provide the full picture.
• Many firms in the sector measure ‘learning hours’ but variability in measurement can complicate efforts to compare data across different organisations. Recently there has been a shift to measuring the population of active learners instead.
• The Financial Services Skills Commission is calling on firms to measure employee learning by refining quantitative metrics (such as hours spent) and balancing them with qualitative metrics (such as engagement levels and learning outcomes)

“Skills are right at the top of the investment you need to drive productivity. It’s hard to imagine anything more important than investing in skills for individuals and indeed for our country.” Andy Haldane, CEO, The RSA

Firms in the financial services sector are facing a double challenge; the need to not just close existing skills gaps, but also prepare colleagues for the skills they will need in the future. Learning is critical to developing a positive workplace; it enables employees to develop their skills set to meet organisational needs, while advancing their career development, leading to higher retention.

To drive performance and respond to the skills challenges in the sector, employers need to put in place ambitious upskilling and reskilling programmes. It is equally important that they can evidence the benefits of investment in these programmes. And yet, without a standard measurement of learning, it is challenging to accurately compare learning progress across the industry.

Our latest data shows that learning has increased to four days per year, up from three days last year. But, is this an accurate view of our learning? The industry has increasingly shifted from tracking learning hours towards the concept of ‘active learners’ – and combining quantitative and qualitative ways to understand learning.

We have analysed the different methods of measuring learning in the industry, the challenges associated with these methods, and recommendations for future practice. We will work with the sector to consistently measure learning; to enable benchmarking and the tracking of its impact.

Measuring progress using ‘learning hours’

Traditionally, learning has been measured by tracking the hours each employee spends on developing new skills. This measurement is typically broken down into mandatory and non-mandatory hours. We have been collecting learning hours data since 2020, allowing us to compare trends over time. Our data is detailed enough to look at mandatory versus non-mandatory learning, and we even have a breakdown by type of learner.

This traditional method of measuring learning hours has several advantages. Firstly, it provides a complete set of data on employee learning in the industry, given that most firms measure learning hours. Secondly, this measure also features in official government statistics, allowing for easy comparison of engagement with education and training. Many firms aim to establish a habit of learning with initiatives such as ‘learn for two hours a week’ or host a learning day once a month for employees to increase time dedicated to learning¹.

However, this method has significant drawbacks. It does not provide insights into the distribution of learning among employees, or across the year, making it difficult to understand who is really engaging with the material, as a small number of active individuals, or peaks of activity, can mask pockets of inactivity.

Furthermore, a decline in learning hours does not necessarily indicate a reduction in learning. For example, employees might be engaging with more efficient learning methods, or self-directed learning that is not captured in the tracked hours. Focusing solely on the quantity of hours can overlook the quality and impact of the learning experience.

The move towards identifying active learners

In response to the limitations of traditional methods, the industry has begun to shift towards the concept of ‘active learners,’ also called engaged learners. This was first widely referenced in the 2020 LinkedIn Learning report and now measurable via Microsoft’s Viva Learning suite. This approach identifies and measures learning engagement more meaningfully, moving beyond merely counting hours. Having collated this data for the first time in 2023, we are now improving data collection and creating greater alignment of measurement across firms.

There are various methods for identifying active learners. Examples include tracking employee completion of work on online platforms, monitoring login activity or employees who reach a minimum number of hours on their learning platform per quarter. Employers often begin by examining professional qualifications. However, learning data is about capturing all forms of learning, not just time spent on formal courses that lead to a qualification. The goal is to identify how many colleagues are genuinely engaged with learning on a consistent basis, rather than how many hours the workforce may complete on average. This would be an industry shift from an output-focussed approach where firms measure hours, into a process-focused approach.

The active learners concept offers several advantages. It allows organisations to clearly see how many people their learning material has reached, providing better insights into the effectiveness of organisational upskilling programmes. This approach can help reward highly engaged learners and motivate the broader team to engage in further upskilling opportunities.

However, there remains wide variability in how active learners are defined and measured, with as many definitions as there are organisations using this method. Firms also measure learning using different time frames, with some firms ranging from three to six months, some occurring annually based on the calendar year, and others are measured on a continuous rolling basis.

This lack of standardisation leads to inconsistent data and difficulties in comparing results across the industry, highlighting the need for a more standardised approach if active learners were to be used as a common frame of reference.

Challenges posed by different methods

One of the major challenges in measuring learning hours and, more so, active learners, is the lack of a unified definition and standardised measurement. This variability can complicate efforts to compare data across different organisations.

Measurement methods can also influence learner behaviour. For example, if employee focus is solely on completing a certain number of hours, learners may engage in activities that meet the hour requirement but offer limited upskill. On the contrary, if active learner metrics are emphasised, there is a risk of incentivising negative behaviour, where individuals engage superficially solely to meet the measurement criteria.

What should the sector do next?

While traditional methods of measuring learning hours provide valuable data, they have significant limitations. The shift towards the concept of active learners offers potential benefits but also presents challenges that need to be addressed. In response, it is crucial that firms balance quantitative metrics (such as hours spent) with qualitative metrics (such as engagement levels and learning outcomes).

Organisations need to understand how well their data on learning is currently serving them and learn from best practice to drive progress. We are working with members to develop a standardised definition of active learners and other learning metrics within the industry.

To reach a consensus on the definition and measurement of learning to compare trends across the sector, we would be interested to hear how others have implemented learning measurement in their organisation. If you’d like to share your experience, contact us at info@financialservicesskills.org.

Footnote ¹: https://www.gov.uk/government/statistics/employer-skills-survey-2022