From Big Data to Generative AI: Overcoming L&D's Data Capability Gap

Written By Anja Peebles-Brown

Generative AI, like ChatGPT and Claude.ai, has the potential to revolutionise Learning and Development (L&D) . With around 100 million weekly users, these tools are already being used for various day-to-day tasks, such as writing assistance, question-answering, and analysis. In L&D, Gen AI could personalise content, assess skills gaps in real-time, and provide advanced analytics and impact analysis - see the article by Chrysanthos Dellarocas on how it can help with increasing the rate at which employees learn. Impact analysis is of particular interest to me as a Data Scientist operating within L&D as I see many companies with a substantial data capability gap. However, to fully leverage the power of Gen AI, L&D teams need clean, robust data and systems capable of processing large quantities of data continuously. Unfortunately, many L&D teams struggle with this foundational requirement, despite having access to vast amounts of data. This blog post will explore some of the groundwork needed to maximise the potential of Gen AI in L&D.

Why does any of this matter?

Without the necessary groundwork, something as massive as Gen AI could pass L&D teams by, and this wouldn’t be the first time. Think back to 2010 to mid-2010s when the HR world was awash with articles about how Big Data would change everything for HR insights. Many articles were written about how Big Data will allow L&D teams to create more personalised and targeted training and predict future training needs. For the most part, the systems that many businesses had and still have in place for L&D reporting are not fit for purpose. And so, five to ten years on, this promised exciting impact on L&D has been very limited for many businesses - if not completely nonexistent. More frightening was the prediction made by Angrave et al 2016, that insufficient operational engagement could mean the exclusion of HR teams from the strategic and board-level influence.

What needs to be in place to make the most of Gen AI?

We are talking about things needed to implement gen AI effectively but these should also be considered more broadly when doing any kind of data analytics, particularly if you are looking to move away from only reporting on completion rates and learning hours.

Centralised Data Repository

You need a centralised data repository that integrates data from various sources, such as your Learning Management System (LMS), Human Resources Information System (HRIS), and other relevant platforms. Learning management systems (LMS) data might not always qualify as “big data”, it is complex, diverse and unfortunately for many L&D teams, really challenging to process. A centralised data repository becomes crucial in this context, as it helps consolidate data from various sources, including the LMS, into a single location. This centralisation simplifies data access and processing, enabling L&D teams to overcome the challenges associated with complex and diverse data sets. Without a central data repository, AI would struggle to find and make sense of scattered data, making it harder to get accurate insights.

If you don't have a centralised data repository in place, you may experience:

  • Difficulty in accessing and combining data from multiple sources

  • Inconsistencies and discrepancies in data formats and structures

  • Time-consuming manual efforts to consolidate and clean data

To address this, consider:

  • Implementing a data integration solution or data warehouse to collect and store data from different systems

  • Establishing data governance practices to ensure data quality, consistency, and accessibility

Some businesses will be more integrated than others and so you may already have in-house teams that have set-up or are in the process of setting this up. This is beyond the scope of what L&D teams can do but HRIT and/or IT teams can help if you are thinking about gen AI or even just streamlined data processing methods for your reporting.

Standardised Data Formats

You need standardised data formats across different systems and platforms to enable seamless data integration and analysis. Imagine if your course materials were in different languages or formats. It would be challenging to compare and analyse them effectively. Similarly, AI needs data in a consistent format to process and generate meaningful insights. Standardised data formats ensure that AI can understand and work with your data smoothly.

If you don't have standardised data formats in place, you may face:

  • Challenges in merging and comparing data from various sources

  • Inconsistent data fields and naming conventions

  • Difficulty in applying analytics and generating meaningful insights

To overcome this, consider:

  • Defining and enforcing data standards and naming conventions across systems

  • Collaborating with IT and data teams to ensure data compatibility and interoperability

Data Quality and Completeness

You need high-quality and complete data to ensure accurate and reliable analysis using generative AI. Just as incomplete or inaccurate course materials can hinder learning, poor quality or incomplete data can lead to misleading AI insights. AI relies on the data it is given, so ensuring your data is accurate and complete is crucial for generating reliable and trustworthy analysis. 

If you don't have data quality and completeness in place, you may encounter:

  • Missing or incomplete data points, leading to skewed insights

  • Inaccurate or inconsistent data, affecting the reliability of AI-generated analysis

  • Difficulty in trusting and making decisions based on the AI outputs

To address this, consider:

  • Implementing data validation and cleansing processes to identify and resolve data quality issues

  • Conducting regular data audits and assessments to ensure completeness and accuracy

L&D will need to work with the team managing the database in which your relevant data is stored. If the data structure and recording process are not well-defined or understood, it can lead to misinterpretation and incorrect conclusions.

Clearly Defined Metrics and KPIs

You need clearly defined metrics and Key Performance Indicators (KPIs) to guide the analysis and measure the impact of L&D initiatives. Clear learning objectives guide the design and delivery of effective training programs. Similarly, well-defined metrics and KPIs help AI focus on the right aspects of your data and provide insights that align with your L&D goals. Without clear metrics, AI analysis may not be relevant or actionable for your specific needs.

If you don't have clearly defined metrics and KPIs in place, you may face:

  • Ambiguity in what to measure and analyse

  • Difficulty in aligning AI-generated insights with business objectives

  • Challenges in demonstrating the value and ROI of L&D programs

To overcome this, consider:

  • Collaborating with stakeholders to identify and define relevant metrics and KPIs

  • Aligning metrics with organisational goals and L&D objectives

  • Communicating and gaining consensus on the chosen metrics and KPIs

AI and Analytics Expertise

You need AI and analytics expertise within your L&D team or access to such expertise to effectively leverage generative AI for impact analysis and advanced analytics. Just as you need instructional design expertise to create effective learning programs, you need AI and analytics expertise to make the most of AI technology. This expertise helps you ask the right questions, interpret AI-generated insights, and translate them into practical actions for improving your L&D initiatives.

If you don't have AI and analytics expertise in place, you may encounter:

  • Limited understanding of AI capabilities and limitations

  • Difficulty in formulating the right questions and interpreting AI-generated insights

  • Challenges in translating insights into actionable recommendations

To address this, consider:

  • Investing in AI and analytics training for L&D team members

  • Collaborating with data science or analytics teams within the organisation

  • Engaging with external AI and analytics experts or consultants

Final thoughts

AI is not and should not be seen as a silver bullet. For data integration and cleansing, human oversight is still required to manage data types that can't be handled by AI. Insights, analytics, and predictive modeling depend on the quality and relevance of the data, so businesses will still need to invest in making their data infrastructure robust. Automated reporting will still require human judgment and storytelling skills to communicate a meaningful message to the wider business.

While AI can certainly help L&D teams close the data capability gap to some extent, it is important to acknowledge its limitations. L&D teams will still need to invest in building the right data infrastructure, governance processes, and in-house skills to leverage AI effectively. Moreover, AI should be seen as a complement to, rather than a replacement for, human expertise and decision-making in L&D.

Anja Peebles-Brown