The report from Kubrick Group highlights the impact of a tightening labour market for AI and data professionals. 

For some context before we get into it, LinkedIn has reported a 20% decline in job vacancies since the pandemic, with large enterprise hiring down by 40% in comparison to pre-pandemic levels.

Is AI to blame or is there more at play? 

Many have suggested that the automation of junior-level tasks is responsible for the slowdown in the labour market. However, with AI yet to be adopted at scale, this would indicate there are broader factors at play.

The World Economic Forum has echoed this sentiment: economic uncertainty, monetary policy shifts, and post pandemic hiring rebalancing are all thought to be having an effect. 

What AI does account for, however, are pockets of change. In organisations which are ahead of the curve (think technology, financial services and healthcare sectors), increased productivity generated by AI means organisations can grow revenue without increasing headcount. 

However, demand for AI and data skills continues to rise despite the broader decline in job postings. 

Kubrick’s study of one million job postings across the UK and US for AI and data professionals gives clear insight into drivers of supply and demand.

A Constrictive Labour Market

Kubrick’s study analysed roles across data engineering, platform engineering, data and AI governance, data product management, AI and machine learning and applied decision intelligence. These were examined for entry level, mid-to-senior, director and executive level roles.

The research found that AI isn’t necessarily replacing roles, rather the experience level required for vacancies has changed. 

Demand for mid-senior roles outweighs entry-level roles in all areas of data and AI. In the UK, demand for mid-senior professionals outweighs requirements for junior roles by 2.5x across the AI and data market, as a whole. 

A striking 68.8% AI  and data job postings were for mid-senior level posts, with just 27.3% for junior vacancies, 3% for director and 0.9% for executive level vacancies. Whilst director and executive vacancies are always in the minority, the split between junior and mid-senior opportunities has changed.

Whilst the entire market is feeling this strain, the problem is aggravated in data engineering. This increases to 3.7x more mid‑senior roles than junior role postings in the UK. 

Kubrick suggested this was being driven by an increase in offshoring; India, Brazil and Poland have become hot spots for the practice due to their competitive pricing. This is coupled with platform modernisation; modern technology and automation are reducing the requirements for foundational and repetitive tasks, often carried out by junior staff. 

Investment in AI has created 1.3 million jobs, Kubrick reports. Interestingly, AI roles have the least discrepancy between junior and mid-senior roles, with more experienced positions just twice (2.07x) as in demand. 

LinkedIn’s Jobs on the Rise report indicated that AI engineering is the fastest growing role across all markets, with a 146% increase in posts in the last year. 

Because these roles span everything from “data prep & solution prototyping” to “deployment and operationalisation”, organisations have the opportunity to employ a diverse team with a breadth of experience.

Experience vs Skills

What is interesting is the demand for experience over skills. However, because many AI and data technologies are relatively new, the difference between junior and experienced professionals’ technical exposure may be smaller than expected. 

The demand for AI and data professionals mirrors the trend we saw in growing demand for software engineers in the last decade. Lessons learned from these waves suggests that organisations should focus on “verifying real capability”, suggests the World Economic Forum, rather than prioritising appointments based on previous job titles or degrees.

When hiring for in demand skillsets, organisations must look beyond traditional candidate pools and consider new approaches such as training programmes and hiring people with domain experience that can upskill in emerging capabilities. 

Costs & Economic Repercussions

Whilst the current market demands are affecting professionals on a personal level, they also pose risks to the wider economy. Sustainable growth, succession planning, skills gaps and workforce planning may be challenged by not giving junior professionals work opportunities. 

This may also create an environment wherein experienced professionals are taking on more junior, monotonous and functional tasks, reducing the value for money organisations get from experienced employees. 

Roles with AI skills command salaries that are 23% higher than comparable roles. This further increases the ROI required for AI projects to be financially viable. Organisations must review their skills and experience mix to ensure AI projects ROI, and consider where junior, more cost-effective resources can be deployed. 

Future-Proofing

Demographic trends are intensifying skills gaps. 6 million Gen X-ers are due to retire within 5 – 7 years reducing our total workforce. Succession plans and talent development programmes must be embedded to ensure organisations can sustain their talent pipelines. 

Recommended reading

• Report: More than Half of Firms Regret Cutting Jobs for AI
• Amazon Targets UK AI Skills Gap with New Training Push
• AI Has Moved From Experimentation to Execution in Enterprise IT

It is becoming apparent that organisations are struggling to embed, scale and capitalise on AI opportunities at the pace they expected.  

The report states: “Board dissatisfaction is rising, as the tension between expectation and reality grows: 81% of Executives believe AI will give them a competitive advantage, yet only 12% have the data  architecture mature enough to support it”.

The report suggests that targeting the same pool of talent as the rest of the market will become a weakness for organisations, reducing points of differentiation, and forcing organisations to pay inflated salaries. 

Organisations can differentiate their strategy by investing in junior professionals, reducing costs, creating “resilience against talent shocks” and unlocking individuals’ potential.