It’s hard to overstate just how important data has become – there’s little room for business intuition and why should there be when you can quantify?

The problem is, the amount of data a company can get its hands on is quite simply staggering, yet the recruitment practices to get the right people in and ensure that end-to-end data project skills are being utilised simply haven’t been keeping up for a while now.

It’s a problem Michael Young, CEO and Founder of Glasgow-based data and tech recruiter, MBN Solutions, identified long ago – recognising the fracturing of ‘tech skills’ into much more refined, often data-driven roles that require specialist talent and training.

Now, with mass adoption of cloud computing and advanced analytics, mass data modernisation projects have spawned. These require companies to quickly build teams of high-performance data engineers.

The demand for these data engineers is increasing by 50% every year which is causing inflated salaries, lower supply of people and huge competition in the recruitment market that’s in danger of being monopolised.

In this interview with DIGIT, we catch up with Young to discuss the ways his own company has embedded itself within the tech ecosystem to nurture better data training and recruitment and to assess the landscape that’s got us to a point where data-specialist staff have become so important.

The data science/data engineering quandary

In many instances, we’re seeing software engineers and more traditional data scientist pivot to try and match up with the growing demand for data engineers.

“This is a problem because it means that a lot of data scientists aren’t learning the traditional software/tech side of things – we’ve seen people struggle to take on end-to-end data science projects,” says Young.

He adds: “Data science is really maturing – we’re seeing more and more models being pushed into production and it looks like ML Ops is going to feature really heavily over the next few years, so data scientists gaining experience in infrastructure and Dev Ops is really an advantage.”

Young notes, however, that none of this changes the fact that there’s an astonishing demand for data engineers and MBN solutions is certainly trying to do its part in addressing the shortfall.

On this, Young says: “We’ve really seen this shift in terms of maturation in the market. As a result, we’ve stated to do work with The Data Lab, Scottish universities, as well the industry at large to run data engineering forums where we’re looking at what these data engineers need to be doing in university to match up to the tangible industry requirements.”

From Young’s well-informed point of view, we really need these graduates to hit the ground running when they come out of university – there isn’t as much of a bedding-in period.

The biggest issue arguably lies in talent acquisition – these companies need to be connected with the appropriate talent.

On this, Young says: “We’ve worked very hard over the past 15 years to help our clients find talent that others can’t, basically.

“Anybody can use LinkedIn now to find people, but we’ve spent this time building communities (The Scotland Data Science and Technology Meetup page recently passed 5,000 members).”

The fact is, an army of primed data engineers isn’t going to just appear – at least certainly not at the scale at which they’re required. To meet the dearth, it’s going to have to come from within.

Young adds: “We need to be doing everything we can to upskill, ease grads coming out of university into adaptive positions, help people transition from things like software engineering to data engineering – whatever it takes.”

The massive amount of data people are using at the moment – particular in line with the rise of cloud computing, has created really buoyant market. Data has become the beating heart of so many industries and if Scotland really wants to compete on a global scale and be a tech forerunner, we have to create an environment in which data engineers can emerge as quickly and efficiently as possible.

As a recruiter in the data space, MBN has gone beyond making those vital social connections that build viable networks – they’ve targeted other areas they feel are critical for building a vocational infrastructure.

“We’ve worked closely with academia, building the MBN academy – where we placed over 350 masters students into industry in Scotland for their summer internships. On the back of that project as well, we’ve trained well over 1,000 masters students within data over 11 universities on employability skills, soft skills, CV building and interview skills.”

For Young, embedding within the market and – in a sense – making things happen in that space in the knowledge that guiding these skilled people towards careers in things like data engineering is not only good for them, but the broader Scottish concern.

One of the big problems that Michael sees with modern businesses is a lack of clarity and differentiation on data jobs and tech jobs – and more broadly, how data is handled in their business and who should be handling it.

There are prevalent misnomers of coverall data roles that are interchangeable with tech roles, that ends up seeing too many skilled people doing tasks they really shouldn’t be. Recruitment in this space has to evolve apace with the ever branching and refining nature of the tech landscape.

“There’s a lot of big organisations that have talent acquisition teams who are tech-based recruiters, working for a long time in that space are undoubtedly good at what they do. But you need a specialist data recruiter – someone who has been embedded in that market for a number of years who can find talent that’s not just traditionally on job boards and LinkedIn.”

What are the traits of a contemporary data engineer and how can companies attract them?

The things that make winning data engineering is something that’s salient not only to the engineers themselves who, of course, want to be able to position themselves as best they can. But conversely, businesses that want to attract the best talent have to do much more than they’ve done historically.

On this, Young says: “The tables have really turned for big companies and well-funded startups. It used to be very straightforward for them in that they would have queues of people wanting to come work for them intuitively.

“Now, there’s so much demand for these skilled data professionals, as well as technology professionals, engineering, sales, marketing – basically across the whole board. These skilled individuals have got choice now.

“Social purpose has become a huge factor in what company’s people choose to work for – you need to back a cause, earnestly. Companies need to be doing a lot more. For example, one of the directors at MBN, his son, was looking for a summer internship while he was doing his computer science degree. We thought he would come back talking the tools he used, the tech etc… but the big headline was about how purpose-led this organisation he interned with was.”

This is symptomatic of where so many companies are falling down – there’s a much more prescient obligation to be part of something bigger than yourself.

With this, Young says you – as a company – must ask yourself: “How are you involved in good causes? How are you nurturing your people? Do you have a good training and development plan? Simply put, purpose-led businesses have become the top choices for the best talent.”

All of this comes together, rather neatly, in Young’s “post-Covid” view of the data talent market in Scotland, the UK and beyond. As his own business pivoted to adapt to the ‘new way of working’ – with flexibility and adaptability at its heart – he examined and refined his own purpose inside and outside the office (including further exploring his fascination with the culture of indigenous Amazonian tribespeople), Young’s view of the role of the data talent specialist is crystal clear: “We can often find the unfindable – the ‘unicorns’.

“We know where they hang out. We’ve been organising events that they’ve come to for years, we probably taught them when they were at university and, they’ll know many of us at MBN as long-term trusted advisors.

Recommended

• ‘Stewart Review’ to support business ambitions for Scots women
• Smart Data foundry appoints new CEO, Frank Gauld
• Glasgow University develops quantum computing tool for gravity research

“We understand what it means to have both talent and purpose and, more than that, we know how to position great brands to these in-demand talents”.

Attracting the best talent in data is going to be the Darwinian marker for survival for so many companies over the next decade. Now, simply offering attractive wages isn’t enough – even more so with the proliferation of remote working. A data rock star’s wage here will likely pale in comparison to someone in Singapore or Boston – so we must do more that goes beyond money.

To make Scotland a data utopia (vocationally-speaking), there has been understanding and respect of the roles that these make up – they’re not tech roles as we’ve always imagined them. Most importantly, looking after your own people’s prospects and the prospects of the world around are becoming every bit as prescient as looking after your talent’s bank accounts.

Get the latest news from DIGIT direct to your inbox

Our newsletter covers the latest technology and IT news from Scotland and beyond, as well as in-depth features and exclusive interviews with leading figures and rising stars.

To subscribe, click here.