Although the term “machine learning” was coined as far back as the 1950s, its only been in the last 20 years that the discipline itself has started to gain traction in the enterprise. In an attempt to understand just how companies are using these sophisticated capabilities O’Reilly Media asked over 11,000 data practitioners about the state of machine learning in business today.

The study focused on areas to answer questions about machine learning deployment such as –

• How experienced are companies with machine learning adoption, in terms of number of years deploying models in production?
• What has the impact been on culture and organisation?
• Who builds machine learning models: internal teams, external consultants, cloud APIs? and
• How are decisions and priorities set and by whom within the organisation?

Investing in Data Science Skills is Key

According to the results in the survey the companies that are making the most progress with machine learning are ones that have invested heavily in developing or acquiring appropriate skills in-house, such as data scientists and data engineers. So far, machine learning systems tend to be the ones developed within the enterprise, rather than sourced from third parties or utilised via cloud services.

Over 50% of respondents use internal data science teams to build their machine learning models, while only 3% use cloud-based services like AutoML, and only 8% rely on external consultancies.

The survey also examined how these machine learning systems are developed, and whether companies are paying attention to new compliance and regulations, as well as eliminating bias in their algorithms.

54% respondents who belong to companies with extensive experience in machine learning check for fairness and bias. Overall, 40% of respondents indicated their organisations check for model fairness and bias.

Similarly, 53% respondents who belong to companies with extensive experience in machine learning check for privacy which could be attributed to the effect of GDPR which mandates “privacy-by-design”. As a result, more companies will add privacy to their machine learning checklist, says O’Reilly.

The most successful companies and deployments use more specialised roles such as data scientist and data engineer in lieu of older roles such as business analyst, and use their own internal data science teams to build machine learning models the study concluded.