Software testing teams have been led to believe that AI will deliver enhanced speed of development, reduced effort, and support better decisions at-scale. But despite the benefits AI could drive, implementation only succeeds if AI testing meets the quality standards set by humans.

Confidence and reliability are as important as speed.

A report from Leapwork titled The Gap Between AI Hype and Test Automation Reality examined how enterprises view AI for the application of testing, and whether the technology was meeting expectations. 

Over 300 QA professionals were surveyed by Leapwork to build a better picture of how AI will impact testing roles. 

The study found intent to use AI was strong across testing teams, however the technology is being applied at varying rates and with mixed success, impacting testers confidence in AI.  

AI Application Rates in Testing

Overall, 65% said they currently use or explore AI across one or more testing activities – with adoption levels even varying between colleagues.

The survey found AI is a priority for 88% of respondents future testing strategies, with 46% suggesting it is critical or high organisational priority.

Where AI could add most value was debated, for most (66%) faster and easier test creation was valued, followed by broader coverage across critical systems (38%), and earlier defect detection (36%).

At present, respondents estimated on average 41% of their testing is automated while 59% remains manual. 

What’s Hindering Adoption?

The majority of concerns for AI adoption at scale were around accuracy and quality (54%) of AI.

This is creating an environment wherein testers know they could use AI, but are hesitant to apply it. When asked what prevents teams from automating more testing today, ‘tests break too often’ ranked first, followed by ‘difficulty automating flows across systems’ and the ‘time required to update tests’.  These combined factors pose challenges in both tester workload capacity and the reliability of AI.

While interest in AI testing capabilities is high, confidence to adopt at scale is low. 

Time constraints were a key issue. In many cases, testers were interested in applying AI to their roles, but 54% lacked the time to experiment and build confidence to allow them to confidently change current practices. 

Other reasons such as system complexity (45%), budget limits (44%), and lack of skills or expertise (40%), were all contributing challenges. 

Of those surveyed, only 12.6% said they are using AI across key testing activities currently, however, 80% said they expect AI will have a positive impact on testing over the next two years. This suggests that testers expect the technologies accuracy and quality to improve in the short-term which will support adoption. 

Recommended reading

• • AI Agents Are Transforming Enterprise AI Adoption
  • AI Agents Go Mainstream, But Fragmentation Is a Major Obstacle
  • New Whitepaper Reveals Urgent Agentic AI Security Risks

When participants were asked if the effort to maintain the quality of applications had become more difficult in the last 12 months, only 11% felt things had got easier, 43% felt difficulty levels had remained the same, whilst 46% felt things had become harder to some degree. 

With the increasing complexity and volume of systems today, testers have an opportunity to apply AI to ease workloads. 

Currently we are facing a gap between the AI hype and test automation reality. Closing the gap between AI hype and testing adoption requires upskilling, allocating time for experimentation and adopting AI incrementally with robust assessments in place.

Building trust in AI outputs requires robust validation which will build sustained confidence for testers – this won’t happen overnight.