A study led by a University of Stirling researcher has looked into how artificial intelligence large language models (LLMs) can improve software for reliability and speed.

Leading the research was Dr Sandy Brownlee, a senior lecturer in the University of Stirling’s Computing Science and Mathematics Division. The team used ChatGPT to automatically update open source software by asking it to make improvements to the code.

“We found that, on the open source project we used as a case study, a LLM was able to produce faster versions of the programme around 15% of the time,” said Brownlee, “which is half as good again as the previous approach.”

“Most interestingly was that the LLM was able to take examples from other parts of the programme that we hadn’t explicitly told it about in order to make these improvements,” Brownlee continued.

The results of the research could be used to improve mobile apps in particular, making them more responsive and less battery intensive on smart phone batteries.

“The most tangible benefit is in your pocket — mobile apps that run more efficiently mean that your battery lasts longer, and the apps will be more responsive when in use,” Brownlee explained.

Software has become increasingly complex, difficult to maintain, and time-consuming for developers to improve. It is also having a growing environmental impact as computers consume more and more energy.

“There is a trade-off here because LLMs cost a lot of energy to make and use,” said Brownlee, “but if they can be used to improve a piece of software that is run many times, the payoff may be worth it.”

Brownlee hopes that the research could help software developers who are working to create more efficient programmes.

“The nature of software developers’ roles will change if automated improvement to software becomes commonplace, moving to a higher level of design, though that is continuing the direction of travel we’ve seen for decades,” he said.

Recommended reading

• ChatGPT a Hit Among Scottish Universities
• Fighting Fire with Fire: Using LLMs to Take Down Cybercriminals
• Report: An Early Look at LLM’s Potential in the Labour Market

A peer-reviewed paper on the study, Enhancing Genetic Improvement Mutations Using Large Language Models, was presented by Brownlee last Friday in San Francisco, at the Symposium on Search Based Software Engineering 2023.

Brownlee led the research and collaborated with Professor Justyna Petke, Professor Federica Sarro and PhD students James Callan and Carol Hanna, all of University College London; Dr Dominik Sobania and PhD student Alina Geiger of Johannes Gutenberg University; and Dr Karine Even-Mendoza of King’s College London.

The study was supported by funding from an Engineering and Physical Science Research grant via UK Research and Innovation, and the European Research Council advanced fellowship grant.