The cost of developing AI language models can be astronomical. For example, OpenAI’s latest language model GPT-4 was the result of a reported $100 million (£78.6m) investment according to the company’s CEO Sam Altman. 

While some chatbots are available to use for free, many organisations look to create their own, which can cost around $4,000 (£3,100), according to a blog from Accubits. Even machine learning datasets can cost over £75 per task, and labelling single image datasets could cost roughly £2.75 per image, according to research from Stanford University. In addition to these costs, organisations also need to rent the necessary cloud computing platforms which can add more costs to the final product. 

Researchers at the University of Edinburgh, in collaboration with the University College London looked into ‘efficient training methods’ for machine learning. They did this to compare conventional and alternative approaches to improve cost-effectiveness.

To do this, the team looked at three main categories of efficiency: Batch selection is processing groups of data elements instead of individual components, this speeds up processes and lowers costs for large datasets; Layer stacking uses multiple layers of neural network units to process data in sequence; Efficient optimisers which are algorithms that accelerate search function and minimise wasteful operations.

Recommended reading

• LinkedIn Hacking Campaign Illustrates Rise in Zero-Day Exploits
• Electric Salmon Tagging Project to Start in Moray Firth
• Ofcom Investigates Telecoms Industry Over Contract Price Rises

The research found that batch selection did not consistently yield significant improvements, layer stacking showed minor improvements in training and validation results but these improvements diminished over longer training periods, and efficient optimisers failed to consistently provide superior results. 

The study therefore concluded that cost saving measures like these may reduce the financial cost, but also reduce the quality of the final product. According to them, reasonable performance levels usually come with a large price tag. “We found only a few settings where some of the considered algorithms improved over the baseline,” said the report.