Given the huge problem-solving potential of artificial intelligence (AI), it wouldn’t be too far-fetched to think that AI could help us combat the climate crisis. However, when we consider the energy needs of artificial intelligence models, it becomes clear that technology is not only a solution to the climate problem, but also a part of it.
Emissions arise from AI-related infrastructure, such as the construction and operation of data centers that process the large amounts of information required to maintain these systems.
But different technological approaches to how we build AI systems can help reduce the carbon footprint. Two technologies in particular show promise for doing this: accelerating neural networks and lifelong learning.
The lifespan of an AI system can be divided into two phases: training and inference. During training, a relevant data set is used to build and tune – improve – the system. In inference, the trained system produces predictions on previously unseen data.
For example, training an artificial intelligence to be used in driverless vehicles will require a dataset consisting of many different driving scenarios and decisions made by human drivers.
After the training phase, the artificial intelligence system will predict effective maneuvers of a driverless car. Artificial neural networks (ANNs) are a fundamental technology used in most existing artificial intelligence systems.
They have many different elements called parameters, the values of which are set during the training phase of the AI system. These parameters can reach over 100 billion in total.
While a large number of parameters improves the capabilities of ANNs, it also makes training and inference processes resource-intensive. To put things in perspective, training GPT-3 (the current ChatGPT’s leading AI system) produced 502 metric tons of carbon; This is equivalent to driving 112 gasoline cars for a year.
GPT-3 also emits 8.4 tonnes of CO₂ per year due to extraction. Since the AI boom began in the early 2010s, the energy requirements of AI systems known as large language models (LLMs), the type of technology behind ChatGPT, have increased by 300,000 times.
This trend will continue as the ubiquity and complexity of AI models increases, potentially making AI a significant contributor to CO₂ emissions. In fact, our current estimates may be lower than the actual carbon footprint of AI due to the lack of standard and accurate techniques for measuring AI-related emissions.
Speeding up neural networks
New technologies mentioned earlier, including neural networks (SNNs) and lifelong learning (L2), have the potential to reduce the ever-increasing carbon footprint of AI, with SNNs acting as an energy-efficient alternative to ANNs.
ANNs work by processing and learning patterns in data, allowing them to make predictions. They work with decimal numbers. The computer must be very precise in order to make accurate calculations, especially when multiplying numbers with decimal places together. Because of these decimal numbers, ANNs require a lot of computing power, memory, and time.
This means that as networks grow larger and more complex, ANNs consume more energy. Both ANN and SNN are inspired by the brain, which contains billions of neurons (nerve cells) connected to each other through synapses.
Like the brain, ANNs and SNNs have components that researchers call neurons; However, these are artificial, not biological. The main difference between the two types of neural networks is the way individual neurons pass information to each other.
Neurons in the human brain communicate with each other by transmitting intermittent electrical signals called spikes. The spikes themselves do not contain information. Instead, the information lies in the timing of these spikes. This binary, all-or-none property of spikes (usually represented as 0 or 1) implies that neurons are active during spikes but are inactive otherwise.
This is one of the reasons for energy efficient processing in the brain.
Just as Morse code uses specific sequences of dots and lines to transmit messages, SNNs use patterns or timings of spikes to process and transmit information. That is, artificial neurons in ANNs are always active, while SNNs consume energy only when a spike occurs.
Otherwise, they have near-zero energy requirements. SNNs can be up to 280 times more energy efficient than ANNs.
My colleagues and I are developing learning algorithms that can bring SNNs closer to the energy efficiency exhibited by the brain. Lower computational requirements also imply that SNNs can make faster decisions.
These features make SNNs useful for a wide range of applications, including space exploration, defense, and self-driving cars, due to the limited energy resources available in these scenarios.
L2 is another strategy we are working on to reduce overall energy requirements over the lifecycle of ANNs.
Training ANNs sequentially (where systems learn from data sets) on new problems causes them to forget their previous knowledge as they learn new tasks. ANNs must be retrained from scratch when their operating environments change, further increasing AI-related emissions.
L2 is a collection of algorithms that enable AI models to be trained sequentially on multiple tasks with little or no forgetting. L2 allows models to learn throughout their lives, building on their existing knowledge, without having to retrain from scratch.
The field of artificial intelligence is growing rapidly and other potential developments are emerging that could reduce the energy demands of this technology. For example, building smaller AI models that exhibit the same predictive capabilities as a larger model.
Advances in quantum computing, a different approach to building computers that leverage phenomena from the world of quantum physics, will also allow for faster training and inference using ANNs and SNNs. The superior computing capabilities offered by quantum computing could enable us to find energy-efficient solutions for AI on a much larger scale.
The problem of climate change requires us to try to find solutions to rapidly developing areas such as artificial intelligence before the carbon footprint grows too much.
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Shirin Dora does not work for, consult, own shares in, or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond her academic duties.