[ad_1]
The 2024 Nobel Prize in Physics has been awarded to John Hopfield and Geoffrey Hinton for their seminal work on artificial neural networks. Today, researchers are continuing along this path by exploring current challenges, in particular by combining software and hardware building blocks to emulate the brain.
The brain remains the king of computers. The most sophisticated machines inspired by it, called “neuromorphic”, today include up to 100 million neurons, or as many as the brain of a small mammal.
These networks of artificial neurons and synapses are the basis of artificial intelligence. They can be emulated in two ways: either with computer simulations, or with electronic components reproducing neurons and biological synapses, assembled into “neuroprocessors”.
These software and hardware approaches are now compatible, which suggests drastic developments in the field of AI.
How does our brain work? Neurons, synapses, networks
The cortex constitutes the outer layer of the brain. A few millimeters thick and the size of a napkin, it contains more than 10 billion neurons that process information in the form of electrical impulses called “action potentials” or “spikes”. The connection point between a neuron that emits a spike (the preneuron) and the neuron that receives it (the postneuron) is the synapse. Each neuron is connected by synapses to around 10,000 other neurons: the connectivity of such a network, the connectome, is therefore prodigious.
The function of neurons is fixed: it consists of summing the signals coming from the synapses, and if this sum reaches a threshold, generating an action potential or spike which will propagate in the axon. It is remarkable to note that part of the processing is analog (the sum of the synaptic signals is continue) while the other is binary (the neuronal response is either the generation of a spikeor nothing). Thus the neuron can be considered as an analog computer associated with a digital communication system. Unlike neurons, synapses are plastic, that is to say they can modulate the intensity of the signal transmitted to the postneuron, and have a “memory” effect, because the state of a synapse can be preserved. in time.
From an anatomical point of view, the cortex is divided into approximately a million cortical columns, which are networks of neurons all having the same interconnection architecture. Cortical columns can be considered as elementary processors, of which neurons are the basic devices and synapses are the memory. From a functional point of view, the cortical columns form a hierarchical network with connections going from the bottom (the sensory sensors) to the top, which allows abstractions, but also from the top to the bottom, to allow predictions: the Our brain’s processors work both ways.
The main challenge of AI is to emulate the functionalities of the cortex with artificial neural networks and synapses. This idea is not new, but it has moved up a gear in recent years with the deep learningor “deep learning”.
Use software to simulate networks of neurons and synapses
The software approach aims to simulate networks of neurons and synapses with a standard computer. It has three ingredients: mathematical models of neurons and synapses, an architecture for interconnecting neurons, and a learning rule that allows the “synaptic weights” to be modified.
The mathematical models of neurons range from the simplest to the most realistic (biologically), but simplicity is required to simulate large networks – several thousand, even millions of neurons – to limit calculation time. The architecture of artificial neural and synapse networks generally includes an input “layer”, containing the “sensory neurons”, and an output layer, the results. Between the two, an intermediate network which can take two main forms: “feedforward” or “recurring”.
In a feedforward network, information is transferred from one “layer” to the next, without a feedback loop to previous layers. On the contrary, in recurrent networks, connections can exist from one layer N to the previous ones N-1, N-2etc. Consequently, the state of a neuron at the moment t depends on both the input data at the moment tbut also the state of the other neurons at the moment t-Δt.which significantly complicates the learning processes.
Learning aims to determine the weight of each synapse, that is to say the intensity with which the spike coming from a preneuron is transmitted to the postneuron, so that the network can respond to a defined objective. There are two main types of learning: supervised when a “teacher” (or “master”) knows the expected result for each input and unsupervised when such a “master” is absent. In the case of supervised learning, it is the comparison between the result obtained for an input and that of the “master” which makes it possible to adjust the synaptic weights. In the case of unsupervised learning, it is a rule like the famous Hebb rule which makes it possible to change the synaptic weights during the different trials.
Building artificial hardware neural and synapse networks
The hardware approach involves designing and manufacturing neuroprocessors that emulate neurons, synapses and interconnections. The most advanced technology is based on the industry of standard semiconductors (known as CMOS), used in our computers, tablets and other smartphones. It is the only sector currently sufficiently mature to manufacture circuits comprising several thousand or millions of neurons and synapses capable of carrying out the complex tasks required by AI, but technologies based on new devices are also offered, for example in spintronic or using memristors.
Like biological networks, artificial hardware neural and synapse networks often combine an analog part for the integration of synaptic signals and a digital part for communications and memorization of synaptic weights.
This type of mixed approach is used in the most advanced technologies, such as the chips from the European Human Brain project, from Intel, or TrueNorth from IBM. The TrueNorth chip, for example, combines a million neurons and 256 million programmable synapses, distributed into 4096 neuromorphic cores – comparable to the cortical columns of life – linked together by a communication network. The power consumption of the TrueNorth chip is 20 mW per cm2while that of a conventional microprocessor is 50 to 100 W per cm2i.e. an energy gain greater than 1000 (the custom is to consider the “surface power density”, because not all chips have the same surface area).
Will the future be hardware or software?
Software artificial neural and synapse networks make it possible to elegantly solve many problems, particularly in the areas of image and sound processing and more recently text generation. But learning networks of recurrent artificial neurons and synapses remains an example of major difficulty, whether by supervised or unsupervised methods. Another problem: the computing power required becomes considerable for the networks of large artificial neurons and synapses needed to solve complex problems.
For example, the impressive results of the conversational program “GPT-3” are based on the largest network of artificial neurons and synapses ever built. It has 175 billion synapses, and requires considerable computing power made up of 295,000 processors which consume electrical power of several megawatts, that is to say equivalent to the power consumed by a city of several thousand inhabitants. This value should be compared to the few watts consumed by a human brain which performs the same task!
The hardware approach and neuroprocessors are much more efficient in terms of energy, but they suffer from a major difficulty: scaling up, that is to say the manufacturing of several million or billion neurons and synapses and their interconnection network.
In the future, and to the extent that neuroprocessors use the same CMOS technology as usual processors, the co-integration of software and hardware approaches can open the way to a new way of conceiving information processing and therefore an efficient and energy-efficient AI.
Alain Cappy, Professor Emeritus in Electronics, University of Lille
This article is republished from The Conversation under a Creative Commons license. Read the original article.
[ad_2]
Source link -100