ABSTRACT:

This report introduces ongoing research concerning hardware implementations of spiking neural network on embedded systems. Goal is to implement a spiking neural network in reconfigurable network, more specifically embedded systems.
Spiky neural networks are widely used in neural modeling, due to their biological relevance and high computational power. This report gives investigation of the usage of spiking dynamics in embedded artificial neural networks, which serve as a control mechanism for evolved autonomous agents performing a delayed-response task. Here an evolved spiky network is compared with evolved McCulloch-Pitts networks, while confronting new questions about the nature of spikiness and its contribution to the neurocontroller's processing. On the behavioral level, it shows that in a memory-dependent task, network solutions that incorporate spiking dynamics can be less complex and easier to evolve than neurocontrollers involving McCulloch-Pitts neurons.
AIM:
This research will investigate if it is possible to implement spiking neural networks on re configurable hardware. More specifically it will concentrate on embedded devices because they have the advantage that a processor is closely linked to the neural module, which can coordinate learning, reconfiguration, etc. Furthermore it will research data representation, suitable spiking neural models, training, architecture and space/time-considerations.
Different spiking neural models will be investigated to see which are best suited for hardware implementation and is computationally the most interesting. Here both speed and area are important aspects.
To this date only very limited learning algorithms for spiking neural networks are available. This is mainly because this field of neural network research is pretty young. This research will address this problem.
Through the use of runtime re configurable hardware it is possible to split large networks into smaller sub modules, which can be implemented separately in hardware. Folding them out in time can thus run very large networks. This splitting of neural networks is actually a space versus time consideration.
Due to the fact that a processor is coupled with the neural hardware, it is possible to implement the additional learning phases (which only need to be performed sporadically during the lifetime of the neural module) on this processor. This way training can be done on a model in memory, and afterwards this model is translated to the actual hardware implementation.
PLANNING:
The research will start by simulating many of the spiking neuron models present in the literature in software. This way it can compare the computational power, learning ability and complexity of these different models. During this stage research needs to be done on neural coding and training of these models because a comprehensive theory has not yet been formed.
In a second stage the best of these neuron models will be implemented in digital hardware (no complex connectivity will yet be implemented). In this stage we can evaluate speed and area requirements of these neuron models. We will use FPGAs to speed up simulation and evaluation significantly.
Then we will look at the interconnection of these neurons. Several different approaches are possible:
-Hardwired interconnections,
-Reconfigurable hardware connections,
-Message passing on a bus-structure, etc.
We will research how it is possible to implement a group of topologies as large as possible on the re configurable hardware without asking much resources, or introducing a big delay.
In the last step the best hardware neuron model and interconnection strategy are combined to form a neural hardware module. Research on reconfiguration bandwidth, memory architecture, pipelining, on-chip learning and space/time-considerations is needed. There will also
be special care in making the neural module as reusable as possible and therefore testability and the use of standard System-on-a-Chip busses will be considered .
As a result of this research, it will have a hardware neural network, implemental in digital re configurable hardware and suitable for embedded applications. It will be space/time-scalable, so that it can be easily adapted to the user requirements.
APPLICATIONS:
All areas where embedded devices are used in complex changing environments, the neural network approach could amount for the adaptive and intelligent behavior. Due to the fact that the neural network is implemented in re configurable hardware it is possible to load the network only when it is needed.
It is now possible to solve complex problems in a hybrid way: partially by a neural network and partially by classic algorithmic techniques. Both techniques can thus be used on the sub problems where they are best. Neural networks can for example be used to do pure pattern recognition on data, which is already preprocessed using classic techniques (for example Fourier transformation). The results of the neural network can then be fed to the processor, which can perform further computations.
A classic problem where we are faced with a complex, changing environment is biometrics. This term identifies all techniques used to measure human (or animal) features, for example voice, handwriting or iris recognition. All these human properties evolve during their lives and are very noisy. Hybrid techniques use neural networks to do low level pattern recognition (detecting single syllables) while algorithmic techniques are used in the higher levels (forming words and sentences with the syllables using dictionaries and statistical techniques) .
Another application is adaptive non-linear control. Most, if not all, machines are subject to significant wear and tear. Complex non-linear machines are locally linearised (in a working point) to enable control with standard techniques (like PID). Due to wear, the working point of the machine tends to shift so that an incorrect linear approximation is used and decent control gets impossible. Neural networks have proven to be very good at controlling highly non-linear systems.
Because these neural networks can be trained during operation, they can be adapted to changing situations like wear. Because this neural model can be implemented on an embedded device it is possible to place the control hardware very close to the detectors and actuators. This way delay (dead-time) is minimized, which normally is a big nuisance.
CONCLUSION:
The study of spiky neural networks in the context of embedded evolutionary agents brings forward basic questions regarding spiking dynamics that have not yet been raised. The simplicity and concreteness of EAA models makes them a promising model for computational neuroscience research, and specifically to study the spikiness properties of neurocontrollers. Here it has shown that the presence of evolved spiking dynamics does not necessarily transcribe to actual spikiness in the network, and that the spikiness level can be defined and quantified in several functionally different ways. On a behavioral level it has shown that in tasks possessing memory-dependent dynamics network solutions that involve spiking neurons can be less complex and easier to evolve, compared with MP networks.