Hardware-realisable neural networks


The pRAM model

The probabilistic RAM (pRAM) model is a hardware-focused neural model that uses probabilities as weights, but quantises these so finely (by storing the values in a 16-bit register) that they can effectively be treated as continuous in the range [0,1]. This allows a wide range of neural network learning methods to be used in a hardware-implementable context, and also makes it possible to analyse the operation of the learning rules using continuum mathematics. The way stochastic behaviour is introduced into the pRAM model is akin to the synaptic noise in real neural systems caused by the tendency of neural junctions to leak small quantities of neurotransmitter even when no signal is being passed, so that neurons may thereby sometimes fire spontaneously. Such noisy neurons were first described by Prof John G Taylor (then of Dept of Mathematics, KCL) in 1972, and this model in part underpinned the proposal of the pRAM model by Gorse and Taylor in 1988. In the early days of the pRAM it was intended to use the model mainly for neurobiological modelling, with electronic hardware possibly being introduced to speed up large-scale neural simulations. However with the involvement in the early 1990s of Prof Trevor G Clarkson, Dr Terry C K Ng and other then-members of the KCL Dept of Electronic and Electrical Engineering, notably Dr Chris C Christodoulou and Dr Yelin Guan, it became apparent pRAM-based technology could also be used for a large number of industrial applications. Theoretical work in the later 1990s also expanded the range of abilities of the pRAM, with models that were able to learn real-valued functions by manipulating the lengths of the spike trains used to represent them, the experimental work using this new model being largely carried out at UCL by Dr David A Romano-Critchley.

Hardware implementation: the pRAM-256 chip

An implementation of the pRAM model with true on-chip learning was first proposed by Clarkson, Ng, Gorse and Taylor in 1992. This was subsequently developed into the pRAM-256 system, in which 256 6-input pRAMs, with user-reconfigurable interconnections, were contained on a single chip. These chips could be connected together to produce systems with thousands of trainable 'silicon neurons'.

pRAM applications

pRAM systems were shown effective in a wide range of tasks, for example speech/speaker recognition, automatic target recognition, ATM connection admission control, time series prediction, and neurocontrol problems such as the classic 'inverted pendulum' scenario. The bit-stream communication between pRAM neurons, rather than being a hindrance to the system when learning, appeared actively beneficial in promoting generalisation. Many pRAM applications exploited the positive use of noise in this way; it is possible the brain also uses this in those regions where there is a high level of synaptic leakage and where spontaneous firing is therefore common.