Topographic maps are simple prototypes for neuronal structures which emerge from an interplay between intrinsic and activity driven mechanisms. Investigation of the goldfish retinotectal mapping has shown [1997, Retinotectal Map in Goldfish] that activity driven mechanisms are needed even for such a simple system to develop correctly. They are responsible e.g. for the refinement of receptive fields to their final sizes.
However, smart intrinsic mechanisms also allow for seemingly contradictory but powerful behavior. As an example, graded markers on the retina and the tectum allow for location-specific ingrowth of fibers or for a flexible ingrowth in response to artificially induced changes of the size of the retina or the tectum.
Other key features of the organization of the brain are its modular structure and functional specificity of each module. The cortex itself is clearly divided into a large number of areas. The emergence of these and of their complex connectivity pattern, which includes many topographic mappings, are not yet understood. The complexity of this phenomenon demands for a sufficiently complex process of pattern formation in favor of a direct description.
In search of this it is helpful to understand the function of the cortex first. The cortical areas are arranged to process information in parallel as well as hierarchically. Putative functions are given by information theoretic models.
If a net generates the data which have been supplied by the environment it can be tested whether it comprises a good model of the environment. Simple Hebbian learning rules follow from the goal of modelling the data by maximum likelihood. Furthermore, the model can be constrained within a maximum a posteriori framework in order to take into account prior knowledge about the data generation process or from biological observations.
Furnished with a sparseness prior on hidden unit activations, learning will shape the receptive fields of the hidden units to resemble localized edge detectors. Boltzmann machines are powerful tools to investigate the emergence of structure, because their structure is very flexibly defined by a full connectivity between all neurons. With a restricted structure, however, and sparse coding, the Boltzmann machine was introduced into modelling of the visual system [1999, Sparsely Coding Boltzmann Machine]. Another prior on spatially correlated activations leads to topographic map development [2001, Self-Organization of Orientation Maps, ...]. This work reconciles detailed biological phenomena with a short mathematical description in terms of neural learning which is important to understand working principles of the brain and to exploit its strategies.
If the parameters which scale the priors vary within the net, then neurons form different groups which can specialize on different aspects of the data. If different data is generated either in a parallel or in a hierarchical fashion, then these groups can self-organize to capture this structure. This has been modelled using the same maximum likelihood algorithm and parameters [2000, Structured models from structured data]. It was repeated with a biologically more realistic, stochastic Helmholtz machine which lends itself to hierarchical modelling [2000, Emergence of modularity within one sheet of intrinsically active stochastic neurons]. This work shows how the cortex can adapt on a large scale to its input modalities, such as in congenitally blind where parts of the visual cortex have been found to process auditory signals.
In order to explain the non-linear response properties of visual cortical neurons, an attractor network was trained in conjunction with a generative model [2001, Self-Organization of Orientation Maps, ...]. The model accounts for a learnt centre-excitatory surround-inhibitory weight profile, for contrast-invariant orientation tuning curves and for shift invariant responses. This lays the foundations for biologically realistic robust object recognition. By the extension of the attractor network to a ``what'' and a ``where'' area, the visual localisation of a learnt object was achieved [2003, Object Localization ...]. The retrieved location was then used as input into a reinforcement-trained network which learnt the skill of docking a robot at a table and grasping the object [2004, Robot Docking ...]. Furnished with speech interaction this yielded the Visually guided grasping robot MIRA.
``Mirror neurons'' which have been found in the motor cortical area F5 are not only active when a monkey performs a specific task, but also if it sees another performing that task. Such sensory properties of the motor system justify the use of generative models also in the motor cortex. We applied our model of the visual system to the motor system in which robotic vision and sensor inputs are associated with appropriate motor actions. Our network can learn a robotic grasping task in an act of ``self-imitation'' and as an attractor network it is able to mentally simulate the task. In our model of F5 we have found as a result of training both neurons with classical motor properties and those with mirror neuron properties, as found in F5.
For describing cortical function, the maximum a posteriori framework still has not reached its limits. Looking into details, the neurons and connections required by these models should find a better match with the biological substrate. Looking at a large scale, the current approach to modularization is very simple and may not justify the large number of cortical areas. It seems reasonable to extend the self-structuring paradigm assuming more sophisticated functions for the cortical areas to compute. Examples would be (i) to investigate under which conditions a ``what'' and a ``where'' pathway can emerge in the visual system, without pre-determining their role; possibly with the use of second order neurons, (ii) to investigate how the different coordinate systems emerge at which objects are represented in PPC (e.g. eye-, head- or body-centred), (iii) to include sensory, motor and motivationally relevant input such as reinforcement signals from basal ganglia, which would lead to prefrontal cortex modelling, (iv) to include more intricate functionality in each module such as working memory.
If we demand more explanation capability and keep to simple models, then our choice of models will be restricted.