ZKM | Center for Art and Media
Basic Research


Adaptive Systems

Dynamical cognitive systems, neural networks and perception.
Starting from the so-called Hodgkin-Huxley-equations we extend the classical neural networks to dynamical ones. The main goal of this project of the Institute for Basic Research is the development of dynamical cognitive systems on the basis of this dynamical neural networks in order to control highly complex dynamical processes, which include humans as agents.
The dynamical cognitive systems are capable to simulate in contrast to the static neural networks which have been successfully trained and applied in several fields. That is why the dynamical cognitive systems are much closer to the functionality of the brains of highly developed mammals. The dynamics recognition, the capability to simulate as well as the possibility of selfmodification are important criteria's for the emergence of creativity.
The project contains not only the development of the theoretical basis of dynamical cognitive systems, where we have been successful already, but also the development of software. Within the scope of this project also lies the the development and application of interface technologies to link humans to artificial brains, and as a consequence of this linkage, the experimental validation of different hypotheses to cognition in psychology. Finally it is planned to construct prototypes of hardware components for later use in industrial applications.
The dynamical structure of neural networks is oriented towards the dynamics of the brain, which is the combination of a simulator and a stimulator. The necessity to process stimulation is self-evident - it is a minimum ability that even primitive brains show. The capability to simulate is restricted to higher developed mammals only, particularly to humans. The simulation is needed for prediction and creative actions. Unlike primitive biological structures highly developed brains with simulation capacity are able to create strategies to organize their environment. The potentiality to creativity is a positive ide effect of the ability to simulate.
In our work done so far, we have shown that the dynamics of the brain is well described with a so-called "chaos control mechanism". On the basis of this procedure we intend to program networks capable for dynamics recognition. Preliminary prototypes exists already, that were excitable through external stimuli. Specifically, we used a human pulse signal for this adaptation process. Furthermore, we think of an intelligent diagnostic system, which will not only have potential applicability in medicine but also in the fields of general cybernetics, that is the control of complex systems.
Parallel to the development of neural networks preliminary experiments to biofeedback have been carried out. This follows the aforementioned intention to link real brains to artificial ones. This should complete the already existing biofeedback applications by linking the dynamical neural networks with the existing interfaces. The exciting question here is: How much can the artificial brains be trained when linked to real ones (via EEG measurements for instance). In other words: To what extend does adaptation take place? It can be assumed that dynamical neural networks will improve the sensitivity for context recognition, which is not sufficiently realized at the moment in the case of classical neural networks trained via Bayesian learning. As a result we want to develop a context-sensitive cognitive system capable to react spontaneously to new situations. Such a system would be maximally convenient for the user.
One can imagine such systems paired with robots, visual-haptic interfaces and video-tracking. The project includes experiments using the mentioned interface technologies. The development of the required system control units is one of our major tasks.
Part of the project is the development of an adaptive gaze based interface for image retrieval "EyeVisionBot". See also the open source variant "openbaar". EyeVisionBot can be seen in the Media Museum of the ZKM since September 2004: Medienmuseum. On June 9, 2005 the project "EyeVisionBot" has been awarded with the doIT Software Award. A list of gaze based interfaces is available at Gaze Based Interaction Group .
Another part of the project is "KI-Arena".

Adaptive cognitive system with stimulus and simulus:

Nested model of the adaptive cognitive system where one simulus simulates the dynamical states of another cognitive system. The goal ist to describe socio-biological behavior:

X-variable of the Rossler system in the chaotic regime:

Parameter adaptation of the target system to the external Rossler signal. The external systems undergoes sudden changes with respect to the value of some parameters which is extremely quickly adapted by the simulus (shown for one parameter):

Human pulse signal:

Adaptation of the target system to the human pulse signal. The target dynamics has arbitrarilly been chosen to be a van der Pol oscillator. A generalisation is posssible:

Series of screenshots showing the adaptation of the target system to a pattern formation emerging in an array of coupled oscillators. This indicates that in the near future an adaptation to even more complex dynamics described by partial differential equations in 2 spatial dimensions is foreseeable:

Neuro-aesthetics measurements (here using electro dermal activity):

Neuro-aesthetics measurements with bio-feedback (here using a visual feedback of ones own processed live video):

Practical works and diploma theses:
Marco Rohrbach, Practical Work (pdf-file ca 1.5 Mbyte), 2001. Co-supervisior: Prof. Horst Prehn, Biomedizinischen Technik an der Fachhochschule, Giessen.
Anette Rohr: "Design eines flexiblen, computerunterst├╝tzten Biofeedback-Systems". Diploma Thesis. Medien und Informationswesen der Fachhochschule Offenburg, February 2003. Co-supervisor: Prof. Riempp, Fachhochschule Offenburg.
Thomas Kenz: Practical work (pdf-file ca 1.7 Mbyte). July 2003.
Lasse Scherffig: It's in your eyes - Gaze based image retrieval in context. ZKM, Karlsruhe, 2005.