Ernst Strüngmann Forum

Contributors Danielle S. Bassett, Randy M. Bruno, Elizabeth A. Buffalo, Michael E. Coulter, Hermann Cuntz, Stanislas Dehaene, James J. DiCarlo, Pascal Fries, Karl J. Friston, Asif A. Ghazanfar, Anne-Lise Giraud, Joshua I. Gold, Scott T. Grafton, Jennifer M. Groh, Elizabeth A. Grove, Saskia Haegens, Kenneth D. Harris, Kristen M. Harris, Nicholas G. Hatsopoulos, Tarik F. Haydar, Takao K. Hensch, Wieland B. Huttner, Matthias Kaschube, Gilles Laurent, David A. Leopold, Johannes Leugering, Belen Lorente-Galdos, Jason N. MacLean, David A. McCormick, Lucia Melloni, Anish Mitra, Zoltán Molnár, Sydney K. Muchnik, Pascal Nieters, Marcel Oberlaender, Bijan Pesaran, Christopher I. Petkov, Gordon Pipa, David Poeppel, Marcus E. Raichle, Pasko Rakic, John H. Reynolds, Ryan V. Raut, John L. Rubenstein, Andrew B. Schwartz, Terrence J. Sejnowski, Nenad Sestan, Debra L. Silver, Wolf Singer, Peter L. Strick, Michael P. Stryker, Mriganka Sur, Mary Elizabeth Sutherland, Maria Antonietta Tosches, William A. Tyler, Martin Vinck, Christopher A. Walsh, Perry Zurn

ISBN: 9780262043243
Series Editor: J. R. Lupp
Editorial Assistance: M. Turner, A. Ducey-Gessner, C. Stephen
Photographie: N. Miguletz
Lektorat: BerlinScienceWorks

---

This Forum was supported by the Deutsche Forschungsgemeinschaft

The German Research Foundation

Cerebral Cortex 3.0: Complexity and Computation

Wolf Singer, Terry Sejnowski, and Pasko Rakic, Chairs

Program Advisory Committee

David Poeppel, Pasko Rakic, Terry Sejnowski, Wolf Singer, Peter Strick, and Julia Lupp

Goals of the Forum

• To explore the commonalities and differences in collaboration as observed in biological, social, and technological systems,
• To identify core drivers of and constraints on collaboration and the conditions for its emergence, stabilization, and fractionation,
• To outline putative generic architectures for processes of collaboration, and
• To model how commons may be created, consumed, and destroyed during collaboration.

Integrating diverse perspectives, the Forum will work to develop a comprehensive framework to support future work.

Context

The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. [For context, collaboration is understood as cooperation between agents toward mutually constructed goals.] Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved.

The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economical means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. On the other, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible.

To remedy this knowledge gap requires a comprehensive research program. At its core, a theoretical framework must link pertinent aspects of collaboration across spatiotemporal scales and contexts. This task is a tall order, yet given current pressures on human–human, human–machine, and future machine–machine collaboration, we believe that an attempt must be made for a first survey.

---

This Forum is supported by the Deutsche Forschungsgemeinschaft

The German Research Foundation

Recent advances in artificial intelligence, addressed as “Deep Neural Networks” (DNNs), have led to the design of artificial systems whose performance on selected cognitive tasks approaches that of humans. Striking structural similarities support the view that DNNs exploit the same computational principles as natural brains: (a) Both DNNs and sensory systems consist of multiple, hierarchically organized layers of integrator units that are connected via diverging and converging feedforward pathways. (b) The gain of these connections is adjusted by an iterative learning process to generate invariant and well-classifiable responses to trained patterns at the output layer. (c) The response properties of the units “recorded” in DNNs trained with natural objects and scenes resemble those of neurons at comparable hierarchical levels of natural systems. However, the implementation of the learning process is different: Unsupervised and supervised Hebbian synaptic modifications in biological systems versus an error-driven backpropagation algorithm in DNNs. Although backpropagation is biologically implausible, it leads to modifications similar to Hebbian learning. DNNs provide a model system for analyzing the principles for how information is distributed in neural populations.

However, several arguments suggest that natural systems have additional strategies of information processing that are likely to differ radically from those currently used in AI systems.

Network nodes in natural systems have a high propensity to oscillate.

• Nodes within the same layer are coupled by a dense mesh of recurrent connections that are endowed with Hebbian synapses, exhibit a high degree of topological specificity, and are modified by experience.
• Interactions within layers are gated by extremely complex, heterogeneous networks of inhibitory interneurons, whose synaptic connections are susceptible to use-dependent modifications and by a host of modulatory systems whose role in information processing is still little understood (e.g. up and down states).
• Important components of the functional architecture develop gradually only during postnatal life and are shaped by experience.
• Natural systems exhibit very rich, high-dimensional nonlinear dynamics.

These new insights justify considering the brain as a complex, self-organized system with nonlinear dynamics in which principles of distributed, parallel processing coexist with serial operations within highly interconnected networks. The observed dynamics suggest that cortical networks are capable of providing an extremely high-dimensional state space in which a large amount of evolutionary and ontogenetically acquired information can coexist and be accessible to rapid parallel search for the interpretation of sensory signals and the generation of complex motor commands.

---

This Forum is supported by the Deutsche Forschungsgemeinschaft

The German Research Foundation

In 1987, a Dahlem Conference on the “Neurobiology of Neocortex” was convened “to identify principles of cortical operations, to challenge system specialists to become more eclectic in their interests, to exchange information about various systems, and to evaluate common properties among cytoarchitectonically and functionally distant areas of the cortex” (Cortex 1.0).At the time, systems neuroscience was guided by a behaviorist stance. Stimulus-response paradigms prevailed and the research strategy consisted of analyzing the transformation of response properties of individual neurons along processing streams, extending from sensory organs to executive structures. This approach was extremely successful and supported the notion of serial processing across hierarchically organized cortical areas. This view agreed with early anatomical data, which emphasized that feedforward connections exhibit high topographic precision and possess strong driving synapses whereas feedback connections are diffuse and only modulatory. However, advances in the analysis of the cortical connectome, the introduction of multisite recording techniques, and the development of imaging methods to assess whole brain activity have generated data that (a) necessitate an extension of classical views, (b) raise novel questions, and (c) are likely to provide new solutions to old problems.

In 1987 genetic determinants of cortical organization were viewed in terms of prevailing notions of the DNA-RNA-Protein sequence, which establish the initial cell phenotype and its subsequent connections. The evolutionary changes were considered to be a result of random gene mutations, which, if reproducible and positive, help survival of the species. In the meantime, the advances made in -omic methods and concepts of molecular biology elaborated and modified this schema to include the role of “epigenetic mechanisms that include regulatory elements such as non-coding DNA and miRNA. The evolutionary expansion and elaboration of the cerebral cortex that culminates in humans is considered to be a result not only of the increased number of cortical neurons, but also the genesis of the new cell phenotypes, modification of neuronal migration, and introduction of new cortical areas along with their local and long distance connections. The genetic and molecular origins of these evolutionary innovations are only beginning to be understood and so far accumulated data indicate the validity of the initial concepts that need to be modified to include findings obtained with new techniques and approaches.

The novel anatomical and functional data suggested that processing is distributed in densely coupled, recurrent networks capable of supporting complex dynamics. Even simple cognitive and executive functions have been shown to involve widely distributed networks. Furthermore, it became increasingly clear that the brain plays an active part not only in the generation of movements but also in the processing of sensory information; this led to the notions of active sensing, predictive coding, and Bayesian inference. These newly discovered processing strategies obviously require a high degree of coordination of the distributed processes, suggesting that special mechanisms are implemented to dynamically bind local processes into coherent global states and to configure functional networks on the fly in a context- and goal-dependent way. To examine the mechanisms serving this self-organizing coordination, an Ernst Strüngmann Forum was convened in 2009 (Cortex 2.0).

Since then there have again been dramatic advances in novel techniques for neuroscientific inquiry and these have been game-changing in terms of our ability to characterize and intervene with neural circuits at ever higher resolutions. Also, new concepts on computational strategies have been developed that may be relevant for neuroscience. Theoreticians have begun to explore and appreciate the computational power of self-organizing recurrent neuronal networks (SORN), the respective concepts being addressed as reservoir, echo-state, and liquid computing. Neurobiological investigations of these concepts are, however, still rare.

---

This Forum is supported by the Deutsche Forschungsgemeinschaft

The German Research Foundation

This Forum (Cortex 3.0) is being convened to explore to which extent the rich data accumulated over the past decade can be embedded in unifying conceptual frameworks. It is our hope that it will contribute to the identification of gaps in knowledge and generate suggestions for promising research directions. To do so, we will approach these goals from four angles, summarized briefly below.

Group 1: Evolution and ontogenetic development of cortical structures

Evolution: What distinguishes cerebral neocortex from other layered structures (cerebellum, tectum, hippocampus, pallium in reptiles and birds) and integrative centers of invertebrates (insects, cephalopods)? Is there any evidence for independent evolution of cortical structures? Are there some unique and novel principles of neuronal organization (question also for Group 2)? Which factors cause the augmentation of cortical surface? What are the characteristics of areas added late in evolution? However, a even more intriguing and challenging question is how human neocortex (generally considered as the biological substrate of some unique cognitive abilities) acquires new types of neurons and pattern of synaptic connections that are not observed in other mammals? How are such additional cortical areas integrated in existing architectures? (question also to Group 2)?

Ontogeny: Are the radial unit and protomap hypotheses of cortical development and evolution, which postulate that columnar, laminar and areal organization of the neocortex are formed by migration of postmitotic neurons from the proliferative ventricular (VZ) and subventricular (SVZ) zones of the dorsal telencephalon, sufficient and the final cornerstone? What is the role of the subplate (SP) zone? How are interneurons, which immigrate from the SVZ of the ventral telencephalon incorporated? How are late-arriving neurons integrated into existing circuitry? When does neurogenesis in the cerebral cortex stop? What determines cytoarchitectonic differences and input-output connections between areas? Which components of cortical circuitry are genetically predetermined and fixed and which are modifiable by epigenetic influences? What distinguishes early (classical critical period) and late (adolescence) developmental periods of the cortex? What is the criterion for maturity? What constrains adult plasticity? How does a deficit in cell production, placement and connectivity impact the cognitive abilities of offspring?

---

Group 2: The cortical connectome

Intraareal connectivity: Is there a canonical circuit for the networks of excitatory and inhibitory neurons across areas and species? Are there specific features which characterize primates and humans? Do cytoarchitectonic differences reflect different computational functions, differences in afferent and efferent connectivity, or both? Is the concept of a cortical module justified? How do areal boundaries affect intrinsic connectivity (tangential connections, dendritic arbors, distribution of modulatory inputs)?

Interareal connectivity: Is the coexistence of cortico-cortical and cortico-thalamo-cortical feedforward loops universal? Does the same principle hold for top-down connections? Are there exceptions to the rule of reciprocity of interareal connections? What is the evidence on inhibitory long-range connections? Are there rules explaining the heterogeneity of conduction velocities (myelinated vs. unmyelinated, thick vs. thin axons) How are noncortical processors connected to neocortex (cerebellum, basal ganglia, hippocampus, tectum, limbic nuclei, modulatory systems), and are there overarching principles? Can graph theory provide unifying concepts? Can the cortical connectome be deconvolved to approximate a deep neuronal network, or are there principle differences?

---

Group 3: Functional properties at the cellular, microcircuit and areal level

Cellular level: Is there anything special with respect to the computational capacity of excitatory and inhibitory neuron classes (variability of conductances, pacemaker currents, coincidence sensitivity etc.)? What are the rules for and mechanisms of synaptic plasticity? Does STDP account for all forms?

Microcircuits: How are the diverse receptive, response, and motion fields generated and what are their functions (cardinal cells or members of Hebbian assemblies? How and why are these responses modulated so extensively by cross-modal interactions, central states, self-generated movements, attention and reward expectation?

What are the specific functions emerging from characteristic features of circuitry (recurrency, inhibitory networks)? Are the resulting dynamics functionally relevant or dysfunctional epiphenomena (oscillations in distinct frequency bands, correlations, synchrony, phase shifts, self-generated spatio-temporal patterns)?

How is “activation” or “inactivation” defined in EEG, MEG, ECoG and fMRI signals, and how are variations in these signals related to underlying network activities?

How are memories (short- and long- term) formed, encoded, and read out? How are priors stored in cortical networks and how are they compared with input signals? What is propagated: error signals, signals matching with expectations (priors), or both, and if so via the same or different pathways?

Interareal interactions: Which are appropriate tools for the analysis of network interactions (e.g. coherence analysis, measures of Granger causality, dynamic causal modeling, multivariate techniques) and which are the relevant temporal and spatial scales (correlated BOLD fluctuations, phase locking and cross frequency coupling of EEG/MEG/ECoG signals, spike–field correlations, spike-spike correlations)? Are functional networks dynamically configured or just reflecting the backbone of fixed anatomical connections?

Is information contained in resting state activity? Can we construct an inventory of computational primitives that hold across different areas (local vs. generic)?

How are distributed representations of sensory objects configured and translated into executive commands? How can arbitrary correspondence between sensory and motor maps be established? How is top-down selection of inputs initiated and realized (e.g., feature-specific attentional selection)?

How can the system distinguish between activity arising from computations toward a result and activity representing a result? What is the signature of consistent activation states that lead to “eureka” experiences, trigger activation of reward systems, and activate now print commands for memory formation?

Are there computational principles beyond those considered in classical neuronal network theory that capitalize on the properties of complex, self-organizing dynamic systems with non-linear dynamics?

---

Group 4: Complexity and computation in human cognition

The neurobiological and functional properties of cerebral cortex, as discussed and debated in detail in the other groups, must be accountable, in the end, to the attributes of human perception, cognition, and affect. The terrific progress at the implementational level of description must “align” with the complex and dynamic features that form the basis for human thought and action. Can we identify linking hypotheses that allow us to go beyond mere correlational descriptions of how cortex underpins the suite of functions comprising cognition, affect and action?

Building on that broader question, what are the computational parts that form the basis for human experience? How might we account for compelling psychological phenomena such as drawing one’s “theory of mind,” the ability to plan/hallucinate/structure the future (mental time travel), or the sense of agency and volition? Can we make paradigm-shifting progress on our understanding of consciousness, binding, reportability?

Can a deeper understanding of processing sequences inform our mechanistic explanations of predictive coding, Bayesian hypotheses, abstract structures such as syntax, and, more broadly, illuminate the temporal structure of perceptual and cognitive experience? How do we store and retrieve the information that lies at the very basis of who we are (self) and what we do (remember facts, process words)?

---

This Forum is supported by the Deutsche Forschungsgemeinschaft

The German Research Foundation