The role of reward and cognitive control in decision making

Here’s an exchange of emails between PL and MC on a recently published paper (Balleine et al., 2007).

Email 1 (from PL):
Have a look at this introductory paragraph from a recent (Aug 2007) J Neurosci article by Balleine, Delgado and Hikosaka. What do they mean by “cognition” here?

The Role of the Dorsal Striatum in Reward and Decision-Making
To choose appropriately between distinct courses of action requires the ability to integrate an estimate of the causal relationship between an action and its consequences, or outcome, with the value, or utility, of the outcome. Any attempt to base decision-making solely on cognition fails fully to determine action selection because any information, such as “action A leads to outcome O,” can be used both to perform A and to avoid performing A. It is interesting to note in this context that, although there is an extensive literature linking the cognitive control of executive functions specifically to the prefrontal cortex (Goldman-Rakic, 1995; Fuster, 2000), more recent studies suggest that these functions depend on reward-related circuitry linking prefrontal, premotor, and sensorimotor cortices with the striatum (Chang et al., 2002; Lauwereyns et al., 2002; Tanaka et al.,2006).


Email 2 (from MC):

It sounds like they are distinguishing cognition from reward processing. I’m not sure why, since ‘cognition’ typically encompasses reward processing now days.

The distinction I think they’re really trying to make is between cognitive control and reward processing. Given that, it’s still a ridiculous paragraph. Why must it be either cognitive control or reward processing? It’s likely (no, virtually certain!) that the two interact during reward-based decision making. For instance, O’Reilly’s stuff shows how this might happen.

Another problem with this paragraph: They equate causal knowledge with cognitive control. Well-known causal knowledge doesn’t involve cognitive control! For instance, routine decision making would involve lower perceptuo-motor circuits, and if it involved differential reward then reward circuits would be engaged as well. Cognitive control has little/no role here.

When cognitive control is involved it’s probably doing a lot more than just retrieving causal relations from semantic memory. For instance, perceptual decision making studies show that cognitive control is involved even in deciding what is being perceived when uncertainty arises.

I guess what they’re trying to do is show that cognitive control doesn’t explain all of decision making since there must be a reward component as well. Perhaps this is a good point to make; they just didn’t do it well.


Email 3 (from PL):

Ahhh, ok I think I see now what they’re trying to say.  It really just struck me as an excessively divisive statement to start out what appeared to be an interesting article.  Can you say “flamebait”?  Perhaps they’re trying to be provocative.

– PL & MC

Magnetoencephalography

In the dark confines behind our eyes lies flesh full of mysterious patterns, constituting our hopes, desires, knowledge, and everything else fundamental to who we are. Since at least the time of Hippocrates we have wondered about the nature of this flesh and its functions. Finally, after thousands of years of wondering we are now able to observe the mysterious patterns of the living brain, with the help of neuroimaging.

First, electroencephalography (EEG) showed us that these brain patterns have some relation in time to our behaviors. EEG showed us when things happen in the brain. More recent technologies such as functional magnetic resonance imaging (fMRI) then showed us where things happen in the brain.

It has been suggested that true insights into these brain patterns will arise when we can understand the patterns’ complex spatio-temporal nature. Thus, only with sufficient spatial and temporal resolution will we be able to decipher the mechanisms behind the brain patterns, and as a result the mechanisms behind ourselves.

Magnetoencephalography (MEG) may help to provide such insight. This method uses superconducting sensors to detect subtle changes in the magnetic fields surrounding the head. These changes reflect the patterns of neural activity as they occur in the brain. Unlike fMRI (and similar methods), MEG can measure neural activity at a very high temporal resolution (>1 kHz). In this respect it is similar to EEG. However, unlike EEG, MEG patterns are not distorted by the skull and scalp, thus providing an unprecedented level of spatio-temporal resolution for observing the neural activity underlying our selves.

Despite being around for several decades, new advances in the technology are providing unprecedented abilities to observe brain activity. Of course, the method is not perfect by any means. As always, it is a method complimentary to others, and should be used in conjunction with other noninvasive (and the occasionally invasive, where appropriate) neuroimaging methods.

MEG relies on something called a superconducting quantum interference device (SQUID). Many of these SQUIDs are built into a helmet, which is cooled with liquid helium and placed around the head. Extremely small magnetic fields created by neural activity can then be detected with these SQUIDs and recorded to a computer for later analysis.

I recently got back from a trip to Finland, where I learned  a great deal about MEG. I’m planning to use the method to observe the flow of information among brain regions during cognitive control tasks involving decision making, learning, and memory. I’m sure news of my work in this area will eventually make it onto this website.

-MC

Redefining Mirror Neurons

Monkey imitating humanIn 1992 Rizzolatti and his colleagues found a special kind of neuron in the premotor cortex of monkeys (Di Pellegrino et al., 1992).

These neurons, which respond to perceiving an action whether it’s performed by the observed monkey or a different monkey (or person) it’s watching, are called mirror neurons

Many neuroscientists, such as V. S. Ramachandran, have seized upon mirror neurons as a potential explanatory ‘holy grail’ of human capabilities such as imitation, empathy, and language. However, to date there are no adequate models explaining exactly how such neurons would provide such amazing capabilities.

Perhaps related to the lack of any clear functional model, mirror neurons have another major problem: Their functional definition is too broad.

Typically, mirror neurons are defined as cells that respond selectively to an action both when the subject performs it and when that subject observes another performing it. A basic assumption is that any such neuron reflects a correspondence between self and other, and that such a correspondence can turn an observation into imitation (or empathy, or language).

However, there are several other reasons a neuron might respond both when an action is performed and observed.

First, there may be an abstract concept (e.g., open hand), which is involved in but not necessary for the action, the observation of the action, or any potential imitation of the action.

Next, there may be a purely sensory representation (e.g., of hands / objects opening) which becomes involved independently of action by an agent.

Finally, a neuron may respond to another subject’s action not because it is performing a mapping between self and other but because the other’s action is a cue to load up the same action plan. In this case the ‘mirror’ mapping is performed by another set of neurons, and this neuron is simply reflecting the action plan, regardless of where the idea to load that plan originated. For instance, a tasty piece of food may cause that neuron to fire because the same motor plan is loaded in anticipation of grasping it.

It is clear that mirror neurons, of the type first described by Rizzolati et al., exist (how else could imitation occur?). However, the practical definition for these neurons is too broad.

How might we improve the definition of mirror neurons? Possibly by verifying that a given cell (or population of cells) responds only while observing a given action and while carrying out that same action.

Alternatively, subtractive methods may be more effective at defining mirror neurons than response properties. For instance, removing a mirror neuron population should make imitation less accurate or impossible. Using this kind of method avoids the possibility that a neuron could respond like a mirror neuron but not actually contribute to behavior thought to depend on mirror neurons.

Of course, the best approach would involve both observing response properties and using controlled lesions. Even better would be to do this with human mirror neurons using less invasive techniques (e.g., fMRI, MEG, TMS), since we are ultimately interested in how mirror neurons contribute to higher-level behaviors most developed in homo sapiens, such as imitation, empathy, and language.

-MC

Image from The Phineas Gage Fan Club (originally from Ferrari et al. (2003)).

History’s Top Brain Computation Insights: Hippocampus binds features

Hippocampus is involved in feature binding for novel stimuli (McClelland, McNaughton, & O'Reilly – 1995, Knight – 1996, Hasselmo – 2001, Ranganath & D'Esposito – 2001)

It was demonstrated by McClelland et al.that, based on its role in episodic memory encoding, hippocampus can learn fast arbitrary association.

This was in contrast to neocortex, which they showed learns slowly in order to develop better generalizations (knowledge not tied to a single episode). This theory was able to explain why patient H.M. knew (for example) about JFK's assassination even though he lost his hippocampus in the 1950s.

Robert Knight provided evidence for a special place for novelty in hippocampal function by showing a different electrical response to novel stimuli in patients with hippocampal damage.

These two findings together suggested that the hippocampus may be important for binding the features of novel stimuli, even over short periods.

This was finally verified by Hasselmo et al. and Ranganath & D'Esposito in 2001. They used functional MRI to show that a portion of the hippocampal formation is more active during working memory delays when novel stimuli are used.

This suggests that hippocampus is not just important for long term memory. Instead, it is important for short term memory and perhaps novel perceptual binding in general.

Some recent evidence suggests that hippocampus may be important for imagining the future, possibly because binding of novel features is necessary to create a world that does not yet exist (for review see Schacter et al. 2007).

-MC

Two Universes, Same Structure

Image of galaxiesThis image is not of a neuron.

This image is of the other universe; the one outside our heads.

It depicts the “evolution of the matter distribution in a cubic region of the Universe over 2 billion light-years”, as computed by the Millennium Simulation. (Click the image above for a better view.)

The next image, of a neuron, is included for comparison.

Image of a neuron

It is tempting to wax philosophical on this structure equivalence. How is it that both the external and internal universes can have such similar structure, and at such vastly different physical scales?

If we choose to go philosophical, we may as well ponder something even more fundamental: Why is it that all complex systems seem to have a similar underlying network-like structure?

Read more

History’s Top Insights Into Brain Computation

This post is the culmination of a month-long chronicling of the major brain computation insights of all time.

Some important insights were certainly left out, so feel free to add comments with your favorites.

Below you will find all 26 insights listed with links to their entries. At the end is the summary of the insights in two (lengthy) sentences.

1) The brain computes the mind (Hippocrates- 460-379 B.C.)

2)  Brain signals are electrical (Galvani – 1791, Rolando – 1809)

3)  Functions are distributed in the brain (Flourens – 1824, Lashley – 1929)

4) Functions can be localized in the brain (Bouillaud – 1825, Broca – 1861, Fritsch & Hitzig – 1870)

5) Neurons are fundamental units of brain computation (Ramon y Cajal – 1889)

6) Neural networks consist of excitatory and inhibitory neurons connected by synapses (Sherrington – 1906)

7) Brain signals are chemical (Dale – 1914, Loewi – 1921)

8) Reward-based reinforcement learning can explain much of behavior (Skinner – 1938, Thorndike – 1911, Pavlov – 1905)

9) Convergence and divergence between layers of neural units can perform abstract computations (Pitts & McCulloch – 1947)

10) The Hebbian learning rule: ‘Neurons that fire together wire together’ [plus corollaries] (Hebb, 1949)

11) Action potentials, the electrical events underlying brain communication, are governed by ion concentrations and voltage differences mediated by ion channels (Hodgkin & Huxley – 1952)

12) Hippocampus is necessary for episodic memory formation (Milner – 1953)

13) Larger cortical space is correlated with greater representational resolution; memories are stored in cortex (Penfield – 1957)

14) Neocortex is composed of columnar functional units (Mountcastle – 1957, Hubel & Wiesel – 1962)

15) Consciousness depends on cortical communication; the cortical hemispheres are functionally specialized (Sperry & Gazzaniga – 1969)

16) Critical periods of cortical development via competition (Hubel & Wiesel – 1970)

17) Reverbatory activity in lateral prefrontal cortex maintains memories and attention over short periods (Fuster – 1971, Jacobsen – 1936, Goldman-Rakic – 2000)

18) Behavior exists on a continuum between controlled and automatic processing (Schneider & Shiffrin – 1977)

19) Neural networks can self-organize via competition (Grossberg – 1978, Kohonen – 1981)

20) Spike-timing dependent plasticity: Getting the brain from correlation to causation (Levy – 1983, Sakmann – 1994, Bi & Poo – 1998, Dan – 2002)

21) Parallel and distributed processing across many neuron-like units can lead to complex behaviors (Rumelhart & McClelland – 1986, O’Reilly – 1996)

22) Recurrent connectivity in neural networks can elicit learning and reproduction of temporal sequences (Jordan – 1986, Elman – 1990, Schneider – 1991)

23) Motor cortex is organized by movement direction (Schwartz  & Georgopoulos – 1986, Schwartz – 2001)

24) Cognitive control processes are distributed within a network of distinct regions (Goldman-Rakic – 1988, Posner – 1990, Wager & Smith – 2004, Dosenbach et al. – 2006, Cole & Schneider – 2007)

25) The dopamine system implements a reward prediction error algorithm (Schultz – 1996, Sutton – 1988)

26) Some complex object categories, such as faces, have dedicated areas of cortex for processing them, but are also represented in a distributed fashion (Kanwisher – 1997, Haxby – 2001)

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions forming specialized and/or overlapping distributed networks involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, dopamine signals for reinforcement learning, and recurrent connectivity for sequential learning.

-MC 

History’s Top Brain Computation Insights: Day 26

26) Some complex object categories, such as faces, have dedicated areas of cortex for processing them, but are also represented in a distributed fashion (Kanwisher – 1997, Haxby – 2001)

Early in her career Nancy Kanwisher used functional MRI (fMRI) to seek modules for perceptual and semantic processing. She was fortunate enough to discover what she termed the fusiform face area; an area of extrastriate cortex specialized for face perception.

This finding was immediately controversial. It was soon shown that other object categories also activate this area. Being the adept scientist that she is, Kanwisher showed that the area was nonetheless more active for faces than any other major object category.

Then came a slew of arguments purporting that the face area was in fact an ‘expertise area’. This hypothesis states that any visual category with sufficient expertise should activate the fusiform face area.

This argument is based on findings in cognitive psychology showing that many aspects of face perception once thought to be unique are in fact due to expertise (Diamond et al., 1986). Thus, a car can show many of the same perceptual effects as faces for a car expert. The jury is still out on this issue, but it appears that there is in fact a small area in the right fusiform gyrus dedicated to face perception (see Kanwisher’s evidence).

James Haxby entered the fray in 2001, showing that even after taking out the face area from his fMRI data he could predict the presence of faces based on distributed and overlapping activity patterns across visual cortex. Thus it was shown that face perception, like visual perception of other kinds of objects, is distributed across visual cortex.

Once again, Kanwisher stepped up to the plate.

Read more

History’s Top Brain Computation Insights: Day 25

Dopamine signal related to reward and reward prediction (Schultz, 1999)25) The dopamine system implements a reward prediction error algorithm (Schultz – 1996, Sutton – 1988)

It used to be that the main thing anyone "knew" about the dopamine system was that it is important for motor control.
  Parkinson's disease, which visibly manifests itself as motor tremors, is caused by disruption of the dopamine system (specifically, the substantia nigra), so this was an understandable conclusion.

When Wolfram Schultz began recording from dopamine neurons in mice and monkeys he was having trouble finding correlations with his motor task. Was he doing something wrong? Was he recording from the right cells?

Instead of towing the line of dopamine = motor control he set out to find out what this system really does. It turns out that it is related to reward.

Schultz observed dopamine cell bursting at the onset of unexpected reward. He also observed that this bursting shifts to a cue (e.g., a bell sound) indicating a reward is forthcoming. When the reward cue occurs but no reward follows he saw that the dopamine cells go silent (below resting firing rate).

This pattern is quite interesting computationally. The dopamine signal mimics the error signal in a form of reinforcement learning called temporal difference learning.

This form of learning was originally developed by Sutton. It is a powerful algorithm for learning to predict reward and learn from errors in attaining reward.

Temporal difference learning basically propagates reward prediction back in time as far as possible, thus facilitating the process of attaining reward in the future.

Figure: (Top) No conditioned stimulus cue is given, so the reward is unexpected and there is a big dopamine burst. (Middle) The animal learns to predict the reward based on the cue and the dopamine burst moves to the cue. (Bottom) The reward is predicted, but since no reward occurs there is a depression in dopamine release.
Source: Figure 2 of Schultz, 1999. (News in Physiological Sciences, Vol. 14, No. 6, 249-255, December 1999)

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions forming specialized networks involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, dopamine signals for reinforcement learning, and recurrent connectivity for sequential learning.

[This post is part of a series chronicling history's top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC

Fair Use and Legal Intimidation in Science

Shelley Batts’s Retrospectacle neuroscience blog recently got hit with a legal threat from journal publisher Wiley for posting some graphs from a recently published article.

This request amounts to underhanded legal intimidation as using these graphs clearly falls under fair use. Shelley had clearly cited the source of the graph and accurately reported the results.

What might be their motivation for the legal threat? According to this comment by Shelley: “I think perhaps what the real issue here is that they were afraid I might bust their ‘press bubble.’ This study has been used as a justification for ‘fruity alcoholic drinks are health food’ and the spin was so ubiquitious throughout news venues it obviously was released that way. The real results do not support that conclusion.”
Many across the blogosphere have chimed in to support Shelley, and I’d like to be counted as one of them.

Science is about openness and sharing knowledge for the higher good of scientific progress. It’s perhaps telling that the legal threats included the parent company’s slogan “SCI – where science meets business”. They’re apparently too caught up in the business side of things, where legal threats for knowledge sharing are the norm.

Update: The disagreement with Wiley has been resolved.
-MC 

History’s Top Brain Computation Insights: Day 24

Cognitive control network (Cole & Schneider, 2007)24) Cognitive control processes are distributed within a network of distinct regions (Goldman-Rakic – 1988, Posner – 1990, Wager & Smith 2004, Cole & Schneider – 2007)

Researchers investigating eye movements and attention recorded from different parts of the primate brain and found several regions showing very similar neural activity. Goldman-Rakic proposed the existence of a specialized network for the control of attention.

This cortical system consists of the lateral frontal cortex (fronto-polar, dorsolateral, frontal eye fields), medial frontal cortex (anterior cingulate, pre-SMA, supplementary eye fields), and posterior parietal cortex. Subcortically, dorsomedial thalamus and superior colliculus are involved, among others.

Many computational modelers emphasize the emergence of attention from the local organization of sensory cortex (e.g., local competition). However, when a shift in attention is task-driven (i.e., top-down) then it appears that a specialized system for attentional control drives activity in sensory cortex. Many properties of attention likely arise from the organization of sensory cortex, but empirical data indicate that this is not sufficient.

With the advent of neuroimaging in humans (PET and fMRI), Posner et al. found very similar regions as those reported by Goldman-Rakic. He found that some regions are related more to orienting to stimuli, while others are related more to cognitive control (i.e., controlled processing).

After many fMRI studies of cognitive control were published, Wager et al. performed a meta-analysis looking at most of this research. They found a set of cortical regions active in nearly all cognitive control tasks.

My own work with Schneider (in press) indicates that these regions form an innate network, which is better connected than the rest of cortex on average. We used resting state correlations of fMRI BOLD activity to determine this. This cognitive control network is involved in controlled processing in that it has greater activity early in practice relative to late in practice, and has greater activity for conflicting responses (e.g., the Stroop task).

Though these regions have similar responses, they are not redundant. Our study showed that lateral prefrontal cortex is involved in maintaining relevant task information, while medial prefrontal cortex is involved in preparing and making response decisions. In most cases these two cognitive demands are invoked at the same time; only by separating them in time were we able to show specialization within the cognitive control network. We expect that other regional specializations will be found with more work.

I’ll be covering my latest study in more detail once it is published (it has been accepted for publication at NeuroImage and should be published soon). The above figure is from that publication. It lists the six regions within the human cognitive control network. These regions include dorsolateral prefrontal cortex (DLPFC), inferior frontal junction (IFJ), dorsal pre-motor cortex (dPMC), anterior cingulate / pre-supplementary motor area (ACC/pSMA), anterior insula cortex (AIC), and posterior parietal cortex (PPC).

A general computational insight arising from this work (starting with Goldman-Rakic) is that cortex is composed of specialized regions that form specialized networks. This new paradigm for viewing brain function weds the old warring concepts of localized specialization and distributed function.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions forming specialized networks involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, and recurrent connectivity for sequential learning.

[This post is part of a series chronicling history’s top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC

History’s Top Brain Computation Insights: Day 23

23) Motor cortex is organized by movement direction (Schwartz  & Georgopoulos – 1986, Schwartz – 2001)

Penfield had shown that motor cortex is organized in a somatotopic map. However, it was unclear how individual neurons are organized. What does each neuron’s activity represent? The final location of a movement, or the direction of that movement?

Shwartz & Georgopoulos found that movement direction correlated best with neural activity. Further, they discovered that the neurons use a population code to specify a given movement. Thus, as illustrated above, a single neuron responds to a variety of movement direction but has one preferred direction (PD).

The preferred direction code across a large population of neurons thus sums to specify each movement.

Schwartz has since demonstrated how these population vectors can be interpreted by a computer to control a prosthetic arm. He has used this to imbue monkeys with Jedi powers; able to move a prosthetic limb in another room (or attached) with only a thought. Using this technology the Schwartz team hopes to allow amputee humans to control artificial limbs as they once did their own.

A general computational insight one might take from the Schwartz & Georgopoulos work is the possibility of population coding across cortex. It appears that all perception, semantics, and action may be coded as distributed population vectors.

Representational specificity within these vectors likely arises from conjunctions of receptive fields, and are dominated by those receptive fields most specific to each representation.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, and recurrent connectivity for sequential learning.

[This post is part of a series chronicling history’s top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC

History’s Top Brain Computation Insights: Day 22

22) Recurrent connectivity in neural networks can elicit learning and reproduction of temporal sequences (Jordan – 1986, Elman – 1990, Schneider – 1991)

Powerful learning algorithms such as Hebbian learning, self-organizing maps, and backpropagation of error illustrated how categorization and stimulus-response mapping might be learned in the brain. However, it remained unclear how sequences and timing discrimination might be learned.

In 1986 Michael Jordan (the computer scientist, not the basketball player) developed a network of neuron-like units that fed back upon itself. Jeff Elman expanded on this, showing how these recurrent networks can learn to recognize sequences of ordered stimuli.

Elman applied his recurrent networks to the problem of language perception. He concluded that language relies heavily on recurrent connectivity in cortex; an unproven but well-accepted statement among many scientists today.

The year after Elman's demonstration of sequence learning with language, Walter Schneider (Schneider & Oliver, 1991) used a recurrent network to implement what he termed a 'goal processor'. This network can learn arbitrary task sequences, effectively expanding recurrent networks beyond language learning to learning new tasks of any type.

See this article for a review of a model implementing a goal processor.

The goal processor has been likened to a part of neocortex (dorsolateral prefrontal cortex) shown to be involved in maintaining goal information in working memory. Also, this maintenance is believed to occur via local (and/or via long-range fronto-parietal connections) recurrent connectivity.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration and recurrent connectivity for sequential learning.

[This post is part of a series chronicling history's top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC

History’s Top Brain Computation Insights: Day 21

21) Parallel and distributed processing across many neuron-like units can lead to complex behaviors (Rumelhart & McClelland – 1986, O'Reilly – 1996)

Pitts & McCullochprovided amazing insight into how brain computations take place. However, their two-layer perceptrons were limited. For instance, they could not implement the logic gate XOR (i.e., 'one but not both'). An extra layer was added to solve this problem, but it became clear that the Pitts & McCulloch perceptrons could not learn anything requiring more than two layers.

Rumelhart solved this problem with two insights.

First, he implemented a non-linear sigmoid function (approximating a neuronal threshold), which turned out to be essential for the next insight.

Second, he developed an algorithm called 'backpropagation of error', which allows the output layer to propagate its error back across all the layers such that the error can be corrected in a distributed fashion. See P.L.'s previous post on the topic for further details.

Rumelhart & McClelland used this new learning algorithm to explore how cognition might be implemented in a parallel and distributed fashion in neuron-like units. Many of their insights are documented in the two-volume PDP series.

Unfortunately, the backpropagation of error algorithm is not very biologically plausible.  Signals have never been shown to flow backward across synapses in the manner necessary for this algorithm to be implemented in actual neural tissue.

However, O'Reilly (whose thesis advisor was McClelland) expanded on Hinton & McClelland (1988) to implement a biologically plausible version of backpropagation of error. This is called the generalized recirculation algorithm, and is based on the contrastive-Hebbian learning algorithm.

O'Reilly and McClelland view the backpropagating error signal as the difference between the expected outcome and the perceived outcome. Under this interpretation these algorithms are quite general, applying to perception as well as action.

The backprop and generalized recirculation algorithms are described in a clear and detailed manner in  Computational Explorations in Cognitive Neuroscience by O'Reilly & Munakata. These algorithms can be explored by downloading the simulations accompanying the book (available for free).

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional communication for functional integration.

[This post is part of a series chronicling history's top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]
-MC

History’s Top Brain Computation Insights: Day 20

Spike-timing dependent plasticity (Bi & Poo 1998; Markram et al. 1997)20) Spike-timing dependent plasticity: Getting the brain from correlation to causation (Levy – 1983, Sakmann – 1994, Bi & Poo – 1998, Dan – 2002)

Hebb's original proposal was worded as such: "When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased." [emphasis added]

The phrase "takes part in firing" implies causation of B's activity via A's activity, not simply a correlation of the two.

There are several ways to go beyond correlation to infer causation. One method is to observe that one event (e.g., cell A's activity) comes just before the caused event (e.g., cell B's activity).

In 1983 Levy showed with hippocampal slices that electrically stimulating cell A to fire before cell B will cause long-lasting strengthening of the synapse from cell A to cell B.  However, when the opposite occurs, and cell A is made to fire after cell B, there is depotentiation of the same synapse. In other words, timing is essential for synaptic learning. Today, this form of learning is called spike-timing dependent plasticity (STDP).

Using this rule, Levy has created a variety of neural network models aimed at understanding  memory in the brain (e.g., especially in the hippocampus; see this paper for a short review). 

More recently, other researchers including Sakmann, Bi, Poo, and Dan have further characterized this phenomenon. They showed that it occurs in vivo, within a specific time window (~8 msec timing difference is optimal), in neocortex, and (using behavioral evidence) in humans.

Figure caption: A) Figure from Bi & Poo (1998) showing the effects of STDP in potentiation and depotentiation with optimal results ~8-10ms in either direction. B) Figure from Markram et al. (1997) showing the timing of the stimulation relative to the post-synaptic cell's EPSP. C) Another figure from Markram et al. (1997) showing the resulting long-term changes in synaptic efficacy due to the manipulations in figure B.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections strengthened by timing-dependent correlated activity. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g.,short-term: prefrontal, long-term: temporal),which depend on inter-regional communication for functional integration.

[This post is part of a series chronicling history's top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC & PL

History’s Top Brain Computation Insights: Day 19

Center-surround organization used in SOMs19) Neural networks can self-organize via competition (Grossberg – 1978, Kohonen – 1981)

Hubel and Wiesel's work with the development  of cortical columns (see previous post) hinted at it, but it wasn't until Grossberg and Kohonen built computational architectures explicitly exploring competition that its importance was made clear.

Grossberg was the first to illustrate the possibility of self-organization via competition. Several years later Kohonen created what is now termed a Kohonen network, or self-organizing map (SOM). This kind of network is composed of layers of neuron-like units connected with local excitation and, just outside that excitation, local inhibition. The above figure illustrates this 'Mexican hat' function in three dimensions, while the figure below represents it in two dimensions along with its inputs.

These networks, which implement Hebbian learning, will spontaneously organize into topographic maps.

For instance, line orientations that are similar to each other will tend to be represented by nearby neural units, while less similar line orientations will tend to be represented by more distant neural units. This occurs even when the map starts out with random synaptic weights. Also, this spontaneous organization will occur for even very complex stimuli (e.g., faces) as long as there are spatio-temporal regularities in the inputs.

Another interesting feature of Kohonen networks is that the more frequent input patterns are represented by larger areas in the map. This is consistent with findings in cortex, where more frequently used representations have larger cortical areas dedicated to them.

There are several computational advantages to having local competition between similar stimuli, which SOMs can provide.

One such advantage is that local competition can increase specificity of the representation by ruling out close alternatives via lateral inhibition. Using this computational trick, the retina can discern visual details better at the edges of objects (due to contrast enhancement).

Another computational advantage is enhancement of what's behaviorally important relative to what isn't. This works on a short time-scale with attention (what's not important is inhibited), and on a longer time-scale with increases in representational space in the map with repeated use, which increases representational resolution (e.g., the hand representation in the somatosensory homonculus).

You can explore SOMs using Topographica, a computational modeling environment for simulating topographic maps in cortex. Of special interest here is the SOM tutorial available at topographica.org.


Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections strengthened by correlated activity. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g.,short-term: prefrontal, long-term: temporal),which depend on inter-regional communication for functional integration.

[This post is part of a series chronicling history's top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

-MC