Caltech scientists decipher the neurological basis of timely movement

Contrary to what one might imagine, the way in which each of us
interacts with
the world is not a simple matter of seeing (or touching, or smelling)
and then
reacting. Even the best baseball hitter eyeing a fastball does not
swing at what
he sees. The neurons and neural connections that make up our sensory
systems are
far too slow for this to work. "Everything we sense is a little bit in
the
past," says Richard A. Andersen of the California Institute of
Technology, who
has now uncovered the trick the brain uses to get around this puzzling
problem.

Work by Andersen, the James G. Boswell Professor of Neuroscience at
Caltech, and
his colleagues Grant Mulliken of MIT and Sam Musallam of McGill
University,
offers the first neural evidence that voluntary limb movements are
guided by our
brain's prediction of what will happen an instant into the future.
"The brain is
generating its own version of the world, a 'forward model,' which
allows you to
know where you actually are in real time. It takes the delays out of
the
system," Andersen says.

The research in Andersen's laboratory is focused on understanding the
neurobiological underpinnings of brain processes, including the senses
of sight,
hearing, balance, and touch, and the neural mechanisms of action. The
lab is
working toward the development of implanted neural prosthetic devices
that would
serve as an interface between severely paralyzed individuals' brain
signals and
their artificial limbs--allowing thoughts to control movement.

Research along these lines conducted at the University of Pittsburgh
and
Carnegie Mellon University recently allowed monkeys to feed themselves
using a
robotic limb that they controlled only with their thoughts. Their
thoughts were
picked up via an array of electrodes sitting on top of the primary
motor cortex,
a lower level brain region responsible for carrying out motor
functions.

Andersen's group focuses on a more high-level area of cortex called
the
posterior parietal cortex (PPC), which is where sensory stimuli are
actually
transformed into movement plans.

In their experiments, Andersen and his colleagues trained two monkeys
to use a
joystick to move a cursor on a computer screen from a small red circle
into a
green circle, while keeping their gaze fixed on the red circle. The
monkeys
typically generated curved trajectories, but to increase the curvature
one
monkey was trained to move the cursor around an obstacle. The obstacle
(a large
blue circle) was placed between the initial location of the cursor and
the
target circle, and the monkey had to guide the cursor around the
obstacle,
without touching it, and over to the green circle. As the monkeys
conducted the
tasks, electrodes measured the activity of neurons in the PPC. This
allowed
Andersen and his colleagues to monitor signals--commands for movement--
in real
time.

The studies showed that neurons in the PPC produce signals that
represent the
brain's estimation of the current and upcoming movement of the cursor.
"An
internal estimate of the current state of the cursor can be used
immediately by
the brain to rapidly correct a movement, avoiding having to rely
entirely on
late-arriving sensory information, which can result in slow and
unstable
control," Mulliken says.

"The idea is that you feed back the command you make for movement into
those
areas of the brain that plan the movement (i.e., the PPC)," Andersen
says. "The
signal about the movement taking place is adjusted to be perfectly
aligned in
time with the actual movement--what you're moving in your head matches
with what
you're moving in the real world." The effect is akin to an athlete
visualizing
his performance in his mind. Studies have previously shown that these
simulations of movement trajectories run through the posterior
parietal cortex,
and run at actual speed, taking the same amount of time as the
activity would in
real life.

In the Pittsburgh robotic arm study, the neural signal driving the
robotic limb
was what is known as a "trajectory signal," which represents the path
that must
be taken to move from one point to another, like using a computer
mouse to drag
an object across a screen. Previously Andersen's lab had shown that a
different
signal in the posterior parietal cortex, called the "goal signal," can
also be
used to directly jump an object from one point to another.

"This goal signal is much faster for reaching a goal than a trajectory
signal,"
Andersen says. "Fast goal decoding is very advantageous for rapid
sequences such
as typing. Our new study shows that the posterior parietal cortex
codes the
trajectory as well as the goal, which makes this brain area an
attractive target
for neural prosthesis. Not only does this increase the versatility and
the
number of prosthetic applications, but it also makes the decoding
easier since
the trajectories can be better estimated if the goal is known."

The paper, "Forward Estimation of Movement State in Posterior Parietal
Cortex,"
will be published in a future print issue the Proceedings of the
National
Academy of Sciences but is now available online. First author, Grant
Mulliken,
was a graduate student at Caltech and is now a postdoctoral fellow at
the
Massachusetts Institute of Technology; coauthor Sam Musallam was a
postdoctoral
fellow at Caltech and is currently an assistant professor at McGill
University
in Montreal, Canada.

Source: California Institute of Technology
http://www.physorg.com/printnews.php?newsid=131970390

Here is a related article:

---------------

NYT
June 10, 2008
Mind
Anticipating the Future to ‘See’ the Present
By BENEDICT CAREY

Staring at a pattern meant to evoke an optical illusion is usually an
act of idle curiosity, akin to palm reading or astrology. The dot
disappears, or it doesn’t. The silhouette of the dancer spins
clockwise or counterclockwise. The three-dimensional face materializes
or not, and the explanation always seems to have something to do with
the eye or creativity or even personality.

That’s the usual cue to nod and feign renewed absorption in the
pattern.

In fact, scientists have investigated such illusions for hundreds of
years, looking for clues to how the brain constructs a seamless whole
from the bouncing kaleidoscope of light coming through the eyes. Brain
researchers today call the illusions perceptual, not optical, because
the entire visual system is involved, and their theories about what is
occurring can sound as exotic as anyone’s.

In the current issue of the journal Cognitive Science, researchers at
the California Institute of Technology and the University of Sussex
argue that the brain’s adaptive ability to see into the near future
creates many common illusions.

“It takes time for the brain to process visual information, so it has
to anticipate the future to perceive the present,” said Mark Changizi,
the lead author of the paper, who is now at Rensselaer Polytechnic
Institute. “One common functional mechanism can explain many of these
seemingly unrelated illusions.” His co-authors were Andrew Hsieh, Romi
Nijhawan, Ryota Kanai and Shinsuke Shimojo.

One fundamental debate in visual research is whether the brain uses a
bag of ad hoc tricks to build a streaming model of the world, or a
general principle, like filling in disjointed images based on
inference from new evidence and past experience. The answer may be
both. But perceptual illusions provide a keyhole to glimpse the
system.

When shown two images in quick succession, one of a dot on the left of
a screen and one with the dot on the right, the brain sees motion from
left to right, even though there was none. The visual system has
apparently constructed the scenario after it has been perceived,
reconciling the jagged images by imputing motion.

In an experiment originated by Dr. Nijhawan, people watch an object
pass a flashbulb. The timing is exact: the bulb flashes precisely as
the object passes. But people perceive that the object has moved past
the bulb before it flashes. Scientists argue that the brain has
evolved to see a split second into the future when it perceives
motion. Because it takes the brain at least a tenth of a second to
model visual information, it is working with old information. By
modeling the future during movement, it is “seeing” the present.

Dr. Changizi and his colleagues hold that it is a general principle
the brain applies to a wide variety of illusions that trick the brain
into sensing motion.

“It’s likely that there are many different neural mechanisms involved
in perceptual illusions,” said Jacob Feldman, a Rutgers psychologist.
“But the idea that there may be some overarching explanation that
accounts for these separate mechanisms is compelling and satisfying to
some scientists.”

Timothy Hubbard, a psychologist at Texas Christian University, said
the principle of perceiving the present was sound, adding, “If a
person’s response to an object, to catch, hit, block, whatever, is to
be optimal, that response should be calibrated to where the object
would be”— not a split second earlier, when the perception occurred.

This is why identical squares arranged around the center of a spoked-
wheel image appear misshapen, said Dr. Changizi, who writes about it
in a book due in 2009, “The Vision Revolution.” The sides of squares
closer to the center appear to bulge. The sides farther out appear
shorter. The radiating lines in the pattern trick the brain into
perceiving motion forward, so it projects objects forward, making
those nearer the center appear closer to the eye.

The same effect can be seen by leaning forward toward a precise
checkerboard. The image seems to bulge forward, this time because the
eyes are moving.

Dr. Changizi says such illusions can also occur in real life. When a
golf ball or baseball rolls through the grass and suddenly drops into
a hole, the brain sometimes perceives a trace of the ball on the other
side of the hole.

“But these are things that we don’t experience very often,” he said,
“because the brain is so good at covering up its mistakes.”