BRAIN REGIONS THAT ENCODE WORDS, GRAMMAR, STORY IDENTIFIED
Some people say that
reading "Harry Potter and the Sorcerer's Stone" taught them the
importance of friends, or that easy decisions are seldom right. Carnegie Mellon
University scientists used a chapter of that book to learn a different lesson:
identifying what different regions of the brain are doing when people read.
Researchers from CMU's
Machine Learning Department performed functional magnetic resonance imaging
(fMRI) scans of eight people as they read a chapter of that Potter book. They
then analyzed the scans, cubic millimeter by cubic millimeter, for every
four-word segment of that chapter. The result was the first integrated
computational model of reading, identifying which parts of the brain are
responsible for such subprocesses as parsing sentences, determining the meaning
of words and understanding relationships between characters.
As Leila Wehbe, a
Ph.D. student in the Machine Learning Department, and Tom Mitchell, the
department head, recently reported in the online journal PLOS ONE,
the model was able to predict fMRI activity for novel text passages with
sufficient accuracy to tell which of two different passages a person was
reading with 74 percent accuracy.
"At first, we
were skeptical of whether this would work at all," Mitchell said, noting
that analyzing multiple subprocesses of the brain at the same time is
unprecedented in cognitive neuroscience. "But it turned out amazingly well
and now we have these wonderful brain maps that describe where in the brain
you're thinking about a wide variety of things."
Wehbe and Mitchell
said the model is still inexact, but might someday be useful in studying and
diagnosing reading disorders, such as dyslexia, or to track the recovery of
patients whose speech was impacted by a stroke. It also might be used by
educators to identify what might be giving a student trouble when learning a
foreign language.
"If I'm having
trouble learning a new language, I may have a hard time figuring out exactly
what I don't get," Mitchell said. "When I can't understand a
sentence, I can't articulate what it is I don't understand. But a brain scan
might show that the region of my brain responsible for grammar isn't activating
properly, or perhaps instead I'm not understanding the individual words."
Researchers at
Carnegie Mellon and elsewhere have used fMRI scans to identify activation
patterns associated with particular words or phrases or even emotions. But
these have always been tightly controlled experiments, with only one variable
analyzed at a time. The experiments were unnatural, usually involving only
single words or phrases, but the slow pace of fMRI -- one scan every two
seconds -- made other approaches seem unfeasible.
Wehbe nevertheless was
convinced that multiple cognitive subprocesses could be studied simultaneously
while people read a compelling story in a near-normal manner. She believed that
using a real text passage as an experimental stimulus would provide a rich
sample of the different word properties, which could help to reveal which brain
regions are associated with these different properties.
"No one falls
asleep in the scanner during Leila's experiments," Mitchell said.
They devised a
technique in which people see one word of a passage every half second -- or
four words for every two-second fMRI scan. For each word, they identified 195
detailed features -- everything from the number of letters in the word to its
part of speech. They then used a machine learning algorithm to analyze the
activation of each cubic centimeter of the brain for each four-word segment.
Bit by bit, the
algorithm was able to associate certain features with certain regions of the
brain, Wehbe said.
"The test
subjects read Chapter 9 of Sorcerer's Stone, which is about Harry's first
flying lesson," she noted. "It turns out that movement of the
characters -- such as when they are flying their brooms -- is associated with
activation in the same brain region that we use to perceive other people's
motion. Similarly, the characters in the story are associated with activation
in the same brain region we use to process other people's intentions."
Exactly how the brain
creates these neural encodings is still a mystery, they said, but it is the
beginning of understanding what the brain is doing when a person reads.
"It's sort of
like a DNA fingerprint -- you may not understand all aspects of DNA's function,
but it guides you in understanding cell function or development," Mitchell
said. "This model of reading initially is that kind of a
fingerprint."
A complementary study
by Wehbe and Mitchell, presented earlier this fall at the Conference on
Empirical Methods in Natural Language Processing, used magnetoencephalography
(MEG) to record brain activity in subjects reading Harry Potter. MEG can record
activity every millisecond, rather than every two seconds as in fMRI scanning,
but can't localize activity with the precision of fMRI. Those findings suggest
how words are integrated into memory -- how the brain first visually perceives
a word and then begins accessing the properties of the word, and fitting it
into the story context.
Comments
Post a Comment