Cognitive Neuroscience Society

The Journal of Cognitive Neuroscience

  • Facebook
  • RSS
  • Twitter
MENUMENU
  • Home
  • Annual Meeting
        • General Information
          • What to Expect at CNS 2023
          • CNS 2023 Mobile App
          • CNS 2023 Giveaway
          • CNS 2023 Giveaway Winners
          • Accessibility at CNS
          • General Information
          • Code of Conduct
          • Dates and Deadlines
          • Hotel Reservations
          • Poster Guidelines
          • Poster Printing Discount
          • Annual Meeting Workshop Policy & Application
          • Exhibit with us!
        • Program
          • Thank you to our Partners
          • CNS 2023 Program Booklet
          • Schedule Overview
          • Program-at-a-Glance
          • CNS 30th Anniversary Dance Party
          • Keynote Address
          • George A. Miller Awardee
          • Distinguished Career Contributions Awardee
          • Young Investigator Awardees
          • CNS at 30: Perspectives on the Roots, Present, and Future of Cognitive Neuroscience
          • Invited-Symposium Sessions
          • Symposium Sessions
          • Data Blitz Session Schedule
          • Poster Schedule & Session Information
          • JoCN Travel Fellowship Award
          • GSA/PFA Award Winners
          • Workshops, Socials & Special Events
        • Registration
          • Registration
          • Registration FAQ
          • Registration Policies, Cancellations & Refunds
        • News/Press
          • CNS 2023 Press Room
          • CNS 2022 Blog
          • CNS 2021 Blog
          • CNS 2020 Blog
        • Submissions
          • 2023 Poster Printing Discount
          • Submission Requirements
          • Submit a Poster
          • Submit a Symposium
          • GSA or PFA Application
          • Data Blitz
          • Frequently Asked Submission Questions
        • Archive
          • CNS 2023 Conference Videos
          • CNS 2022 Conference Videos
          • CNS 2021 Conference Videos
          • CNS 2020 Conference Videos
          • CNS 2019 Conference Videos
          • CNS 2018 Conference Videos
          • CNS 2017 Conference Videos
          • CNS 2016 Conference Videos
          • CNS 2015 Conference Videos
          • Previous Meetings Programs & Abstracts
  • About CNS
    • Boards and Committees
    • CNS Statement: Black Lives Matter
  • Membership
    • Information and Benefits
    • Join or Renew Membership
    • Membership FAQs
    • Member Discounts
    • Newsletter
      • Submit an Announcement
      • Current Newsletter
      • Newsletter FAQs
      • Past Newsletters
  • Awards
    • George A. Miller Award
    • Fred Kavli Distinguished Career Contributions Award
    • Young Investigator Award
    • Young Investigator Award Nominations
    • 2023 YIA Nomination Form
    • JoCN Travel Fellowship Award
  • News Center
    • CNS Blog
    • CNS 2023 Press Room
    • CNS 2023 Blog
    • CNS 2022 Blog
    • CNS 2021 Blog
    • CNS 2020 Blog
    • CNS 2019 Blog
    • Blog Archives
    • Quick Tips for Getting Started on Twitter
    • Media Contact
  • My CNS
  • Contact Us
post

Revealing the Cognitive Sorcery of Human Intelligence

January 7, 2020

Q&A with Sam Gershman

In the last decade, computational techniques have expanded the toolkit for scientists across disciplines. In neuroscience, computational models are increasingly rendering “visible things that were previously invisible,” says Samuel Gershman, a cognitive neuroscientist at Harvard University.

“Computational modeling is not a niche activity. It’s the same theory-building activity in which all cognitive neuroscientists engage,” Gershman explains. “The difference is that computational models allow us to be very precise about our hypotheses, making them more falsifiable. The hypotheses also tend to be more sophisticated.”

For his computational work on how humans and animals learn to make long-term decisions, an area known as “reinforcement learning,” Gershman has received the CNS Young Investigator Award. “The long-term goal of my work is to understand how the brain produces cognition, and in particular all the cognitive sorcery that we identify as intelligence,” Gershman says.

We spoke with Gershman about this body of work, which he will present at the CNS 2020 meeting in Boston this March, as well as how he got started in the field, some myths surrounding dopamine, and next steps for his work.

CNS: Why do you personally study cognitive neuroscience, and reinforcement learning in particular? What got you interested in the topic?

Gershman: I got really interested in cognitive neuroscience after taking a class on it in my first year of college. It immediately captured my imagination ⁠— this idea that we could deconstruct our minds into underlying mechanisms realized in a physical system. I had been exposed to artificial intelligence and cognitive science for many years ⁠— my dad was an old-school AI researcher — and cognitive neuroscience seemed like an exciting way to pursue those old questions about intelligence.

I started working in research labs, bouncing around between many different topics ⁠— including episodic memory, attention, emotion regulation ⁠— before working on reinforcement learning in Nathaniel Daw’s lab at NYU. At first, I was hopelessly confused about reinforcement learning, like most topics in computational neuroscience. I had a sense that this was important and I should understand the math, but it took a long time before things started to make sense.

One of the remarkable confluences of modern cognitive neuroscience was the discovery that the same algorithms computer scientists use to solve these problems are also used by the brain.

CNS: So how do you like to describe reinforcement learning to people outside the field?

Gershman: Most of our consequential decisions ⁠— where to go to school, what assets to invest in, what job to take, when to start a family ⁠— require long-term planning; we have to think not just about a single action but also about sequences of actions with possibly distant consequences. These decisions are difficult because there is a branching tree of future possibilities. Reinforcement learning is the study of how to efficiently solve such problems. One of the remarkable confluences of modern cognitive neuroscience was the discovery that the same algorithms computer scientists use to solve these problems are also used by the brain.

CNS: What have been the biggest insights your body of computational work has brought to bear on understanding reinforcement learning?

Gershman: Before I started thinking about these problems, there were some very successful theoretical ideas about how the brain might implement reinforcement learning, most notably the idea that dopamine neurons report a “reward prediction error.” This error is the discrepancy between observed and expected reward ⁠— which can be used by the basal ganglia to update reward expectations and was usually thought to be encoded by neurons in the striatum. However, there were many indications that things were more complicated.

Just looking narrowly at dopamine, it was clear that dopamine neurons were sensitive to different forms of uncertainty, and also that dopamine was sensitive to things other than reward, like sensory attributes of stimuli. My contribution has been to develop a new generation of theories that make sense of these deviations from the classical theory, which in turn led to new experiments.

CNS: Can you give an example of this please?

Gershman: I showed how the sensitivity to sensory attributes could be explained as a form of “generalized prediction error” ⁠— the discrepancy between observed and expected sensory attributes ⁠— which turns out to be very useful computationally for solving the reinforcement learning problem. I collaborated with Geoff Schoenbaum at NIH to test predictions of this model; his lab showed that you could decode sensory information from ensembles of dopamine neurons, and that the information diminished over the course of learning, as you would expect from a generalized prediction error that gradually diminished.

CNS: We see dopamine in the news a lot. What do you find to be a common misconception about dopamine?

Gershman: The major misconception of dopamine, often called the “pleasure molecule,” is that it directly reports reward. But classic studies by Wolfram Schultz showed that dopamine neurons fire only in response to unexpected reward. If the reward is entirely predictable, dopamine neurons do not respond nearly as much.

CNS: Is there a single piece of data or study you are most excited to share in your CNS 2020 talk in March in Boston?

Gershman: I haven’t figured out yet exactly which data I’m going to show in my talk. Probably I will talk about the relationship between dopamine and belief states, which led to the finding that the dopamine response to reward can sometimes be a non-monotonic function of reward magnitude (Babayan et al., 2018), which was a highly non-trivial prediction of the theory.

CNS: What are the next steps for your work?

Gershman: I’m currently involved in several projects aiming to tie together different circuits ⁠— striatum, midbrain, hippocampus, prefrontal cortex ⁠— that play distinct roles in the brain’s reinforcement learning architecture.

CNS: Finally, what are you most looking forward to about the CNS meeting in Boston?

Gershman: I’m most looking forward to catching up with old friends.

 

-Lisa M.P. Munoz

 

By lmunoz Filed Under: Uncategorized Tagged With: cns 2020, decision-making, dopamine, learning, yia Leave a Comment

Previous article: Mapping the Brain’s Visual Behavior One Tidbit at a Time
Next article: Modeling Learning Across the Lifespan

Leave a Reply Cancel reply

You must be logged in to post a comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Recent Posts

  • Trainees Learn About Navigating a New World with a New Mindset
  • Mentorship Enables a High School Senior to Lead Sleep Research Study
  • CNS 2023: Day 4 Highlights
  • Psychedelics and Cognition: A New Look
  • CNS 2023: Day 3 Highlights

Blog Archives

Quick Tips for Getting Started on Twitter

Cognitive Neuroscience Society
c/o Center for Mind and Brain
267 Cousteau Place, Davis, CA 95618
916-955-6080: for CNS Membership Questions
805-450-7490: for annual meeting questions about- registration, posters, symposium
916-409-5069: Fax Line
email: meeting@cogneurosociety.org

Recent Posts

  • Trainees Learn About Navigating a New World with a New Mindset
  • Mentorship Enables a High School Senior to Lead Sleep Research Study
  • CNS 2023: Day 4 Highlights
  • Psychedelics and Cognition: A New Look
  • CNS 2023: Day 3 Highlights

Archives

Blog Archives

Previous Meeting Programs and Abstracts

Past Newsletters

All contents © Cognitive Neuroscience Society 1995-2019

Add to Calendar

Add to Calendar
04/16/2022 11:00 AM
04/16/2022 12:00 PM
America/Los_Angeles
How Prior Knowledge Shapes Encoding of New Memories
Description of the event
Grand Ballroom A
Create an Account

Login Utility