95
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: not found

      Rational regulation of learning dynamics by pupil–linked arousal systems

      research-article

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The ability to make inferences about the current state of a dynamic process requires ongoing assessments of the stability and reliability of data generated by that process. We found that these assessments, as defined by a normative model, were reflected in non–luminance–mediated changes in pupil diameter of human subjects performing a predictive–inference task. Brief changes in pupil diameter reflected assessed instabilities in a process that generated noisy data. Baseline pupil diameter reflected the reliability with which recent data indicated the current state of the data–generating process and individual differences in expectations about the rate of instabilities. Together these pupil metrics predicted the influence of new data on subsequent inferences. Moreover, a task– and luminance–independent manipulation of pupil diameter predictably altered the influence of new data. Thus, pupil–linked arousal systems can help regulate the influence of incoming data on existing beliefs in a dynamic environment.

          Related collections

          Most cited references33

          • Record: found
          • Abstract: found
          • Article: not found

          Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function.

          An important dimension of cognitive control is the adaptive regulation of the balance between exploitation (pursuing known sources of reward) and exploration (seeking new ones) in response to changes in task utility. Recent studies have suggested that the locus coeruleus-norepinephrine system may play an important role in this function and that pupil diameter can be used to index locus coeruleus activity. On the basis of this, we reasoned that pupil diameter may correlate closely with control state and associated changes in behavior. Specifically, we predicted that increases in baseline pupil diameter would be associated with decreases in task utility and disengagement from the task (exploration), whereas reduced baseline diameter (but increases in task-evoked dilations) would be associated with task engagement (exploitation). Findings in three experiments were consistent with these predictions, suggesting that pupillometry may be useful as an index of both control state and, indirectly, locus coeruleus function.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment.

            Maintaining appropriate beliefs about variables needed for effective decision making can be difficult in a dynamic environment. One key issue is the amount of influence that unexpected outcomes should have on existing beliefs. In general, outcomes that are unexpected because of a fundamental change in the environment should carry more influence than outcomes that are unexpected because of persistent environmental stochasticity. Here we use a novel task to characterize how well human subjects follow these principles under a range of conditions. We show that the influence of an outcome depends on both the error made in predicting that outcome and the number of similar outcomes experienced previously. We also show that the exact nature of these tendencies varies considerably across subjects. Finally, we show that these patterns of behavior are consistent with a computationally simple reduction of an ideal-observer model. The model adjusts the influence of newly experienced outcomes according to ongoing estimates of uncertainty and the probability of a fundamental change in the process by which outcomes are generated. A prior that quantifies the expected frequency of such environmental changes accounts for individual variability, including a positive relationship between subjective certainty and the degree to which new information influences existing beliefs. The results suggest that the brain adaptively regulates the influence of decision outcomes on existing beliefs using straightforward updating rules that take into account both recent outcomes and prior expectations about higher-order environmental structure.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory.

              The adaptive regulation of the balance between exploitation and exploration is critical for the optimization of behavioral performance. Animal research and computational modeling have suggested that changes in exploitative versus exploratory control state in response to changes in task utility are mediated by the neuromodulatory locus coeruleus-norepinephrine (LC-NE) system. Recent studies have suggested that utility-driven changes in control state correlate with pupil diameter, and that pupil diameter can be used as an indirect marker of LC activity. We measured participants' pupil diameter while they performed a gambling task with a gradually changing payoff structure. Each choice in this task can be classified as exploitative or exploratory using a computational model of reinforcement learning. We examined the relationship between pupil diameter, task utility, and choice strategy (exploitation vs. exploration), and found that (i) exploratory choices were preceded by a larger baseline pupil diameter than exploitative choices; (ii) individual differences in baseline pupil diameter were predictive of an individual's tendency to explore; and (iii) changes in pupil diameter surrounding the transition between exploitative and exploratory choices correlated with changes in task utility. These findings provide novel evidence that pupil diameter correlates closely with control state, and are consistent with a role for the LC-NE system in the regulation of the exploration-exploitation trade-off in humans.
                Bookmark

                Author and article information

                Journal
                9809671
                21092
                Nat Neurosci
                Nat. Neurosci.
                Nature neuroscience
                1097-6256
                1546-1726
                9 May 2012
                03 June 2012
                01 January 2013
                : 15
                : 7
                : 1040-1046
                Affiliations
                [1 ]Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104
                [2 ]Department of Psychology, Princeton University, Princeton, New Jersey 08540
                Article
                NIHMS376105
                10.1038/nn.3130
                3386464
                22660479
                6c1b0dbb-bf38-4125-b15d-2ba9b1f69b69

                Users may view, print, copy, download and text and data- mine the content in such documents, for the purposes of academic research, subject always to the full Conditions of use: http://www.nature.com/authors/editorial_policies/license.html#terms

                History
                Funding
                Funded by: National Institute of Mental Health : NIMH
                Award ID: R21 MH093904 || MH
                Funded by: National Institute of Mental Health : NIMH
                Award ID: F31 MH093099 || MH
                Categories
                Article

                Neurosciences
                Neurosciences

                Comments

                Comment on this article