Approaches to Learning to Control Dynamic Uncertainty

In dynamic environments, when faced with a choice of which learning strategy to adopt, do people choose to mostly explore (maximizing their long term gains) or exploit (maximizing their short term gains)? More to the point, how does this choice of learning strategy influence one’s later ability to c...

Full description

Bibliographic Details
Main Authors: Magda Osman, Brian D. Glass, Zuzana Hola
Format: Article
Language:English
Published: MDPI AG 2015-10-01
Series:Systems
Subjects:
Online Access:http://www.mdpi.com/2079-8954/3/4/211
id doaj-42d8da87c24147c88a354449374fd293
record_format Article
spelling doaj-42d8da87c24147c88a354449374fd2932020-11-24T23:48:05ZengMDPI AGSystems2079-89542015-10-013421123610.3390/systems3040211systems3040211Approaches to Learning to Control Dynamic UncertaintyMagda Osman0Brian D. Glass1Zuzana Hola2Biological and Experimental Psychology Centre, School of Biological and Chemical Sciences, Queen Mary, University of London, London E1 4NS, UKDepartment of Computer Science, University College of London, London WC1E 6BT, UKBiological and Experimental Psychology Centre, School of Biological and Chemical Sciences, Queen Mary, University of London, London E1 4NS, UKIn dynamic environments, when faced with a choice of which learning strategy to adopt, do people choose to mostly explore (maximizing their long term gains) or exploit (maximizing their short term gains)? More to the point, how does this choice of learning strategy influence one’s later ability to control the environment? In the present study, we explore whether people’s self-reported learning strategies and levels of arousal (i.e., surprise, stress) correspond to performance measures of controlling a Highly Uncertain or Moderately Uncertain dynamic environment. Generally, self-reports suggest a preference for exploring the environment to begin with. After which, those in the Highly Uncertain environment generally indicated they exploited more than those in the Moderately Uncertain environment; this difference did not impact on performance on later tests of people’s ability to control the dynamic environment. Levels of arousal were also differentially associated with the uncertainty of the environment. Going beyond behavioral data, our model of dynamic decision-making revealed that, in actual fact, there was no difference in exploitation levels between those in the highly uncertain or moderately uncertain environments, but there were differences based on sensitivity to negative reinforcement. We consider the implications of our findings with respect to learning and strategic approaches to controlling dynamic uncertainty.http://www.mdpi.com/2079-8954/3/4/211dynamicdecision makingexplorationcomputational modeling
collection DOAJ
language English
format Article
sources DOAJ
author Magda Osman
Brian D. Glass
Zuzana Hola
spellingShingle Magda Osman
Brian D. Glass
Zuzana Hola
Approaches to Learning to Control Dynamic Uncertainty
Systems
dynamic
decision making
exploration
computational modeling
author_facet Magda Osman
Brian D. Glass
Zuzana Hola
author_sort Magda Osman
title Approaches to Learning to Control Dynamic Uncertainty
title_short Approaches to Learning to Control Dynamic Uncertainty
title_full Approaches to Learning to Control Dynamic Uncertainty
title_fullStr Approaches to Learning to Control Dynamic Uncertainty
title_full_unstemmed Approaches to Learning to Control Dynamic Uncertainty
title_sort approaches to learning to control dynamic uncertainty
publisher MDPI AG
series Systems
issn 2079-8954
publishDate 2015-10-01
description In dynamic environments, when faced with a choice of which learning strategy to adopt, do people choose to mostly explore (maximizing their long term gains) or exploit (maximizing their short term gains)? More to the point, how does this choice of learning strategy influence one’s later ability to control the environment? In the present study, we explore whether people’s self-reported learning strategies and levels of arousal (i.e., surprise, stress) correspond to performance measures of controlling a Highly Uncertain or Moderately Uncertain dynamic environment. Generally, self-reports suggest a preference for exploring the environment to begin with. After which, those in the Highly Uncertain environment generally indicated they exploited more than those in the Moderately Uncertain environment; this difference did not impact on performance on later tests of people’s ability to control the dynamic environment. Levels of arousal were also differentially associated with the uncertainty of the environment. Going beyond behavioral data, our model of dynamic decision-making revealed that, in actual fact, there was no difference in exploitation levels between those in the highly uncertain or moderately uncertain environments, but there were differences based on sensitivity to negative reinforcement. We consider the implications of our findings with respect to learning and strategic approaches to controlling dynamic uncertainty.
topic dynamic
decision making
exploration
computational modeling
url http://www.mdpi.com/2079-8954/3/4/211
work_keys_str_mv AT magdaosman approachestolearningtocontroldynamicuncertainty
AT briandglass approachestolearningtocontroldynamicuncertainty
AT zuzanahola approachestolearningtocontroldynamicuncertainty
_version_ 1725487389735911424