COCO-Search18 fixation dataset for predicting goal-directed attention control
Abstract Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control – saliency. We introduce COCO-Search18, the first dataset of laboratory...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Nature Publishing Group
2021-04-01
|
Series: | Scientific Reports |
Online Access: | https://doi.org/10.1038/s41598-021-87715-9 |
id |
doaj-2a4346091b884f6db7395c2b18c34b46 |
---|---|
record_format |
Article |
spelling |
doaj-2a4346091b884f6db7395c2b18c34b462021-04-25T11:34:31ZengNature Publishing GroupScientific Reports2045-23222021-04-0111111110.1038/s41598-021-87715-9COCO-Search18 fixation dataset for predicting goal-directed attention controlYupei Chen0Zhibo Yang1Seoyoung Ahn2Dimitris Samaras3Minh Hoai4Gregory Zelinsky5Department of Psychology, Stony Brook UniversityDepartment of Computer Science, Stony Brook UniversityDepartment of Psychology, Stony Brook UniversityDepartment of Computer Science, Stony Brook UniversityDepartment of Computer Science, Stony Brook UniversityDepartment of Psychology, Stony Brook UniversityAbstract Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control – saliency. We introduce COCO-Search18, the first dataset of laboratory-quality goal-directed behavior large enough to train deep-network models. We collected eye-movement behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding $$\sim$$ ∼ 300,000 search fixations. We thoroughly characterize COCO-Search18, and benchmark it using three machine-learning methods: a ResNet50 object detector, a ResNet50 trained on fixation-density maps, and an inverse-reinforcement-learning model trained on behavioral search scanpaths. Models were also trained/tested on images transformed to approximate a foveated retina, a fundamental biological constraint. These models, each having a different reliance on behavioral training, collectively comprise the new state-of-the-art in predicting goal-directed search fixations. Our expectation is that future work using COCO-Search18 will far surpass these initial efforts, finding applications in domains ranging from human-computer interactive systems that can anticipate a person’s intent and render assistance to the potentially early identification of attention-related clinical disorders (ADHD, PTSD, phobia) based on deviation from neurotypical fixation behavior.https://doi.org/10.1038/s41598-021-87715-9 |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Yupei Chen Zhibo Yang Seoyoung Ahn Dimitris Samaras Minh Hoai Gregory Zelinsky |
spellingShingle |
Yupei Chen Zhibo Yang Seoyoung Ahn Dimitris Samaras Minh Hoai Gregory Zelinsky COCO-Search18 fixation dataset for predicting goal-directed attention control Scientific Reports |
author_facet |
Yupei Chen Zhibo Yang Seoyoung Ahn Dimitris Samaras Minh Hoai Gregory Zelinsky |
author_sort |
Yupei Chen |
title |
COCO-Search18 fixation dataset for predicting goal-directed attention control |
title_short |
COCO-Search18 fixation dataset for predicting goal-directed attention control |
title_full |
COCO-Search18 fixation dataset for predicting goal-directed attention control |
title_fullStr |
COCO-Search18 fixation dataset for predicting goal-directed attention control |
title_full_unstemmed |
COCO-Search18 fixation dataset for predicting goal-directed attention control |
title_sort |
coco-search18 fixation dataset for predicting goal-directed attention control |
publisher |
Nature Publishing Group |
series |
Scientific Reports |
issn |
2045-2322 |
publishDate |
2021-04-01 |
description |
Abstract Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control – saliency. We introduce COCO-Search18, the first dataset of laboratory-quality goal-directed behavior large enough to train deep-network models. We collected eye-movement behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding $$\sim$$ ∼ 300,000 search fixations. We thoroughly characterize COCO-Search18, and benchmark it using three machine-learning methods: a ResNet50 object detector, a ResNet50 trained on fixation-density maps, and an inverse-reinforcement-learning model trained on behavioral search scanpaths. Models were also trained/tested on images transformed to approximate a foveated retina, a fundamental biological constraint. These models, each having a different reliance on behavioral training, collectively comprise the new state-of-the-art in predicting goal-directed search fixations. Our expectation is that future work using COCO-Search18 will far surpass these initial efforts, finding applications in domains ranging from human-computer interactive systems that can anticipate a person’s intent and render assistance to the potentially early identification of attention-related clinical disorders (ADHD, PTSD, phobia) based on deviation from neurotypical fixation behavior. |
url |
https://doi.org/10.1038/s41598-021-87715-9 |
work_keys_str_mv |
AT yupeichen cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol AT zhiboyang cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol AT seoyoungahn cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol AT dimitrissamaras cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol AT minhhoai cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol AT gregoryzelinsky cocosearch18fixationdatasetforpredictinggoaldirectedattentioncontrol |
_version_ |
1721509548468994048 |