Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer
Background: Citizen science, scientific research conducted by non-specialists, has the potential to facilitate biomedical research using available large-scale data, however validating the results is challenging. The Cell Slider is a citizen science project that intends to share images from tumors wi...
Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Elsevier
2015-07-01
|
Series: | EBioMedicine |
Subjects: | |
Online Access: | http://www.sciencedirect.com/science/article/pii/S2352396415300165 |
id |
doaj-157e2947e3ef4014b1d620cfd827eaf4 |
---|---|
record_format |
Article |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Francisco J. Candido dos Reis Stuart Lynn H. Raza Ali Diana Eccles Andrew Hanby Elena Provenzano Carlos Caldas William J. Howat Leigh-Anne McDuffus Bin Liu Frances Daley Penny Coulson Rupesh J. Vyas Leslie M. Harris Joanna M. Owens Amy F.M. Carton Janette P. McQuillan Andy M. Paterson Zohra Hirji Sarah K. Christie Amber R. Holmes Marjanka K. Schmidt Montserrat Garcia-Closas Douglas F. Easton Manjeet K. Bolla Qin Wang Javier Benitez Roger L. Milne Arto Mannermaa Fergus Couch Peter Devilee Robert A.E.M. Tollenaar Caroline Seynaeve Angela Cox Simon S. Cross Fiona M. Blows Joyce Sanders Renate de Groot Jonine Figueroa Mark Sherman Maartje Hooning Hermann Brenner Bernd Holleczek Christa Stegmaier Chris Lintott Paul D.P. Pharoah |
spellingShingle |
Francisco J. Candido dos Reis Stuart Lynn H. Raza Ali Diana Eccles Andrew Hanby Elena Provenzano Carlos Caldas William J. Howat Leigh-Anne McDuffus Bin Liu Frances Daley Penny Coulson Rupesh J. Vyas Leslie M. Harris Joanna M. Owens Amy F.M. Carton Janette P. McQuillan Andy M. Paterson Zohra Hirji Sarah K. Christie Amber R. Holmes Marjanka K. Schmidt Montserrat Garcia-Closas Douglas F. Easton Manjeet K. Bolla Qin Wang Javier Benitez Roger L. Milne Arto Mannermaa Fergus Couch Peter Devilee Robert A.E.M. Tollenaar Caroline Seynaeve Angela Cox Simon S. Cross Fiona M. Blows Joyce Sanders Renate de Groot Jonine Figueroa Mark Sherman Maartje Hooning Hermann Brenner Bernd Holleczek Christa Stegmaier Chris Lintott Paul D.P. Pharoah Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer EBioMedicine Citizen science Crowd science Crowdsourcing Breast cancer |
author_facet |
Francisco J. Candido dos Reis Stuart Lynn H. Raza Ali Diana Eccles Andrew Hanby Elena Provenzano Carlos Caldas William J. Howat Leigh-Anne McDuffus Bin Liu Frances Daley Penny Coulson Rupesh J. Vyas Leslie M. Harris Joanna M. Owens Amy F.M. Carton Janette P. McQuillan Andy M. Paterson Zohra Hirji Sarah K. Christie Amber R. Holmes Marjanka K. Schmidt Montserrat Garcia-Closas Douglas F. Easton Manjeet K. Bolla Qin Wang Javier Benitez Roger L. Milne Arto Mannermaa Fergus Couch Peter Devilee Robert A.E.M. Tollenaar Caroline Seynaeve Angela Cox Simon S. Cross Fiona M. Blows Joyce Sanders Renate de Groot Jonine Figueroa Mark Sherman Maartje Hooning Hermann Brenner Bernd Holleczek Christa Stegmaier Chris Lintott Paul D.P. Pharoah |
author_sort |
Francisco J. Candido dos Reis |
title |
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer |
title_short |
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer |
title_full |
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer |
title_fullStr |
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer |
title_full_unstemmed |
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer |
title_sort |
crowdsourcing the general public for large scale molecular pathology studies in cancer |
publisher |
Elsevier |
series |
EBioMedicine |
issn |
2352-3964 |
publishDate |
2015-07-01 |
description |
Background: Citizen science, scientific research conducted by non-specialists, has the potential to facilitate biomedical research using available large-scale data, however validating the results is challenging. The Cell Slider is a citizen science project that intends to share images from tumors with the general public, enabling them to score tumor markers independently through an internet-based interface.
Methods: From October 2012 to June 2014, 98,293 Citizen Scientists accessed the Cell Slider web page and scored 180,172 sub-images derived from images of 12,326 tissue microarray cores labeled for estrogen receptor (ER). We evaluated the accuracy of Citizen Scientist's ER classification, and the association between ER status and prognosis by comparing their test performance against trained pathologists.
Findings: The area under ROC curve was 0.95 (95% CI 0.94 to 0.96) for cancer cell identification and 0.97 (95% CI 0.96 to 0.97) for ER status. ER positive tumors scored by Citizen Scientists were associated with survival in a similar way to that scored by trained pathologists. Survival probability at 15 years were 0.78 (95% CI 0.76 to 0.80) for ER-positive and 0.72 (95% CI 0.68 to 0.77) for ER-negative tumors based on Citizen Scientists classification. Based on pathologist classification, survival probability was 0.79 (95% CI 0.77 to 0.81) for ER-positive and 0.71 (95% CI 0.67 to 0.74) for ER-negative tumors. The hazard ratio for death was 0.26 (95% CI 0.18 to 0.37) at diagnosis and became greater than one after 6.5 years of follow-up for ER scored by Citizen Scientists, and 0.24 (95% CI 0.18 to 0.33) at diagnosis increasing thereafter to one after 6.7 (95% CI 4.1 to 10.9) years of follow-up for ER scored by pathologists.
Interpretation: Crowdsourcing of the general public to classify cancer pathology data for research is viable, engages the public and provides accurate ER data. Crowdsourced classification of research data may offer a valid solution to problems of throughput requiring human input. |
topic |
Citizen science Crowd science Crowdsourcing Breast cancer |
url |
http://www.sciencedirect.com/science/article/pii/S2352396415300165 |
work_keys_str_mv |
AT franciscojcandidodosreis crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT stuartlynn crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT hrazaali crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT dianaeccles crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT andrewhanby crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT elenaprovenzano crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT carloscaldas crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT williamjhowat crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT leighannemcduffus crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT binliu crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT francesdaley crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT pennycoulson crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT rupeshjvyas crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT lesliemharris crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT joannamowens crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT amyfmcarton crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT janettepmcquillan crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT andympaterson crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT zohrahirji crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT sarahkchristie crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT amberrholmes crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT marjankakschmidt crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT montserratgarciaclosas crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT douglasfeaston crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT manjeetkbolla crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT qinwang crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT javierbenitez crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT rogerlmilne crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT artomannermaa crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT ferguscouch crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT peterdevilee crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT robertaemtollenaar crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT carolineseynaeve crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT angelacox crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT simonscross crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT fionamblows crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT joycesanders crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT renatedegroot crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT joninefigueroa crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT marksherman crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT maartjehooning crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT hermannbrenner crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT berndholleczek crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT christastegmaier crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT chrislintott crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer AT pauldppharoah crowdsourcingthegeneralpublicforlargescalemolecularpathologystudiesincancer |
_version_ |
1725079338819256320 |
spelling |
doaj-157e2947e3ef4014b1d620cfd827eaf42020-11-25T01:33:06ZengElsevierEBioMedicine2352-39642015-07-012768168910.1016/j.ebiom.2015.05.009Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in CancerFrancisco J. Candido dos Reis0Stuart Lynn1H. Raza Ali2Diana Eccles3Andrew Hanby4Elena Provenzano5Carlos Caldas6William J. Howat7Leigh-Anne McDuffus8Bin Liu9Frances Daley10Penny Coulson11Rupesh J. Vyas12Leslie M. Harris13Joanna M. Owens14Amy F.M. Carton15Janette P. McQuillan16Andy M. Paterson17Zohra Hirji18Sarah K. Christie19Amber R. Holmes20Marjanka K. Schmidt21Montserrat Garcia-Closas22Douglas F. Easton23Manjeet K. Bolla24Qin Wang25Javier Benitez26Roger L. Milne27Arto Mannermaa28Fergus Couch29Peter Devilee30Robert A.E.M. Tollenaar31Caroline Seynaeve32Angela Cox33Simon S. Cross34Fiona M. Blows35Joyce Sanders36Renate de Groot37Jonine Figueroa38Mark Sherman39Maartje Hooning40Hermann Brenner41Bernd Holleczek42Christa Stegmaier43Chris Lintott44Paul D.P. Pharoah45Department of Oncology, University of Cambridge, Cambridge, UKDepartment of Physics (Astrophysics), University of Oxford, Oxford, UKCancer Research UK, Cambridge Institute, Cambridge, UKUniversity of Southampton, Southampton, UKUniversity of Leeds, Leeds, UKAddenbrookes Hospital NHS Trust, Cambridge, UKCancer Research UK, Cambridge Institute, Cambridge, UKCancer Research UK, Cambridge Institute, Cambridge, UKCancer Research UK, Cambridge Institute, Cambridge, UKCancer Research UK, Cambridge Institute, Cambridge, UKInstitute of Cancer Research, London, UKInstitute of Cancer Research, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKCancer Research UK, London, UKNetherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The NetherlandsInstitute of Cancer Research, London, UKDepartment of Oncology, University of Cambridge, Cambridge, UKDepartment of Public Health and Primary Care, University of Cambridge, Cambridge, UKDepartment of Public Health and Primary Care, University of Cambridge, Cambridge, UKHuman Genotyping (CEGEN) Unit, Human Cancer Genetics Program, Spanish National Cancer Research Centre (CNIO), Madrid, SpainCancer Epidemiology Centre, Cancer Council Victoria, Melbourne, AustraliaInstitute of Clinical Medicine, Pathology and Forensic Medicine, University of Eastern Finland, FinlandDepartment of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USADepartment of Human Genetics & Department of Pathology, Leiden University Medical Center, Leiden, The NetherlandsDepartment of Surgical Oncology, Leiden University Medical Center, Leiden, The NetherlandsDepartment of Medical Oncology, Family Cancer Clinic, Erasmus MC Cancer Institute, Rotterdam, The NetherlandsSheffield Cancer Research, Department of Oncology, University of Sheffield, Sheffield, UKAcademic Unit of Pathology, Department of Neuroscience, University of Sheffield, Sheffield, UKDepartment of Oncology, University of Cambridge, Cambridge, UKNetherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The NetherlandsNetherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The NetherlandsNational Cancer Institute, USANational Cancer Institute, USADepartment of Medical Oncology, Family Cancer Clinic, Erasmus MC Cancer Institute, Rotterdam, The NetherlandsDivision of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, GermanySaarland Cancer Registry, Saarbrücken, GermanySaarland Cancer Registry, Saarbrücken, GermanyDepartment of Physics (Astrophysics), University of Oxford, Oxford, UKDepartment of Oncology, University of Cambridge, Cambridge, UKBackground: Citizen science, scientific research conducted by non-specialists, has the potential to facilitate biomedical research using available large-scale data, however validating the results is challenging. The Cell Slider is a citizen science project that intends to share images from tumors with the general public, enabling them to score tumor markers independently through an internet-based interface. Methods: From October 2012 to June 2014, 98,293 Citizen Scientists accessed the Cell Slider web page and scored 180,172 sub-images derived from images of 12,326 tissue microarray cores labeled for estrogen receptor (ER). We evaluated the accuracy of Citizen Scientist's ER classification, and the association between ER status and prognosis by comparing their test performance against trained pathologists. Findings: The area under ROC curve was 0.95 (95% CI 0.94 to 0.96) for cancer cell identification and 0.97 (95% CI 0.96 to 0.97) for ER status. ER positive tumors scored by Citizen Scientists were associated with survival in a similar way to that scored by trained pathologists. Survival probability at 15 years were 0.78 (95% CI 0.76 to 0.80) for ER-positive and 0.72 (95% CI 0.68 to 0.77) for ER-negative tumors based on Citizen Scientists classification. Based on pathologist classification, survival probability was 0.79 (95% CI 0.77 to 0.81) for ER-positive and 0.71 (95% CI 0.67 to 0.74) for ER-negative tumors. The hazard ratio for death was 0.26 (95% CI 0.18 to 0.37) at diagnosis and became greater than one after 6.5 years of follow-up for ER scored by Citizen Scientists, and 0.24 (95% CI 0.18 to 0.33) at diagnosis increasing thereafter to one after 6.7 (95% CI 4.1 to 10.9) years of follow-up for ER scored by pathologists. Interpretation: Crowdsourcing of the general public to classify cancer pathology data for research is viable, engages the public and provides accurate ER data. Crowdsourced classification of research data may offer a valid solution to problems of throughput requiring human input.http://www.sciencedirect.com/science/article/pii/S2352396415300165Citizen scienceCrowd scienceCrowdsourcingBreast cancer |