Vision without the Image
Novel image sensors transduce the stream of photons directly into asynchronous electrical pulses, rather than forming an image. Classical approaches to vision start from a good quality image and therefore it is tempting to consider image reconstruction as a first step to image analysis. We propose t...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2016-04-01
|
Series: | Sensors |
Subjects: | |
Online Access: | http://www.mdpi.com/1424-8220/16/4/484 |
id |
doaj-e3e2644501f14bb58ae815d3e2491bbb |
---|---|
record_format |
Article |
spelling |
doaj-e3e2644501f14bb58ae815d3e2491bbb2020-11-25T01:05:58ZengMDPI AGSensors1424-82202016-04-0116448410.3390/s16040484s16040484Vision without the ImageBo Chen0Pietro Perona1Computation and Neural Systems, California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USAComputation and Neural Systems, California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USANovel image sensors transduce the stream of photons directly into asynchronous electrical pulses, rather than forming an image. Classical approaches to vision start from a good quality image and therefore it is tempting to consider image reconstruction as a first step to image analysis. We propose that, instead, one should focus on the task at hand (e.g., detection, tracking or control) and design algorithms that compute the relevant variables (class, position, velocity) directly from the stream of photons. We discuss three examples of such computer vision algorithms and test them on simulated data from photon-counting sensors. Such algorithms work just-in-time, i.e., they complete classification, search and tracking with high accuracy as soon as the information is sufficient, which is typically before there are enough photons to form a high-quality image. We argue that this is particularly useful when the photons are few or expensive, e.g., in astronomy, biological imaging, surveillance and night vision.http://www.mdpi.com/1424-8220/16/4/484photon-counting sensorsvisual recognitionlow-light computer vision |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Bo Chen Pietro Perona |
spellingShingle |
Bo Chen Pietro Perona Vision without the Image Sensors photon-counting sensors visual recognition low-light computer vision |
author_facet |
Bo Chen Pietro Perona |
author_sort |
Bo Chen |
title |
Vision without the Image |
title_short |
Vision without the Image |
title_full |
Vision without the Image |
title_fullStr |
Vision without the Image |
title_full_unstemmed |
Vision without the Image |
title_sort |
vision without the image |
publisher |
MDPI AG |
series |
Sensors |
issn |
1424-8220 |
publishDate |
2016-04-01 |
description |
Novel image sensors transduce the stream of photons directly into asynchronous electrical pulses, rather than forming an image. Classical approaches to vision start from a good quality image and therefore it is tempting to consider image reconstruction as a first step to image analysis. We propose that, instead, one should focus on the task at hand (e.g., detection, tracking or control) and design algorithms that compute the relevant variables (class, position, velocity) directly from the stream of photons. We discuss three examples of such computer vision algorithms and test them on simulated data from photon-counting sensors. Such algorithms work just-in-time, i.e., they complete classification, search and tracking with high accuracy as soon as the information is sufficient, which is typically before there are enough photons to form a high-quality image. We argue that this is particularly useful when the photons are few or expensive, e.g., in astronomy, biological imaging, surveillance and night vision. |
topic |
photon-counting sensors visual recognition low-light computer vision |
url |
http://www.mdpi.com/1424-8220/16/4/484 |
work_keys_str_mv |
AT bochen visionwithouttheimage AT pietroperona visionwithouttheimage |
_version_ |
1725192166079201280 |