| |
| |
| Active Confocal Imaging for Visual Prosthesis |
 |
|
| Figure 1. The background de-cluttering can be achieved using an active confocal imaging system based on light-field technology. |
|
Visual prostheses and sensory substitution devices have been developed over the years for restoring functional vision in blind users. Most of these devices use a high-resolution head-mounted video camera and convert those images to a low resolution (under 1,000 electrodes) and limited dynamic range (2-4 gray levels) to display on users’ tongue, retina, cortex, etc.
Objects of interest in front of background clutter, as frequently found in natural environment, presented at the low resolution and dynamic range are difficult to interpret. We are working on methods to effectively suppress background clutter and isolate the object of interest, background de-cluttering, to improve the functionality of visual prostheses.
Background de-cluttering can be implemented by improving the camera and image processing. We proposed an active confocal imaging system based on light-field technology that would enable a blind user of any visual prosthesis to efficiently scan, focus on, and "see" only an object of interest, while suppressing interference from background clutter. In addition, background de-cluttering may be achieved by motion parallax caused by the user’s head movement, which facilitates separation of objects at various distances with a gaze-locking assistance.
 |
 |
| Figure 2. The effect of separating the target from background using motion parallax without (left) and with (right) the gaze-locking assistance. |
|
Yitzhak Yitzhaky, Ph.D.
Avi Caspi, Ph.D. - Second Sight, Inc
Daniel Palanker, Ph.D. - Stanford University
Zahide Pamir, Ph.D.
Maha Ramamurthy, Ph.D.
Cheng Qiu, Ph.D.
Navid Mostofi, Ph.D.
DoD Grant W81XWH-16-1-0033, "Active Confocal Imaging System for Visual Prostheses"
Peli E, Jung J-H, and Qiu C (filed Mar 30, 2017) Motion parallax in object recognition. US Patent US11270451 (Issued 3/8/2022) [ PDF 1.14 MB]
Peli E and Jung J-H. Active confocal imaging systems and methods for visual prostheses. US Patent 10,130,513 B2, (Issued 11/20/2018) [ PDF 3.08 MB]
Peli E and Jung J-H. Active confocal imaging systems and methods for visual prostheses. (A continuation of US patent 10,130,513 B2). U.S. Patent 10,857,036 (Issued 12/08/2020) [PDF 3.30 MB]
Kadosh, M., Fraiman, A., Peli, E. & Yitzhaky, Y. (2023) Efficient depth localization of objects in a 3D space using computational integral imaging. , In Artificial Intelligence for Security and Defence Applications (Vol. 12742, pp. 54-57). SPIE. DOI:10.1117/12.2683627 [ PDF 1.0MB]
Avraham D, Samuels G, Peli E, Jung JH, Yitzhaky Y. (2022) Computational integral imaging based on a novel miniature camera array. 3D Image Acquisition and Display: Technology, Perception and Applications (pp. 3Tu5A-2). Optica Publishing Group Optica Publishing Group [ PDF 288KB]
Hoogsteen KMP, Szpiro S, Kreiman G, Peli E. (2022) Beyond the Cane: Describing Visual Scenes to Blind People During Outdoor Mobility. ACM Transactions on Accessible Computing. 15(3): 1-29 DOI: 10.1145/3522757 [ PDF 1.00MB]
Pamir Z, Jung J-H, Peli E. (2022) Preparing participants for the use of the tongue visual sensory substitution device. Disability and Rehabilitation: Assistive Technology, 17:8, 888-896. DOI:10.1080/17483107.2020.1821102 [PDF 2.50MB]
Avraham D, Yitzhaky, Y, Jung J-H, Peli E. (2021) Retinal Prosthetic Vision Simulation: Temporal Aspects, Journal of Neural Engineering, 18; 0460d9. DOI: 10.1088/1741-2552/ac1b6c, PDF not Available.
Peli E. (2020) Testing Vision is not Testing For Vision. Translational Vision Science & Technology, (9)32, 1-11. DOI:10.1167/tvst.9.13.32 [PDF 0.46MB]
Pamir Z, Canoluk MU, Jung J-H, Peli E. (2020) Poor resolution at the back of the tongue is the bottleneck for spatial pattern recognition. Scientific Reports, 10:2435. [ PDF 1.38MB]
Jung JH, Peli E. (2018) Light-field background de-cluttering for visual prostheses. Invited presentation in OSA 3-D Image Acquisition and Display: Technology, Perception and Applications Topical Meeting 25-28 June 2018 Orlando, FL, OSA Imaging and Applied Optics Congress. [PDF 294 KB]
Qiu C, Lee K, Jung JH, Goldstein R, Peli E.(2018) Motion Parallax Improves Object Recognition in the Presence of Clutter in Simulated Prosthetic Vision. Translational Vision Science & Technology, 2018; 7(5):29 doi: 10.1167/tvst.7.5.29 [PDF 6.27MB] Data from this paper is available on our "Data Sharing" page.
Han S. Qiu C, Lee K, Jung JH, Peli E. Word recognition: Re-thinking prosthetic vision evaluation. Journal of Neural Engineering, 2018; 15: 055003 (13pp). doi.org/10.1088/1741-2552/aac663 [ PDF 1.35MB]
Aloni, D., Yitzhaky, Y. (2017). Effects of elemental images’ quantity on threedimensional segmentation using computational integral imaging. Appl. Opt. 56, 2132-2140 (2017) [PDF 2.64 MB]
Jung, J.-H., Pu, T., & Peli, E. (2016). Comparing object recognition from binary and bipolar edge features. Electronic Imaging, 2016(16), 1–5. doi.org/10.2352/ISSN.2470-1173.2016.16.HVEI-111
Jung, J.-H., Aloni, D., Yitzhaky, Y., & Peli, E. (2015). Active confocal imaging for visual prostheses. Vision Research, 111, Part B, 182–196. doi.org/10.1016/j.visres.2014.10.023
