Retinal photography
Overview of Data Domain
Retinal photography is a non-invasive procedure that photographs the posterior segment of an eye, also known as the fundus. It provides two-dimensional images of the fundus and can be performed with different filters of different wavelengths when an eye is undilated or dilated. The main structures that can be visualized inon a fundus photo are the optic nerve, macula, retinal vasculature, and central and peripheral retina. Fundus photography is used to record the condition of these structures to document the presence of abnormalities and monitor the changes over time.
Color fundus photography
Color fundus photography is a direct visualization of the fundus in the form of color images.
Figure 1: Normal fundus photographs of the right eye (A) and left eye (B).
Color fundus photography can capture various abnormalities in diabetic retinopathy, such as microaneurysms, retinal hemorrhages, cotton wool spots, hard or soft exudates, and neovascularization of the disc or the retina.
Figure 2: Examples of lesions in diabetic retinopathy captured by color fundus photography. Reference: Alyoubi WL, Abulkhair MF, Shalash WM. Diabetic Retinopathy Fundus Image Classification and Lesions Localization System Using Deep Learning. Sensors. 2021; 21(11):3704. https://doi.org/10.3390/s21113704
Fundus autofluorescence photography
Fundus autofluorescence photography (FAF) utilizes the intrinsic fluorescent nature of lipofuscin found in the retinal pigment epithelium (RPE) to generate images. Lipofuscin, a byproduct of photoreceptor outer segment breakdown in lysosomes, absorbs blue light and emits yellow-green light. By illuminating with a light source and capturing the emitted signals with a detector, an image is formed that serves as a lipofuscin density map, where brighter zones indicate higher lipofuscin concentration.
In a normal fundus without retinal pathology, blood vessels and the optic nerve usually appear dark on FAF photos. With blue or green short-wavelength FAF, the fovea is usually visualized as a spot of hypo-autofluorescence due to the high concentration of light-absorbing xanthophyll pigment in this area. FAF has become an important tool for evaluating diseases that result in alterations of RPE or lipofuscin levels, such as in age-related macular degeneration and hereditary retinal diseases.
Figure 3: Comparison of common imaging systems available for fundus autofluorescence. Images of normal, healthy retina were obtained using the fundus camera with a Spaide filter (a), confocal scanning laser ophthalmoscope (cSLO) (b), and Optos ultra-widefield systems (c). Reference: Yung, M., Klufas, M.A. & Sarraf, D. Clinical applications of fundus autofluorescence in retinal disease. Int J Retin Vitr 2, 12 (2016). https://doi.org/10.1186/s40942-016-0035-x
Fundus infrared photography
Compared to traditional color fundus photography and FAF, infrared imaging offers certain advantages. Infrared has a longer depth of penetration that can reach deeper subretinal layers. It produces a better vessel to background contrast which improves the detection of subretinal pathologies.
Moreover, it removes the out-of-focus, scattered components of the reflected light, improving the quality of illumination. Infrared imaging has the potential to generate deeper visualization of the retina and detect pathologies even in the presence of media opacity such as hemorrhages and cataracts. It is routinely used during optical coherence tomography (OCT) imaging to visualize the structure of the optic nerve, retina, subretinal lesions, choroidal lesions, and choroidal neovascularization.
Reference: Ajaz A, Kumar DK. Infrared retinal images for flashless detection of macular edema. Sci Rep. 2020;10(1):14384. Published 2020 Sep 1. doi:10.1038/s41598-020-71010-0
Overview of AI-READI Retinal Photography Data
Overview of available images
| Manufacturer | Manufacturer's Model (device) | Anatomic Regions* | Imaging Types* | Dilation |
|---|---|---|---|---|
| Optomed | Aurora | Macula, Optic Disc | Color Photography | No |
| iCare | Eidon | Macula, Nasal, Temporal Periphery, Mosaic | Color Photography, Infrared Reflectance, Autofluorescence | Yes |
| Topcon | Maestro2 | Macula, Wide Field | Color Photography, Infrared Reflectance | Yes |
| Topcon | Triton | Macula, Optic Disc | Color Photography | Yes |
| Heidelberg | Spectralis | Macula, Optic Disc | Infrared Reflectance | Yes |
*More comprehensive details regarding each imaging file are available in the “manifest.tsv” file located within the retinal_photography folder.
*Based on the anatomic regions and Imaging types, images can be mapped to protocols that were given by the manufacturers, as shown in the table below.
According to the AI-READI imaging protocols, the Triton device primarily captures color fundus photos (CFP) as retinal photography images. However, for two participants (4150 and 7111), infrared (IR) images were taken as retinal photography images. Since the data were usable, we included them in the dataset. Just a note: the majority of Triton retinal photography images are CFP.
Mapping of Anatomic Regions and Imaging Types to Manufacturers' Protocol Names
| Protocol Name | Anatomic Regions* | Width | Height | Number of frames | Notes |
|---|---|---|---|---|---|
| OptoMed-Disc centered-CFP | Optic Disc | Color Photography | 1776 | 2368 | One can differentiate between these two protocols by visually confirming the images where they were centered |
| OptoMed-Mac centered-CFP | Macula | Color Photography | |||
| Eidon-UWF Central-IR | Macula | Infrared Reflectance | 3288 | 3680 | |
| Eidon-UWF Central-FAF | Macula | Autofluorescence | 3288 | 3680 | |
| Eidon-UWF Central-CFP | Macula | Color Photography | 3288 | 3680 | |
| Eidon-UWF Nasal-CFP | Nasal | Color Photography | 3288 | 3680 | |
| Eidon-UWF Temporal-CFP | Temporal Periphery | Color Photography | 3288 | 3680 | |
| Eidon-Create Mosaic Image-CFP | Mosaic | Color Photography | varies | varies | Mosaic images are larger images created by multiple images. |
| Spectralis-ONH-RC-HR-OCT | Optic Disc | Infrared Reflectance | 1536 | 1536 | IR images are from OCT scans. This OCT protocol produces OCT B scans and associated IR scans. |
| Spectralis-PPole Mac-HR-61 lines-OCT | Macula | Infrared Reflectancee | 1536 or 768 | 1536 or 768 | IR images are from OCT scans. This OCT protocol produces OCT B scans and associated IR scans. Two versions of the protocols were used for the pilot but in the future 1536, and 1536 will be used. |
| Maestro2-3D Wide(H) 12x9-OCT | Wide Field | Color Photography | 1958 | 2576 | Color fundus scans are from OCT scans. This OCT protocol produces OCT B scans and associated color fundus scans. |
| Maestro2-3D Macula 6x6-OCT | Macula | Color Photography | 1958 | 2576 | Color fundus scans are from OCT scans. This OCT protocol produces OCT B scans and associated color fundus scans. |
| Maestro2-Mac 6x6-360x360-(rep3)-OCTA | Optic Disc | Color Photography | 480 | 640 | IR images are from OCTA scans. This OCTA protocol produces flow cube scans, segmentation scans, en face scans, OCT B scans, and associated IR scans. 6 files have different dimensions of 1958 (height), 2576 (width) and they are in color due to a change in imaging method. |
| Triton-3D(H)+Radial 12x9-OCT | Macula, 6 x 6 | Color Photography | 1934 | 2576 | Color fundus scans are from OCT scans. This OCT protocol produces OCT B scans and associated color fundus scans. |
| Triton-Macula 6x6-OCTA | Macula, 12 x 12 | Color Photography | 1934 | 2576 | Color fundus scans are from OCTA scans. This OCTA protocol produces flow cube scans, segmentation scans, en face scans, OCT B scans, and associated color fundus scans. |
| Triton-Macula 12x12-OCTA | Color Photography | Color Photography | 1934 | 2576 | Color fundus scans are from OCTA scans. This OCTA protocol produces flow cube scans, segmentation scans, en face scans, OCT B scans, and associated color fundus scans. |
* More information on individual files, please refer to manifest.csv within the retinal_photography folder
* Note that all color photographs' photometric interpretation is YBR (YBR_FULL_422), which is primarily used with JPEG compressed bit streams or pixel data in an uncompressed format. This was defined by the manufacturers. To view color photographs in RGB using pydicom, please consider using their convert_color_space function (from YBR_FULL to RGB). They are automatically shown in intended color when using a viewer like MicroDicom.
* Note that retinal photographs from Spectralis, Maestro2, and Triton are associated with OCT or OCTA scans.
* Note that there are 6 color fundus photos generated from Maestro2-Mac 6x6-360x360-(rep3)-OCTA although the majority of them are IR images due to a change in methods. These photos are located with the IR images to organize files consistently based on the protocols.
CFP - Color fundus photo, IR - Infrared reflectance, OCT - Optical coherence tomography, OCTA - Optical Coherence Tomography Angiography, UWF - Ultra-widefield
Example Images
Color fundus photographs of diabetic retinopathy acquired with the hand-held Optomed Aurora fundus camera.
Figure 4: Fundus photographs (50°) of patients with (a) and without (b) signs of diabetic retinopathy acquired with the hand-held Optomed Aurora fundus camera. Reference: Lupidi M, Danieli L, Fruttini D, et al. Artificial intelligence in diabetic retinopathy screening: clinical assessment using handheld fundus camera in a real-life setting. Acta Diabetol. 2023;60(8):1083-1088. doi:10.1007/s00592-023-02104-0.
Color fundus photographs of diabetic retinopathy
Figure 5: Color fundus photos show different stages of diabetic retinopathy (DR). (a) No DR. (b) Mild non-proliferative DR. (c) Moderate non-proliferative DR. (d) Severe non-proliferative DR. (e) Proliferative DR.
Data Processing
File format
The file format is in DICOM (Digital Imaging and Communications in Medicine) format which is a technical standard for the digital storage and transmission of medical images and related information.
Data Standard
For all files in retinal_photography folder, the exported files from the devices were formatted according to the NEMA DICOM standards on Ophthalmic Photography 8 Bit Image. The detailed information about the tags and values can be found in the hyperlink. All mandatory modules (M) were included, and within the mandatory modules, the tags that are categorized as Type 1 (Tag and value both needed) and Type 2 (Tag needed value can be empty) were evaluated and filled in accordingly.
File processing
Files were processed to comply with the NEMA DICOM standards on Ophthalmic Photography 8 Bit Image by removing, adding, and editing tags and values.
Note that tags that are not listed in Ophthalmic Photography 8 Bit Image IOD Modules and did not provide additional information are removed for consistency and clarity.
In addition to ensuring files are NEMA compliant, further processing in the following was done to ensure consistency across the files:
| DICOM Tag Number | DICOM Tag Name | Values | Action, Rationale |
|---|---|---|---|
| (0010,0010) | Patient Name | Blank | Removal, Patient information |
| (0010,0030) | Patient Birth Date | Blank | Removal, Patient information |
| (0010,0040) | Patient Sex | A unified value of “M” | Removal, Patient information removed |
| (0008,0090) | Referring Physician Name | Blank | Removal, Unnecessary information and inconsistent across devices |
| (0020, 0010) | Study ID | Blank | Removal, Unnecessary information and inconsistent across devices |
| (0008, 0050) | Accession Number | Blank | Removal, Unnecessary information and inconsistent across devices |
| (0008, 0008) | Anatomic Region Sequence | e.g., ORIGINAL, PRIMARY, , COLOR ORIGINAL, PRIMARY, , INFRARED ORIGINAL, PRIMARY, , AUTOFLUORESCENCE | Removal, Unnecessary information and inconsistent across devices |
| (0020, 0020) | Patient Orientation | e.g., L, F | Harmonized, Consistency |
| (0008, 002A) | Acquisition Date Time | e.g., 20231011145705 | Harmonized, Consistency |
| (0022, 0016) | Illumination Sequence | e.g., 410462008, SCT, Fine slit beam direct illumination | Harmonized, up-to-date information. Based on dicom.nema.org. DICOM PS3.16 2024b - Content Mapping Resource B.DCMR Context Groups (Normative) |
| (0008, 2218) | Anatomic Region Sequence | e.g., 5665001, SCT, Retina | |
| (0008, 2218) | Acquisition Device Type Code Sequence | e.g., 409898007, SCT, Fundus Camera | |
| (0022, 000D) | Pupil Dilated | e.g., YES/NO | Addition, Missing value |
| (0022, 0058) | Mydriatic Agent Sequence | e.g., empty | Addition, Missing tag (tag is needed if Pupil Dilated is YES) |
| (0022, 000E) | Degree Of Dilation | e.g., empty | Addition, Missing tag (tag is needed if Pupil Dilated is YES) |
| (0028, 0030) | Pixel Spacing | e.g., 0.00634, 0.00634 | Addition, Missing value |
| (0018, 1030) | Protocol Name | e.g., eidon central color retinal photography | Addition, Better information delivery |
Metadata Example
| Attribute Name | Tags | Attribute Description |
|---|---|---|
| Patient ID | (0010, 0020) | Primary identifier for the patient. |
| Image Type | (0008, 0008) | Image identification characteristics. |
| Manufacturer | (0008, 0070) | Manufacturer of the device. |
| Manufacturer's Model Name | (0008, 1090) | Manufacturer's model name of the device. |
| Image Laterality | (0020, 0062) | Laterality of object image examined. |
| Pupil Dilated | (0022, 000D) | Whether or not the patient's pupils were pharmacologically dilated for this acquisition. |
| Samples Per Pixel | (0028, 0002) | Number of samples (planes) in this image. |
| Number of Frames | (0028, 0008) | Number of frames in a Multi-frame Image. |
| Rows | (0028, 0010) | Number of rows in the image. |
| Columns | (0028, 0011) | Number of columns in the image. |