Digital microscopy, 3D visualization

Advances in scientific disciplines such as the digitization of tissue plates, the handling of vast amounts of data, and the revolution in virtual reality are helping and motivating factors that open up new perspectives in digital pathology. Over time, the tasks of pathologists are increasingly supported by computers, which reduces the time required and the burden of tedious tasks. The digital pathology system must have interactive and attractive features, serve pathologists effectively, expand their cognitive abilities, and meet their needs.

Digitization of tissue samples is a process that allows for three-dimensional visualization that mimics the nature of tissue samples, but the huge amount of histological data is challenging in three-dimensional visualization as the size reaches gigabytes and thus takes a lot of time to load into memory. In addition to all existing requirements, the visualization tool should be user-friendly and easily accessible to researchers and pathologists.

The project aims to create a visualization interface that takes advantage of game engines and virtual reality. This project makes it possible to load two-dimensional medical images and create three-dimensional representations of specific slides using histograms. These images are loaded along with their associated metadata, which is later used to create a heat map of the various annotation sites within the slide. The project is able to provide these functionalities at different magnifications of the slide as well as at different sections of the slide.

Tissue samples were scanned at different magnifications, where each magnification level consisted of mosaics, and these images were saved in digital file format. The “.Net” library was used to convert these digital slides to a known image format suitable for processing with the Godot engine. The “.Net” script also processed the metadata within the Godot engine. The processed metadata was compiled as a heat map and then combined with different pieces of the sample. With Oculus Rift, the user can interact with the environment and control the visualization.