3D Medical Imaging

Virtual Reality Becoming Real: Interactive 3D Medical Imaging

The ability of 3D rendering systems to process actual MRI and computer tomography data into 3D images has major implications for surgery and for medical education.


  • Page 1 of 1
    Bookmark and Share

Article Media

Since Wilhelm Röntgen first made an X-ray image of his wife’s hand in 1895, medical imaging has advanced in its insight, resolution and applications but is still regarded as “incomplete” even with the huge progress in magnetic resonance imaging (MRI), computer tomography (CT) and ultrasound. Another big leap now appears to be in progress with the development of the ability to render high-resolution 3D images from individual patients and to analyze and interact with them in 3D space.

Interestingly, the advances do not stem from new imaging technology, but rather from the ability of high-powered graphics processing to render the data gathered from imaging systems like MRI and CT and present them as 3D objects that can be rotated in 3D space, sliced and examined and used for advanced diagnostics as well as for surgical planning. EchoPixel has combined 3D rendering and imaging software with a powerful workstation from zSpace, which is based on the powerful NVIDIA Quadro K4200 GPU to image the human body in interactive 3D.

According to EchoPixel’s Sergio Aguirre, the current state of imaging has doctors looking at a series of 2D images and trying to reproduce the 3D structure in their minds and while there are some 3D rendering systems, the images are presented on a 2D screen. So, he notes, “Doctors are busy solving the 3D view instead of the physical problem they should be working on.” This can lead to difficulties on a number of levels. For one thing it’s not just the loss of clinical information, but also impacts productivity and communication with colleagues and with patients. Often, Aquirre says, a surgeon will walk into the OR with a hand-drawn piece of paper as a surgical plan.

However, the data gathered by an MRI or CT scan consists of a large number of 2D image slices through the tissue under examination. This aggregate stack of 2D cross sections is inherently 3D and can theoretically be treated as such given the proper algorithms and processing power. Basically, the software is capable of taking the existing data from a variety of imaging devices that is stored, for example, on hospital servers and process it into 3D objects that can be viewed and manipulated by doctors using 3D glasses on the desktop machine from zSpace running the EchoPixel True 3D Viewer software (Figure 1).

Figure 1
The EchoPixel True 3D Viewer software runs on a 3D system from zSpace that is based on the NVIDIA Quadro K4200 GPU and lets the user wear 3D glasses to interact with models in 3D space.

Using existing MRI or CT data can mean working with a varying number of “slices” for a given image and of course, the system works better with more slices and higher resolution data. But it can also work well over a range of slice depths. In general, the voxels should have a ratio of in-plane spatial resolution to slice space of equal or less than 1:5. In general, CTs have a ratio of 1:1.5, and MRs with a ratio of 1:2, or 1:3. Of course, the higher the resolution of the image, the better.

By using true 3D rendering, the user is able to interact with the object in terms of its characteristics as a three-dimensional solid. Thus the doctor can characterize his stylus as, say, a scalpel. He can then “cut” into and object, be it a liver, intestine, heart or brain, and see the interior as it actually is, perhaps discovering a beginning tumor that was previously unrecognized. He can select a given body part and isolate it and use the 3D rendering for surgical planning on that unique individual. The surgeon can then actually see the response of the tissue as he moves across a patient’s kidney or liver in the response of the data to whatever virtual instrument he or she is applying to it (Figure 2).

Figure 2
This colon image is from a specific application for colon cancer screening developed at the University of California San Francisco.

There are often anomalies within individual bodies, such as the position or routing of an artery, that are discovered only when the incision has been made on the operating table. Then the surgical team must alter its strategy, which often prolongs the time the patient must remain under anesthesia. By discovering such anomalies in advance, the surgeon can make a more accurate plan. Beyond that, he can practice the actual procedure and build muscle memory that will aid him in the actual operation.

According to Aquirre, the system has been used with newborn pediatric patients with the need for pulmonary artery reconstruction, which not surprisingly, is a very complex and delicate procedure. In clinical trials on improving the surgical plan, the detection time for insight into the operation was improved by 90 percent and the surgeons expect to be able to reduce the actual time for the operation from four hours to an hour and a half.

The availability of such accurate and interactive data has large implications for the future of internal medicine, for communication among physicians and physicians with patients and also for medical education. For example, surgeon must get the patient to sign off for the operation, which usually requires that the patient understand the procedure. To no one’s surprise, describing complex surgical procedures to a lay person can be quite difficult and time-consuming. The ability to show the patient exactly what is involved is expected to reduce the time and difficulty of that required step.

In addition, medical schools and universities have a need for cadavers, which are in limited supply and quite expensive. The availability of accurate, interactive anatomical models can possibly reduce—though not eliminate—the need for cadavers in medical education. In addition, professors of anatomy will be able to use the system as models of various anatomical variations and anomalies are collected. This will help medical students better grasp the variety they can expect in dealing with real patients (Figure 3).

Figure 3
This image looking inside a brain is able to identify the precise size, shape and location of tumors.

Interestingly, Aquirre says they consider this initial product to be an entry level product and are preparing to delve into larger architectures and more exciting applications. While the system, which has received FDA approval, is now used for diagnostics and surgical planning, surgeons are now reportedly expressing the desire for real-time support. They would like to be able to make a surgical plan, save it to a file and send it to the operating room. There they would have the map available to them in real time. This will depend heavily on the performance of the graphics processor. EchoPizel is now looking beyond the Kepler architecture-based Quadro to newer architectures coming out of NVIDIA such as the Maxwell and the soon anticipated Pascal devices.

As with many new technology developments, we appear to be at the beginning of a major application in medicine. It is fully to be expected that medical professionals in a wide range of fields will recognize the potential of the technology and motivate the system designers to expand its applications to areas they initially could not imagine.

Mountain View, CA

Santa Clara, CA
(408) 486-2000

Sunnyvale, CA

(877) 977-2231