Making the Invisible...Visible

A Revolutionary Step in Higher Quality Images for X-Ray Diagnostic Imaging Systems

The image processing engine has a significant effect on the image quality of still and video images produced by X-ray systems.
The development that resulted in catching up and leaping to the top originated from a game-changing idea.

A Challenging Video Image

"You can make this visible?"
Hisanori Morita, the Image Group Manager in the Data Processing Unit at the Technology Research Laboratory of Shimadzu Corporation, was at a loss of words.
The video image was about five seconds long and had been brought in by Kazuhiro Mori, the FPD/Application Group Manager, and Mitsuru Umeda, the CVS Group Manager in the R&D Department of the Medical Systems Division. It showed a heart with a blood vessel circled around it and also a stent that was expanding the blood vessel. It was the raw image, before image processing was applied, and it included noise throughout the image that made the blood vessel appear fuzzy. Consequently, the location of the blood vessel and of the wire that was inserted through it, was unclear.
"I'll try, but...," was the best answer Morita could offer.

Hisanori Morita

 

Rapid Advancements in Image Processing Technology

Angiography systems use X-rays to acquire images of cardiac blood vessels. They serve as the "eyes" that are essential for coronary artery catheterization procedures involving blood vessels narrowed by arterial sclerosis.
Catheterization procedures, which appeared in the 1980s, were quickly adopted into widespread use because they caused less stress on patients than the bypass operation normally used at the time. In recent years, however, improvements in the performance of angiography systems, stents, the guide wires used to deliver the stents, and so on, have resulted in broadening the scope of catheterization procedures. Stents only 2 mm in diameter are now available, so that the procedure can even be used for blood vessels that are quite small, which has helped save many lives.
Angiography systems continuously display an X-ray image of the patient's chest area on the monitor throughout the catheterization procedure. Of course, to minimize radiation exposure, X-ray levels should be as low as possible. However, if the X-ray level is too low, it is like trying to find a crow in the dark by candlelight, which results in seeing only blurry images of blood vessels, stents, and the heart.
To overcome this, image processing technology is used to improve visibility. The basic concept is similar to the processing of digital camera images on a computer. However, angiography systems display video images and users do not have any time to stop the video during the procedure so that images can be processed one frame at a time. Users require that the data is processed automatically at about 15 frames per second, essentially in real time.

In 1991, when Mori joined Shimadzu, analog technology had just been displaced by digital.
"Due to the processing capacity of computers at the time, only very simple image processing was available. However, thanks to the rapid revolutionary technological progress that has been made in the space of just 20 years, it is now possible to apply complex still image processing techniques to video as well" (Mori).

Mitsuru Umeda

Of course, image processing technology is not the only thing that provides high image quality. It is also important to optimize the performance of the X-ray tube and detector and of the overall system. However, due to dramatic improvements in computer performance and software technology, image quality is increasingly determined by image processing technology.
Naturally, the Medical Systems Division invests considerable effort in image processing to progressively improve products with higher technological capabilities. In 2012, we released angiography systems equipped with a newly developed high speed image processing engine.
However, by that time competitors had already moved a step ahead, which resulted in harsh feedback from users.
"The images acquired by competing systems were definitely able to show very narrow blood vessels that were not visible on our system. I remember feeling a sense of panic that we were in trouble if this situation were to continue" (Umeda).
At this point, Morita's team, which was involved in researching image processing technology at the Technology Research Laboratory, was brought in.
" Developing Image Quality for the Future "
That is how Morita saw his mission, to keep researching image processing technology for obtaining high image quality that would benefit products five and ten years into the future. However, Morita recalls that even he was not very confident about delivering on what was requested.
"Because X-ray levels were kept low to minimize patient exposure levels, the noise levels became too high. In some locations, it was even difficult to differentiate between the noise that was supposed to be eliminated and the blood vessels that were supposed to be retained" (Morita).

Adding What Cannot Be Added

When performing image processing, the most effective way to reduce noise in video images is to repeatedly add data from previous images. Overlaying multiple images to add image information can make random noise disappear and increase contrast, thus enabling sharper images to be obtained.
However, if the object being viewed is moving, then the outline of the object in the added image information will not be aligned with the captured image, which will leave residual images.
"To prevent residual images nothing that moves should be added to the current image. However, blood vessels, stents, and other points of interest in fluoroscopic images of the heart are constantly moving due to the heartbeat, so that the information from the object of interest cannot be added to images. Consequently, noise that is overlaid on the item of interest does not disappear, and the item cannot be seen clearly. We tried various alternative processes for eliminating noise without adding information, but we failed to achieve the desired image quality" (Morita).

Contrast Enhanced Image of Left Coronary Artery Acquired Using the New Trinias Series Equipped with the SCORE PRO Advance Image Processing Engine

Morita and Ryo Takeda, a third-year employee under Morita continued trial and error efforts day after day. Then one day, several months later, Takeda happened to mutter.
"What would happen if we tracked the object before adding its information?"
It was a game-changing idea! It meant determining all the parts of the object of interest that moved in the next frame and then adding the information to the frame. That would surely eliminate the noise without generating residual images. We had known for a long time that it was possible, but computer processing power had been insufficient to keep up with the video image, so it was assumed that it would take years before it could be achieved in reality.
"We even consulted with a hardware expert, who told us the massive number of calculations required could probably not be accomplished on a computer small enough to fit inside the operating room" (Morita).
Nevertheless, Morita gave the idea the green light. It was our only hope. Takeda poured his heart and soul into writing the software until eventually he completed the algorithms.

Never-Ending Battle

Mori and Umeda then made another visit to the Technology Research Laboratory, where they watched a demo made using the program created by Morita and Takeda.
The resulting image showed the object.
It showed a clear image of the stent.

Kazuhiro Mori

The next step was to install the program onto a device. However, no matter how much Morita, Mori, and Umeda worked on the problem, they were unable to create a device based on the proposed design concept that would be able to process the unprecedented massive amounts of data. It was Naoki Hasegawa, a seventh-year employee under Mori, who finally discovered how to solve the problem. Hasegawa realized that if processing took too much time, it would cause a delay in displaying images after the object moves. Therefore, he suggested a design that saved the minimum data required for performing the various processes in parallel. This meant that all the processes could be processed at the same time. Mori and Umeda realized the potential of that design and worked with Morita and Takeda to decide the specifications. Hasegawa and Takeda then fleshed out the details, which eventually resulted in the birth of the new SCORE PRO Advance image processing engine in April 2014. Not long after that, the new Trinias series equipped with that engine was released.

"Even when customers point out the attractiveness of images on competing systems, they recognize that the images on our system are free of the slight residual images that are visible on competing systems. Many users have praised our system" (Umeda).
Of course, this is not the end.
"To meet the needs of physicians who want work on even narrower blood vessels or more complicated medical cases, stents and wires have been getting narrower and narrower. We are very aware that unless we keep improving our image processing technology, our image quality will eventually become inadequate" (Mori).
The battle to achieve the image quality of the future will continue.

* Affiliates and titles of the persons in this article are current as of the time of reporting.

SCORE PRO Advance Image Processing Engine That Includes Algorithms for Processing Massive Amounts of Information