About the research topic
This project addresses the use of real-time 3D ultrasound for
image guided liver interventions. Nowadays, 3D ultrasound is mainly used for
diagnostic imaging. It however has large potential to be used for image
guidance, but a 2D slice display or a 3D rendering, which currently are the
standard visualizations, do not provide support for guidance. Purpose of the
research presented in this thesis is to develop and evaluate image processing
techniques that enable the effective use of 3D ultrasound for image guided
interventions. In particular we focussed on TIPS (transjugular intrahepatic
portosystemic shunt) intervention, one of the most complex interventions in
radiology, where a needle needs to puncture from an hepatic vein into a portal
vein through the liver parenchyma. Conventionally, this intervention is guided
with X-ray imaging, which does not show the needle position in 3D, and also
needs additional equipment to visualize the portal vein, or 2D ultrasound, which
is complex as well, as it requires a skilled 2nd clinician to manipulate the
transducer in a coordinated way with the interventional radiologist. 3D
ultrasound has the potential to visualize the needle and the puncture trajectory
(including hepatic artery and target portal vein) simultaneously.
Together with TU Delft (Industrial Design) and clinical experts, a workflow and visualization modes for this intervention were designed, which were integrated in a 3D Ultrasound image-based navigation system for TIPS. Developing the enabling image processing technology was the main purpose of this project. To this end, we worked on the following four subjects:
-
Rotationally invariant feature extraction from 3D US images, based on local binary patterns
-
Fast 3D ultrasound registration, in a novel framework that is based on block-matching and game-theory-based outlier rejection methods, which was implemented on a GPU system.
-
A two-stage 3D US tracking approach based on the fast 3D ultrasound registration. This approach, which was able to align pre-operative 3D US planning data with live 3D US images in real-time, was integrated in a system for ultrasound-guided TIPS (Transjugular Intrahepatic Portosystemic Shunt) interventions. The approach was also used for and evaluated in a 3D US tracking challenge at MICCAI 2015.
Output
3D LBP-Based Rotationally Invariant Region Description, Banerjee et al., ACCV 2012 Workshop : Local binary
patterns are popular texture descriptors in many image analysis tasks. One of
the important aspects of this texture descriptor is their rotational invariance.
Most work in LBP has focused on 2D images. Here, we present a three dimensional
LBP with a rotational invariant operator using spherical harmonics. Unlike Fehr
and Burkhardt, the invariance is constructed implicitly, without considering all
possible combinations of the pattern. We demonstrate the 3D LBP on phantom data
and a clinical CTA dataset.
Fast and robust 3D ultrasound registration – Block and game theoretic matching Banerjee et al., MedIA
2015 :
Real-time 3D US has potential for image guidance in minimally invasive liver
interventions. However, motion caused by patient breathing makes it hard to
visualize a localized area, and to maintain alignment with pre-operative
information. In this work we develop a fast affine registration framework to
compensate in real-time for liver motion/displacement due to breathing. The
affine registration of two consecutive ultrasound volumes in time is performed
using block-matching. For a set of evenly distributed points in one volume and
their correspondences in the other volume, we propose a robust outlier
rejection method to reject false matches. The inliers are then used to
determine the affine transformation. The approach is evaluated on 13 4D
ultrasound sequences acquired from 8 subjects. For 91 pairs of 3D ultrasound
volumes selected from these sequences, a mean registration error of 1.8 mm is
achieved. A graphics processing unit (GPU) implementation runs the 3D US
registration at 8 Hz.
4D Ultrasound Tracking of Liver and its Verification for TIPS Guidance Banerjee
et al., TMI 2016: In this work we describe a
4D registration method for on the fly stabilization of ultrasound volumes for
improving image guidance for transjugular intrahepatic portosystemic shunt
(TIPS) interventions. The purpose of the method is to enable a continuous
visualization of the relevant anatomical planes (determined in a planning
stage) in a free breathing patient during the intervention. This requires
registration of the planning information to the interventional images, which is
achieved in two steps. In the first step tracking is performed across the
streaming input. An approximate transformation between the reference image and
the incoming image is estimated by composing the intermediate transformations
obtained from the tracking. In the second step a subsequent registration is
performed between the reference image and the approximately transformed
incoming image to account for the accumulation of error. The two step approach
helps in reducing the search range and is robust under rotation. We
additionally present an approach to initialize and verify the registration.
Verification is required when the reference image (containing planning
information) is acquired in the past and is not part of the (interventional) 4D
ultrasound sequence. The verification score will help in invalidating the
registration outcome, for instance, in the case of insufficient overlap or
information between the registering images due to probe motion or loss of
contact, respectively. We evaluate the method over thirteen 4D US sequences
acquired from eight subjects. A graphics processing unit implementation runs
the 4D tracking at 9 Hz with a mean registration error of 1.7 mm.
