4D US for improved image guidance in minimally invasive interventions

4D US for TIPS This project addresses the use of real-time 3D ultrasound for image guided liver interventions. Nowadays, 3D ultrasound is mainly used for diagnostic imaging. It however has large potential to be used for image guidance, but a 2D slice display or a 3D rendering, which currently are the standard visualizations, do not provide support for guidance. Purpose of the research presented in this thesis is to develop and evaluate image processing techniques that enable the effective use of 3D ultrasound for image guided interventions. In particular we focussed on TIPS (transjugular intrahepatic portosystemic shunt) intervention, one of the most complex interventions in radiology, where a needle needs to puncture from an hepatic vein into a portal vein through the liver parenchyma. Conventionally, this intervention is guided with X-ray imaging, which does not show the needle position in 3D, and also needs additional equipment to visualize the portal vein, or 2D ultrasound, which is complex as well, as it requires a skilled 2nd clinician to manipulate the transducer in a coordinated way with the interventional radiologist. 3D ultrasound has the potential to visualize the needle and the puncture trajectory (including hepatic artery and target portal vein) simultaneously.

Together with TU Delft (Industrial Design) and clinical experts, a workflow and visualization modes for this intervention were designed, which were integrated in a 3D Ultrasound image-based navigation system for TIPS. Developing the enabling image processing technology was the main purpose of this project. To this end, we worked on the following four subjects:

  • Rotationally invariant feature extraction from 3D US images, based on local binary patterns

  • Fast 3D ultrasound registration, in a novel framework that is based on block-matching and game-theory-based outlier rejection methods, which was implemented on a GPU system.

  • A two-stage 3D US tracking approach based on the fast 3D ultrasound registration. This approach, which was able to align pre-operative 3D US planning data with live 3D US images in real-time, was integrated in a system for ultrasound-guided TIPS (Transjugular Intrahepatic Portosystemic Shunt) interventions. The approach was also used for and evaluated in a 3D US tracking challenge at MICCAI 2015.

3D Local Binary Patterns Banerjee et al., ACCV 2012 Workshop : Local binary patterns are popular texture descriptors in many image analysis tasks. One of the important aspects of this texture descriptor is their rotational invariance. Most work in LBP has focused on 2D images. Here, we present a three dimensional LBP with a rotational invariant operator using spherical harmonics. Unlike Fehr and Burkhardt, the invariance is constructed implicitly, without considering all possible combinations of the pattern. We demonstrate the 3D LBP on phantom data and a clinical CTA dataset.

3D US registration Banerjee et al., MedIA 2015 : Real-time 3D US has potential for image guidance in minimally invasive liver interventions. However, motion caused by patient breathing makes it hard to visualize a localized area, and to maintain alignment with pre-operative information. In this work we develop a fast affine registration framework to compensate in real-time for liver motion/displacement due to breathing. The affine registration of two consecutive ultrasound volumes in time is performed using block-matching. For a set of evenly distributed points in one volume and their correspondences in the other volume, we propose a robust outlier rejection method to reject false matches. The inliers are then used to determine the affine transformation. The approach is evaluated on 13 4D ultrasound sequences acquired from 8 subjects. For 91 pairs of 3D ultrasound volumes selected from these sequences, a mean registration error of 1.8 mm is achieved. A graphics processing unit (GPU) implementation runs the 3D US registration at 8 Hz.

4D US for TIPS Banerjee et al., TMI 2016 : In this work we describe a 4D registration method for on the fly stabilization of ultrasound volumes for improving image guidance for transjugular intrahepatic portosystemic shunt (TIPS) interventions. The purpose of the method is to enable a continuous visualization of the relevant anatomical planes (determined in a planning stage) in a free breathing patient during the intervention. This requires registration of the planning information to the interventional images, which is achieved in two steps. In the first step tracking is performed across the streaming input. An approximate transformation between the reference image and the incoming image is estimated by composing the intermediate transformations obtained from the tracking. In the second step a subsequent registration is performed between the reference image and the approximately transformed incoming image to account for the accumulation of error. The two step approach helps in reducing the search range and is robust under rotation. We additionally present an approach to initialize and verify the registration. Verification is required when the reference image (containing planning information) is acquired in the past and is not part of the (interventional) 4D ultrasound sequence. The verification score will help in invalidating the registration outcome, for instance, in the case of insufficient overlap or information between the registering images due to probe motion or loss of contact, respectively. We evaluate the method over thirteen 4D US sequences acquired from eight subjects. A graphics processing unit implementation runs the 4D tracking at 9 Hz with a mean registration error of 1.7 mm.

Associated People

  • Joytirmoy Banerjee
  • Cecile Cuijpers (TU Delft, Industrial Design)
  • Adinda Freudenthal (TU Delft, Industrial Design)
  • Camiel Klink
  • Adriaan Moelker
  • Wolf Song (TU Delft, Industrial Design)
  • Edit Varga (TU Delft, Industrial Design)
  • Theo van Walsum