Background
Craniofacial researchers have utilized direct anthropometric measurements from the human head in both research and medical practice for decades. However, such measurements are inefficient in population studies involving thousands of individuals. Furthermore, the intricate complexity of craniofacial morphology isn’t adequately captured by simple Euclidean geometric measurements. With advancements in 3D surface imaging technologies, computational methods have been developed to diagnose craniofacial syndromes and analyze the craniofacial morphology. By integrating advanced computer vision and image analysis techniques, we can now achieve enhanced and automated diagnosis, as well as quantification, of craniofacial syndromes and their associated surgical treatments. These advancements enable further research into the intricate relationships between craniofacial morphology, genetic factors, environmental influences, and health implications.
Methodologies
3D photogrammetry
3D photogrammetry is an upcoming imaging modality that uses optical sensors to acquire multiple 2D images from different angles which are reconstructed into a 3D digital model of the subject’s surface.
In contemporary practice, photogrammetry captures craniofacial morphology, offering a safe, radiation-free way to monitor conditions and treatment effects over time. Using just two images and understanding the sensors’ relative positions, one can construct detailed 3D models.
While 3D photogrammetry has significantly impacted craniofacial assessments, its reach extends across varied medical domains. From evaluating dental implants and documenting skin injuries to assessment of the scoliotic spine and truncal deformities, its applications are vast.
The availability of high-quality 3D data is opening doors to innovative analysis techniques. As AI tools become more prevalent, various medical treatments and approaches are benefiting. Photogrammetry’s versatility positions it as a powerful non-invasive imaging option, suitable for both basic measurements, planning, and detailed analysis of the relationship between form and treatment in the field of craniofacial surgery.
3D mesh processing
For accurate analysis, it’s crucial to have a consistent dataset that’s free from noise and artifacts, and is aligned to a common reference frame with uniform resolution. Our integrated pipeline for mesh pre-processing includes fiducial registration for data alignment, mesh resampling to standardize image resolution, and region-focused clipping. The result is a streamlined mesh suitable for both basic anthropometric assessments and advanced shape analysis. This pipeline has been published as an open-source tool on Github and can also be run as a standalone executable. For access or more information see CraniumPy.
Automated extraction of head measurements from 3D images
Traditional head measurements like head circumference and cephalic index have long been integral to the pediatric physical examination, especially for children with congenital craniofacial conditions such as craniosynostosis. While these fundamental measurements can’t capture the intricate three-dimensional skull development, their simplicity, cost-effectiveness, and manual obtainability make them indispensable in clinical settings. Yet, the manual approach can introduce human error, and its acceptability might be challenging in very young children. Even with established protocols detailing the measurement process—like identifying specific skull regions and ensuring the measuring tape’s correct positioning—the accuracy often hinges on the practitioner’s expertise and judgment.
Patients with craniosynostosis treated at the Erasmus MC are monitored closely. In addition to visits to the specialist(s), 3D images are acquired during regular follow-up moments. In an attempt to relieve patient discomfort, minimize clinical visit durations, and enhance measurement reliability, we developed an algorithm to extract the most important head measurements from a 3D image in an automated and reproducible manner. These measurements do not only include the traditional head circumference and cephalic index but also an estimated intracranial volume (mesh volume above the nasion-tragus plane). Our validation study demonstrated that our algorithm can be used interchangeably with manual measurements, enhancing both consistency and patient comfort.
Automated measurement of facial asymmetry
We developed an automated pipeline for quantifying facial asymmetry from three-dimensional craniofacial images. The process begins by taking the facial shape of each subject as input, after which a mirrored reflection is generated. This mirrored image was then automatically aligned with the input via rigid registration. The vertex-wise distance between these two aligned facial shapes was then computed. Finally, two quantitative outcomes were produced for each subject: 1) a facial heatmap based on the distance where zero-error regions are perfectly symmetric while higher-error regions are more asymmetric; 2) a mean facial asymmetry (MFA) based on the mean of the error. This pipeline has been implemented as an inline function within CraniumPy.
Geometric deep learning on 3D mesh for derivation of endophenotypes
After the registration of the facial images, each facial mesh has the same vertex number and edge connectivity. Thus, a 3D graph convolutional autoencoder can be applied to these facial meshes for dimensionality reduction. The low-dimensional representations are defined as endophenotypes. By decoding (one, or multiple) these endophenotype(s) via the docoder, the visual representation of endophenotype(s) can be shown with a faical heatmap, where red areas refer to inward changes while blue areas refer to outward changes of the face with respect to the geometric center of the head. Conventional statistical analysis can be applied to these endophenotypes, in scenarios of epidemiological or genetic studies.
Applications
Insights into the effects of different surgical techniques on the post-operative cranial shape
Sagittal synostosis, a craniosynostosis subtype involving the premature fusion of the sagittal suture, presents a unique challenge in cranial surgery: while multiple interventions exist, there’s still no consensus on the optimal type and timing of treatment. In our recent study, we leverage the precision of 3D models to objectively evaluate the outcomes of different surgical techniques, highlighting the importance of post-operative growth patterns and any persistent cranial features. Analysis of axial, sagittal, and coronal slices showed that, although post-surgery head shapes tend to normalize, certain nuances remain. Early diagnosis is pivotal, influencing both surgical choice and long-term results. For a deeper insight into our findings and their implications, explore our research.
Epidemiological and genetics studies
Using geometric deep learning techniques, endophenotypes are derived from facial images, and used in population studies.
One example is the association between prenatal alcohol exposure and children’s facial shape, where a statistically significant association was found even in mild alcohol consumption less than 12 g alcohol (about a bottle of beer) per week. The statistically significant endophenotypes are mapped to the face via the decoder (described in 1.2), and therefore a better understanding of the detected association is enabled.
Participants (N=3,149) from the Generation R Study were included in the analysis.
The endophenotypes are also used in genetic analysis, where a genome-wide-association study (GWAS) is conducted. This project is in collaboration with department of genetic identification, which provides the combine-GWAS (C-GWAS) techniques. We thus propose a novel pipeline, which integrats AI-based endophenotypes and C-GWAS techniques, for GWAS with complex traits. In total of 43 genomic loci were identified, of which 12 are novel.
Participants from the Generation R Study (N=3,314) and the Rotterdam Study (N=3,995) were included in the analysis.
Publications
X Liu, S A Kushner, H Tiemeier, F Rivadeneira, V W V Jaddoe, W J Niessen, E B Wolvius, G V Roshchupkin. Association between prenatal alcohol exposure and children’s facial shape: a prospective population-based cohort study, Human Reproduction, Volume 38, Issue 5, May 2023, Pages 961–972 | https://doi.org/10.1093/humrep/dead006
T Abdel-Alim, M Kurniawan, I Mathijssen, M Dremmen, C Dirven, W Niessen, G Roshchupkin & ML van Veelen. Sagittal Craniosynostosis: Comparing Surgical Techniques using 3D Photogrammetry. Plastic and Reconstructive Surgery ():10.1097/PRS.0000000000010441, 152(4):p 675e-688e, October 2023. | https://doi.org/10.1097/prs.0000000000010441
T Abdel-Alim, P Tio, M Kurniawan, I Mathijssen, C Dirven, W Niessen, G Roshchupkin, ML van Veelen. Reliability and Agreement of Automated Head Measurements From 3-Dimensional Photogrammetry in Young Children. The Journal of Craniofacial Surgery 34(6):p 1629-1634, September 2023. | https://doi.org/10.1097/scs.0000000000009448
T Abdel-Alim, R Iping, E Wolvius, I Mathijssen, C Dirven, W Niessen, ML van Veelen, G Roshchupkin. Three-Dimensional Stereophotogrammetry in the Evaluation of Craniosynostosis: Current and Potential Use Cases. Journal of Craniofacial Surgery 32(3):p 956-963, May 2021. | https://doi.org/10.1097/scs.0000000000007379
X Liu, B Li, E E Bron, W J Niessen, E B Wolvius, G V Roshchupkin. Projection-Wise Disentangling for Fair and Interpretable Representation Learning: Application to 3D Facial Shape Analysis. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12905. Springer, Cham. https://doi.org/10.1007/978-3-030-87240-3_78
Code repositories
CraniumPy: An open-source tool for streamlined 3D craniofacial image processing and measurement extraction.
Description: CraniumPy is a user-friendly tool designed for registering 3D meshes (.ply, .obj, .stl) for cranial analysis using three landmarks. In its current state, meshes are registered to a template and then clipped based on the region of interest (either cranial or facial analysis). The tool employs Voronoi-based clustering for resampling, and a patching algorithm is implemented to identify and rectify imaging artifacts. After registration, cephalometric measurements can be automatically extracted.
Access: https://github.com/T-AbdelAlim/CraniumPy
LBOmeshfilter: 3D mesh reconstruction using selective precision from their surface harmonics.
Description: Every surface mesh can be characterized by a combination of its most prominent frequency components, or surface harmonics. The Laplace-Beltrami operator, is a discretized method to determine the surface harmonics at each vertex. LBOmeshfilter allows reconstruction of a 3D surface mesh using the first n eigenvectors, which can be useful for different mesh processing tasks.
Access: https://github.com/T-AbdelAlim/LBOmeshfilter
FairAssociation: An AI tool to examine and visualize confounder-free association in medical imaging
Description: AI has greatly enhanced medical image analysis, yet its use in epidemiological imaging studies remains limited due to visualization challenges in non-linear models (i.e., the black-box problem) and confounder control. Addressing this, we introduce an AI method emphasizing semantic feature interpretation and resilience against multiple confounders.
Access: https://github.com/tsingmessage/projection_wise_disentangling_FRL/
Internship opportunities
We’re always on the lookout for enthusiastic BSc and MSc students with an interest in the field of craniofacial shape analysis and image processing. If you’re drawn to areas like geometry processing, shape analysis, computer vision, or machine learning, we’d love to hear from you. Feel free to reach out to Tareq Abdel Alim (t.abdelalim@erasmusmc.nl) or Xianjing Liu (x.liu.1@erasmusmc.nl) to explore opportunities!
