TractoFormer: A Novel Fiber-level Whole Brain Tractography Analysis Framework Using Spectral Embedding and Vision Transformers

[Audio] Hello everyone, in this presentation, we will introduce our work on, TractoFormer, A Novel Fiber-level Whole Brain Tractography Analysis Framework Using Spectral Embedding and Vision Transformers..

[Audio] Diffusion MRI tractography is an advanced imaging technique that enables in vivo reconstruction of the brain's white matter connections, and it provides an important tool for quantitative mapping of the brain's connectivity to study the brain in health and disease. Currently, there are several challenges when performing disease classification using whole brain tractography data and machine learning. First, performing whole brain tractography on one individual subject can generate hundreds of thousands of fiber streamlines. Defining a good data representation of tractography for machine learning is still an open challenge, especially at the fiber level. The second challenge is the limited sample size, i.e., number of subjects. Small sample sizes limit the use of recently proposed advanced learning techniques such as Vision Transformers, which are highly accurate but usually require many samples to avoid overfitting. The third challenge is about result interpretability to pinpoint location(s) in the brain that are predictive of disease or affected by disease..

[Audio] In this paper, we propose a novel parcellation-free whole brain tractography analysis framework, TractoFormer, that leverages tractography information at the level of individual fiber streamlines and provides a natural mechanism for interpretation of results using the self-attention scheme of vision transformers. TractoFormer includes two main contributions. First, we propose a novel 2D image representation of WBT, referred to as TractoEmbedding, based on a spectral embedding of fibers from tractography. Second, we propose a ViT-based network that performs effective and interpretable group classification..

[Audio] The TractoEmbedding process includes three major steps. First, we perform spectral embedding to represent each fiber in the whole brain tractography data as a point in a latent space, where nearby points correspond to spatially proximate fibers. Second, the coordinates of each fiber of the tractography data are discretized onto a 2D grid for creation of an image. In our study, we choose the first two dimensions for each point and discretize them onto a 2D embedding gird. Third, we map the measure of interest associated with each fiber to the corresponding pixel on the embedding grid as its intensity value. This generates a 2D image, i.e., the TractoEmbedding image..

[Audio] Then, based on the TractoEmbedding images, we propose ViT-based Framework for group classification. Our design aims to address the aforementioned challenges of sample size and interpretability. First, we leverage the multi-sample data augmentation to reduce the known overfitting issue of ViTs on small sample size datasets. One important feature of out framework is that multiple TractoEmbedding images can be generated from one subject by performing random downsampling of the whole brain tractography data to increase sample size. Second, we leverage the self-attention scheme in ViT for result interpretation. The ViT attention maps are aided by our proposed multi-channel architecture, which can enable inspection of the independent contributions of different brain regions. As a result, we can identify the fibers that are discriminative for result interpretation..

[Audio] We first perform an experiment using synthetic data to provide a proof-of-concept evaluation. To do so, we create a realistic synthetic dataset with true group differences in the corticospinal tract. For results, TractoFormer achieved, as expected, 100% group classification accuracy because of the added synthetic feature changes to G2. The identified discriminative fibers are generally similar to the CST fibers with synthetic changes..

[Audio] In the second experiment, we performed disease classification between heathy controls and schizophrenia. The goal is to evaluate the TractoFormer in a real neuroscientific application for brain disease classification. The evaluation dataset includes data from 103 healthy controls and 47 schizophrenia patients., We compare our method with several baseline methods. As shown in Table 1 our method gave the best overall quantitative result. The figure gives a visualization of the discriminative fibers, showing that the superficial fibers in the frontal and parietal lobes have high importance when performing classification..

[Audio] In this paper, we present a novel parcellation-free whole brain tractography analysis framework, TractoFormer. Overall, TractoFormer suggests the potential for deep learning analysis of whole brain tractography represented as images..

[Audio] Thank you for your attention.. Thank you!.