AI and Simulations for Decoding the Spatiotemporal Dynamics of Immune Responses

Timeframe: 2023 — 2028
Financially supported by: NWO (Dutch Research Council)

From autoimmunity to defence against viruses or cancer, human immune responses are highly dynamic. Immune cells critically rely on adapting their motion as they navigate the maze of tissues in the human body, interact and communicate with other cells, search for signs of anomalies, and swarm to sites of infection. Understanding the spatiotemporal dynamics of cell motion is key to many areas of immunology and beyond. Special microscopes can record these processes in time-lapse images that are rich in information but limited in the amount of (annotated) data for any given application. As a result, existing AI techniques are not directly applicable and — as of yet — no robust methods exist to decode these time-lapse images and zoom in on the where, when, and how of key immunological events. Up until now, researchers have failed to leverage a key resource for understanding cell dynamics: knowledge of the underlying biophysics. In this project, we combine AI with simulations capturing biophysics to overcome the current impasse in exploiting cell movement data. Specifically, the project integrates data-driven and model-driven approaches into hybrid AI, where simulations complement AI in several ways: simulations provide unlimited synthetic data, ways to inject prior knowledge into AI components, a platform to validate the developed approaches, and a link back from observed data patterns to interpretable biophysical parameters. Vice versa, AI provides a way to detect and extract new patterns from the data to create better simulation models.

Multimodal data fusion to guide the development of immunotherapies for uveal melanoma patients

Timeframe: 2022 — 2025
Financially supported by: Hanarth Foundation

Uveal melanoma occurs only in ~200 patients a year in the Netherlands and up to half of the patients with a primary uveal melanoma will develop metastases. Uveal melanoma differs clinically and genetically from cutaneous melanoma and response rates to immunotherapy are low. Previously, it was believed that the development of UM at an immune-privileged site, the eye, but infiltration of immune cells in uveal melanoma is present. In this project, we will analyze multiplex immunohistochemistry imaging and flow cytometry data of multiple patient cohorts to investigate tumor-immune interactions. To avoid the risk of overfitting our real data and overcome the challenges with multimodal data fusion, we will first develop and optimize our analysis approach using simulated data resembling real data. By optimal analysis of the combined data from multiple heterogeneous sources, we aim to obtain mechanistical understanding of immunological processes which will help develop strong theoretical rationales for clinical trials in the future to improve the outcome of uveal melanoma patients.

Transfer Learning for Immune Cell Landscape Analysis in Salivary Gland Cancer

Timeframe: 2021 — 2025
Financially supported by: Hanarth Foundation

Salivary gland cancer is a rare type of head-and-neck cancer with 150-200 diagnoses per year in the Netherlands, and the most aggressive subtypes have poor prognosis. To develop new treatment options, we are imaging the interactions between immune system cells and tumor cells within patient biopsies using high-resolution digital microscopy. Machine learning approaches are the state of the art for analyzing such data, but they can require very large datasets to train on, which are usually not available for rare cancer types. In our project, we will address this problem using 'transfer learning' methodology that allows machine learning algorithms to benefit from experience gained on larger datasets from more common cancer types and train more effectively on smaller datasets. Leveraging existing data and knowledge in this manner, we hope that our project will help to build a rationale for future immunotherapy treatments for salivary gland caner patients.

T Cell Crowd Control

Timeframe: 2021 — 2024
Financially supported by: Human Frontiers Science Program

T cells are key effectors of adaptive immunity that are constantly moving, can enter most tissues, and operate in large crowds. Millions of densely packed T cells continuously roam through lymphatic tissue in search of foreign antigen; activated T cells divide vigorously, flock to tissues, mount local immune responses to pathogens or cancer cells, and remain there to guard against further intrusions. Crowding is normally expected to deter motion and lead to jamming, an effect confirmed in very diverse systems ranging from pedestrians and vehicular traffic to cellular monolayers and granular material. Yet T cells defy this general principle and are capable of fluid motion even in extremely dense environments such as lymphatic tissue and the epidermis, where there is no apparent room to maneuver. How they achieve this is unclear: although novel microscopy modalities can visualize migrating T cells in living tissue, studies of T cell movement have primarily focused on individual or few cells and have not addressed potential crowding effects. We hypothesize that (1) T cells have evolved to cooperate effectively in large groups and avoid crowding issues, such that (2) impaired T cell crowd operation is largely confined to anomalous tissue environments such as tumors. Here, we aim to unravel the mechanisms that facilitate fluid, “jam-free” motion of densely packed T cells, and determine in which situations T cell traffic might nevertheless be disturbed by emerging detrimental crowding effects. We will study T cell crowds in a wide range of conditions in silico, in vitro and in vivo: we will (1) conduct simulations to predict how T cell crowds operate in challenging conditions and which emerging crowd behavior is expected; (2) design microfluidic devices to expose T cell crowds to hallmark crowding scenarios, identify molecules involved in cellular crosstalk within crowds, and identify transcriptional signatures associated with efficient crowd operation; (3) examine T cell crowds in human and mouse tissues to test whether dynamic or static patterns indicating jamming can be confirmed in physiological or pathological conditions in vivo. Thus, by combining our unique joint expertise in crowd dynamics, immunology, and computational biology, we will shed light on how T cell crowds, a system quite unlike any other many-particle system studied so far, maintain motion with such remarkable efficiency in many conditions and what it would take to disrupt their smooth flow.

Artificial Immunological Intelligence

Timeframe: 2020 — 2025
Financially supported by: NWO (Dutch Research Council)

Our body harbours two cognitive systems: the central nervous system (CNS) and the adap- tive immune system (IS), which consists of repertoires of very diverse cells that recognize highly specific molecular patterns. The CNS has inspired artificial neural networks, which have matured into powerful machine learning systems. IS-inspired artificial intelligence (AI) is far less developed: existing computational IS repertoire models scale poorly and cannot process high-dimensional data. In previous work, our group has cleared this roadblock by creating an algorithmic framework based on finite state machines that scales up repertoire models by several orders of magnitude. We are now going to harness these techniques to seek answers to a fundamental question: how does the IS learn?

We will address this question in two very different ways. On one hand, we will build realistic-scale simulation models of how T cell repertoires develop and respond to pathogens. On the other hand, we will build the first IS repertoire models that can process any kind of text and image data, and train these models to perform classification tasks such as recognizing hand-written numbers. Contrasting the way in which the IS solves such well-studied machine learning problems to other methods like neural networks, we will pinpoint how the IS achieves learning targets such as generalization, memory, and robustness against adversarial information. Due to its unique architecture, the IS may achieve these targets in very different ways than other systems like the brain. Such insights could help to better understand and ultimately predict T cell immune responses to viral, bacterial or cancer antigens, which would be very useful for informing vaccine design and cancer immunotherapies.

Thus, drawing on our group's expertise in AI, immunology, and algorithms, this project aims to deliver large-scale models of the adaptive IS. Accomplishing this would habe broad implications for both AI and immunology.

The ImmuNet

Timeframe: 2017 — 2021
Financially supported by: KWF (Dutch Cancer Research Foundation)

Despite the increasing success of cancer immunotherapies, not all patients benefit from such treatments. Few biomarkers that predict treatment outcome have been found yet, and a consensus is emerging that multiple immune-related parameters will have to be integrated for optimizing personalized therapy. Relevant data items can be simple, like cell counts, or complex, like tumour samples; and they can be static, like genetic information, or dynamic, like lymphocyte phenotypes. Integrating the enormous amount of diverse data generated by current technology into a coherent diagnostic framework is a challenging task. It requires not only the best possible pre-clinical and clinical science, but also the best possible data science.

Causal Bayesian networks are a popular methodology for integrating heterogeneous data. Our group has extensive experience in this area and has built methodology and software that is now widely used by researchers from various fields, especially in the context of complex observational studies. Attractive characteristics of causal networks include that (1) they are modular, allowing smooth extension of existing networks with new diagnostic items; (2) they can deal with incomplete information; (3) they can distinguish between observational data (such as retrospective studies) and interventional data (from randomized trials), a feature not offered by statistical methods currently used in oncology; and (4) they are intuitive, allowing humans to follow the reasoning made by the network and safeguarding against unintended consequences.

Our project aims to build an “ImmuNet” – a causal network model of what determines patient response to tumour immunotherapy. The ImmuNet will contain all available routine diagnostic items, as well as two complex immunological elements: digital immunohistochemistry images of the tumour microenvironment, and flow cytometry measurements of immune cell surface markers. This will allow a more comprehensive assessment of a patient’s immune system before and during therapy. The ImmuNet will synthesize all available diagnostic information that explains variation in treatment outcomes between different patients, and use this information to provide a reasonable prognosis about the outcome for future patients. Importantly, it will also quantify the uncertainty of its prognosis, especially when some diagnostic items are missing. The ultimate goal is to assist doctors with the use of complex data for personalized therapy choices. I will make the ImmuNet and related tools resulting from this project available as open-source software for the benefit of the entire oncological community.