Inverse problems in biomedical imaging

Ivan Ezhov · Sevgi Gokce Kafali

Our group is working on inverse problems in biomedical imaging and their solution using artificial intelligence and machine learning.

The development of algorithms to solve inverse problems arising in sensor and imaging systems has a long tradition. Examples include compressed sensing approaches, e.g. for medical and computational imaging. Until recently, most algorithms for inverse problems were based on statistical or physical signal models, such as wavelets or sparse representations. Our research focuses on novel approaches based on deep learning to accelerate solving such problems.

We study how these deep learning-based approaches can be optimized for clinical applications and how they can be combined with image analysis methods. Deep learning-based approaches for reconstructing magnetic resonance imaging (MRI) or computed tomography (CT) provide efficient AI models, allowing the reconstruction of high-quality MRI images, and high-quality CT images from low-dose X-ray images. Recent works on generative models have shown great promise for accelerating reconstruction tasks to shorten the scan time in MRI, as well as generating images with much higher resolution than the acquired resolution (e.g. super-resolution). Here, we tackle these problems by utilizing AI (i.e., diffusion models) guided by readily available MR images from other organs/tissues, MRI scanning parameters, or other MRI physics-guided information.

Privacy-preserving and trustworthy AI in medicine

Alexander Ziller

Our group is developing the next generation of privacy-preserving, secure, and trustworthy AI algorithms for medical applications.

AI in medicine requires large, diverse, and representative datasets to train fair, generalizable, and reliable models. However, such datasets often contain sensitive personal information. Privacy-preserving machine learning bridges the gap between data utilization and data protection by enabling the training of AI models on private data while providing formal privacy guarantees. Our group focuses on:

• Differential privacy (DP) theory and applications to machine learning and deep learning, targeting both unstructured datasets (e.g., images) and structured data (e.g., tabular and graph databases).
• Generative models and their applications, such as large language models (LLMs) and agentic systems, integrating differential privacy to ensure secure and privacy-preserving outcomes.
• Data attribution techniques, which enable transparent and accountable data usage in training and inference.
• Developing techniques to mitigate trade-offs between privacy, model utility, and computational efficiency.
• AI security, including the study of vulnerabilities in collaborative machine learning protocols (e.g., federated learning) and designing robust defense mechanisms against adversarial attacks.

Building trust in AI necessitates a comprehensive approach encompassing privacy, reliability, and security. Our work on trustworthy machine learning includes quantifying uncertainty in model outputs, incorporating domain expertise, developing probabilistic models to counteract poorly calibrated predictions, employing computational Bayesian techniques, and exploring the intersection of probabilistic and privacy-preserving machine learning. As AI systems increasingly integrate generative models like LLMs, we also work on establishing formal guarantees of safety and reliability for agentic LLM applications, exploring robustness in generative systems, and ensuring alignment of generative models with human values and ethical guidelines, particularly in high-stakes domains like healthcare.

Multimodal AI

Philip Müller

Our group develops advanced machine learning methods to integrate and analyze diverse medical data, ranging from imaging modalities such as X-ray, CT, ultrasound, and MRI to signals such as ECG and EEG to lab tests and textual reports. We are working on:

Multimodal representation learning and information transfer. Clinical data is inherently multimodal, with each modality providing unique but complementary information. Some modalities, while highly informative, are invasive, expensive, or unavailable in many clinical settings. By aligning representations across modalities, we enable the transfer of information from stronger but less accessible modalities to weaker yet readily available ones, improving cross-modal phenotype prediction and early risk assessment, while also enabling cross-retrieval and simplifying multimodal analysis. We develop representation learning methods that align modalities into shared spaces and transfer information between them. We demonstrate their utility in clinical applications across diverse medical disciplines, with a specific focus on cardiovascular medicine.

Multimodal fusion. Critical diagnostic information is often distributed across multiple medical modalities, each providing an incomplete view of a patient’s health. Combining them can reveal latent multimodal signals and improve diagnostic and prognostic accuracy, but modalities are heterogeneous in structure and information density, ranging from 2D imaging, volumetric data, and videos to signals and text. We develop fusion mechanisms and architectures that handle this heterogeneity, along with unsupervised training and pretraining methods, and robustness to missing modalities, applying these across diverse medical disciplines.

Grounded vision-language modeling. Automatic report generation promises to replace the manual and often time-intensive writing of clinical reports, but seamless integration into clinical workflows requires models that avoid hallucinations and produce interpretable predictions that clinicians can verify. We develop vision-language models that explicitly ground inputs in the input modalities, for example through anatomical regions or pathological clues, and produce interpretable outputs such as localization and structured predictions. Our work also targets interactive human-AI collaboration and computational efficiency to enable the integration of large or multiple modalities.

AI for vision

Martin Menten

The AI for Vision group focuses on blue-sky research in medical image analysis with a particular focus on the application of machine learning and computer vision algorithms in the field of ophthalmology. Specifically, we are working on:

Self-supervised learning. Labeling medical data is very expensive, as it is time-consuming and requires expert knowledge. Moreover, medical data often includes highly sensitive information, making it challenging to share without compromising the privacy of the subjects involved. To overcome the limited availability of large annotated medical datasets, we are researching self-supervised learning, leveraging unlabeled medical data to enable neural networks to extract meaningful features that can be effectively adapted to a wide range of downstream tasks.

Multimodal deep learning. Clinicians rarely rely on a single source of information when diagnosing patients and deciding on a course of action. They consider an array of multimodal data, such as demographic and genomic information, patient interviews, laboratory test results and biomedical images. Our research focuses on developing deep learning algorithms capable of integrating diverse multimodal data to support autonomous and effective clinical decision making.

Deep learning for ophthalmology. Good vision is essential for navigating our environment, communicating, and performing everyday activities. As of 2020, more than 200 million people worldwide suffered from moderate to severe vision impairment. Driven by the comparative ease of imaging the eye and obtaining large imaging datasets, ophthalmology has been an early adopter of deep learning in healthcare. Our group’s work in machine learning for ophthalmology simultaneously evaluates new algorithmic innovations while aiming to improve medical care for patients affected by ocular diseases.

Agents and World Models in Health

Jiazhen Pan

Our group develops AI methods for medicine and healthcare along two complementary directions: advancing large language models and AI agents for medical applications, and building biomedical world models to understand biological systems and predict their responses to interventions.

LLMs and agents for medicine and healthcare. We investigate how large language models and multimodal models can support medical reasoning, patient interaction, and clinical workflows. Our research focuses on systems that integrate medical images, clinical records, and biomedical knowledge, gather relevant evidence, and use specialised tools to complete complex tasks. We explore reinforcement learning, reasoning, and agent collaboration to improve their capabilities, with applications including medical dialogue, imaging analysis, and clinical decision support. We also develop benchmarks and auditing methods to assess their reliability in realistic healthcare settings.

Biomedical world models. Our long-term goal is to build models that capture how biological systems evolve and respond to interventions. We aim to combine multimodal biomedical data, biological knowledge, and mechanistic and learned models to connect processes across cells, organs, and patients. This includes exploring digital organisms in which agents coordinate specialised models for simulation and scientific investigation. Potential applications include studying disease progression, comparing candidate interventions, and guiding experiments. A central research question is how to establish when these predictions are sufficiently supported by evidence for their intended use.

Across both directions, we emphasise rigorous evaluation, uncertainty, and clinical relevance. Our aim is to develop AI systems that support clinicians and researchers, with capabilities and limitations that can be clearly assessed.