Abstract: Bone tumor classification presents significant challenges due to the subtle visual differences among tumor entities, even for expert radiologists. This thesis aims to enhance diagnostic capabilities using vision-language pretraining to classify bone tumors from X-ray images.

Deep-learning models achieve high performance on various biomedical prediction tasks but often rely on extensive annotated datasets. Especially in biomedicine, such datasets are frequently unavailable due to the difficulty and cost of annotation.

Hyperspectral imaging (HSI) is an optical technique that processes the electromagnetic spectrum at a multitude of monochromatic, adjacent frequency bands. The wide-bandwidth spectral signature of a target object’s reflectance allows fingerprinting its physical, biochemical, and physiological properties.

Description Image segmentation seeks to classify individual pixels in an image into semantic classes, such as e.g. the organs in a CT scan. State-of-the-art approaches to image segmentation [5], while very accurate, have limitations in terms of topological correctness.

Description: Large Language Models (LLMs) have shown exceptional capabilities in understanding and generating human-like text. In the medical field, these models hold the potential to revolutionize patient care, medical research, and healthcare administration.

Description Equivariant convolutions are a novel approach that incorporate additional geometric properties of the input domain during the convolution process (i.e. symmetry properties such as rotations and reflections) [1]. This additional inductive bias allows the model to learn more robust and general features from less data, rendering them highly promising for application in the medical domain.

This year’s seminar will look at aspects of multi-modal machine learning in medicine and healthcare, focusing on: Vision language models (VLMs) for medical and healthcare applications Generic multi-modal AI models utilising imaging data, clinical reports, lab test results, electronic health records, and genomics Foundation models for multi-modal medicine Objectives: At the end of the module students should have:

In this course students are given the chance to apply their abilities and knowledge in deep learning to real-world medical data. Students will be assigned a medical dataset and in close consultation with medical doctors create a project plan.