Profile
Karsten Roth is a PhD researcher at the Explainable Machine Learning group as part of the European Laboratory for Learning and Intelligent Systems (ELLIS) and the International Max-Planck Research School for Intelligent Systems (IMPRS-IS) co-supervised by Prof. Zeynep Akata and Hon. Prof. Oriol Vinyals at Google DeepMind. He is supported by the Qualcomm Innovation Fellowship 2023.
Karsten is currently working at Google DeepMind London as part of his ELLIS exchange on large multimodal models. He has completed both Bachelor and Master studies in Physics at Heidelberg University (2021). During that time, Karsten spent time abroad in Canada as a research intern at the Montreal Institute for Learning Algorithms (MILA) supervised by Dr. Joseph Paul Cohen and Prof. Yoshua Bengio, and the Vector Institute supervised by Prof. Marzyeh Ghassemi, working on all manners of representation learning and their applications to the medical domain.
As a research intern, Karsten has also worked at the Amazon AWS research lablet in Tuebingen on Anomaly Detection with Peter Gehler and Thomas Brox, and Meta AI in Paris on Disentangled Representation Learning with Mark Ibrahim, Pascal Vincent and Diane Bouchacourt.
His primary interest: Making models generalize better. This covers approaches to effective representation learning under different forms of distribution shifts, including zero-shot, few-shot and continual learning problems, as well as understanding generalisation behaviour of learned (multimodal) representations and foundation models. He is also very interested in their application to medicine and the sciences.
Publications
"A Practitioner's Guide to Continual Multimodal Pretraining"
Karsten Roth *, Vishaal Udandarao *, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals, Olivier Hénaff, Samuel Albanie, Matthias Bethge, Zeynep Akata
Thirty-eighth Annual Conference on Neural Information Processing Systems, NeurIPS 2024
"ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization"
Luca Eyring *, Shyamgopal Karthik *, Karsten Roth, Alexey Dosovitskiy, Zeynep Akata
Thirty-eighth Annual Conference on Neural Information Processing Systems, NeurIPS 2024
"Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Models"
Nishad Singhi, Jae Myung Kim, Karsten Roth, Zeynep Akata
European Conference on Computer Vision, ECCV 2024 (supervised Master Thesis)
"Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models"
Lukas Thede *, Karsten Roth *, Olivier J. Hénaff, Matthias Bethge, Zeynep Akata
Conference on Lifelong Learning Agents, CoLLAs 2024
"ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections"
Massimo Bini, Karsten Roth, Zeynep Akata, Anna Khoreva
International Conference on Machine Learning, ICML 2024
"Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model"
Karsten Roth *, Lukas Thede *, A. Sophia Koepke, Oriol Vinyals, Olivier Henaff, Zeynep Akata
Spotlight at International Conference on Learning Representations, ICLR 2024
"Vision-by-Language for Training-Free Compositional Image Retrieval"
Shyamgopal Karthik *, Karsten Roth *, Massimilano Mancini, Zeynep Akata
International Conference on Learning Representations, ICLR 2024
"Waffling around for Performance: Visual Classification with Random Words and Broad Concepts"
Karsten Roth *, Jae Myung Kim *, Almut Sophia Koepke, Oriol Vinyals, Cordelia Schmid, Zeynep Akata
IEEE International Conference for Computer Vision, ICCV 2023
"Disentanglement of Correlated Factors via Hausdorff Factorized Support"
Karsten Roth, Mark Ibrahim, Zeynep Akata, Pascal Vincent *, Diane Bouchacourt *
International Conference on Learning Representations, ICLR 2023
"Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning"
Zafir Stojanovski *, Karsten Roth *, Zeynep Akata
Best Paper at INTERPOLATE Workshop @ Conference on Neural Information Processing Systems, NeurIPS 2022
Result of Master Thesis
"A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning"
Michael Kirchhof *, Karsten Roth *, Zeynep Akata, Enkelejda Kasneci
European Conference on Computer Vision, ECCV 2022
"Uniform Priors for Data-Efficient Learning"
Samarth Sinha *, Karsten Roth *, Anirudh Goyal, Marzyeh Ghassemi, Zeynep Akata, Hugo Larochelle, Animesh Garg
Workshop on Learning with Limited Labels @ IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2022
"Non-isotropy Regularization for Proxy-based Deep Metric Learning"
Karsten Roth, Oriol Vinyals, Zeynep Akata
IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2022
"Integrating Language Guidance into Vision-based Deep Metric Learning"
Karsten Roth, Oriol Vinyals, Zeynep Akata
Oral at IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2022
Community Service
Reviewer for
- ECCV '22 & '24, CVPR '22 & '23, ICLR '24 (all outstanding reviewer)
- MICCAI '20,'21, ICCV '23, IJCV & TPAMI since '22
- NeurIPS '23 & '24, ICML '24 & '25, CVPR '24
- Lots of CVPR/NeurIPS workshops
Supervision
Master Theses and Research Projects:
- Realistic Concept Bottleneck Models, Nishad Singhi. Master Thesis April 2024, with Jae Myung Kim. Workshop Paper, Conference Submission.
- Effective Decoder-free Disentanglement, Jasper Touissant. Research Project (with Shyamgopal Karthik) Nov 2023.
- Topological and Equivariant Self-Supervised Learning, Madhav Iyengar. Master Thesis Nov 2023.
- Improving Industrial Anomaly Detection, Stefan Wezel. Master Thesis July 2023. Collaboration with Maddox AI. Workshop Paper.
- Continual Learning with Foundation Models, Zafir Stojanovski. Master Thesis May 2023. Workshop Best Paper.
- Scene Sketch-based Image Retrieval, Jessica Bader. Master Thesis May 2023.
- Topological Properties of Metric Spaces, Zafir Stojanovski, Jessica Bader. Research Project 2023.