Alexander Timans

Mail | LinkedIn | Google Scholar | GitHub | CV

Last updated: November 2024

About me

I am an ELLIS PhD candidate at the University of Amsterdam in the Machine Learning Lab, jointly supervised by Eric Nalisnick (from Johns Hopkins) and Christian Naesseth. I am part of the Delta Lab, a research collaboration with the Bosch Center for Artificial Intelligence. In that context, I am also advised by Bosch research scientists Christoph-Nikolas Straehle and Kaspar Sakmann.

My research interests focus on principled and efficient uncertainty quantification for deep learning, with the goal of facilitating model reasoning and decision-making. This includes probabilistic approaches based on frequentist statistics such as conformal prediction, risk control or e-values, but also those relying on Bayesian principles. I am also interested in the connection to other notions of model reliability such as calibration and forecaster scoring, robustness and generalization, or interpretability. Applications of interest include vision, time series, and online settings.

I graduated with an MSc in Statistics from ETH Zurich, specialising in machine learning and computational statistics. My master thesis was an interdisciplinary project with the MIE Lab on uncertainty quantification in traffic prediction (see here). I obtained a BSc in Industrial Engineering from the Karlsruhe Institute of Technology (KIT), focusing on statistics and finance.

Updates

Sept 26, 2024: Happy to have our work on risk control for early-exiting accepted at NeurIPS 2024! After promoting its usefulness for efficient but safe inference already at ICML this summer, we are also continuing the message in (wintery?) Vancouver. Side topic #1: The COPA symposium in Milan was great, and I was impressed how positive, down-to-earth and tight-knit the conformal community is. Would love to attend again should the opportunity arise. Side topic #2: After quite the lead time finally got admitted to the ELLIS PhD program! I’m on the industry track, since we’re working with Bosch.

Aug 21, 2024: Our accepted paper at ECCV received an Oral (top 3%), providing us with an opportunity to present our motivation to a wider audience! Seems like the vision community is indeed excited about the possibilities of conformal prediction to advance safety aspects in the field. We will also be presenting at the ECCV Workshop on Uncertainty Quantification for Computer Vision amidst the uncertainty community.

Jul 24, 2024: I’m glad to announce a new preprint, great joint work with my lab mate Metod Jazbec. Our paper titled Fast yet Safe: Early-Exiting with Risk Control addresses how to select the exit threshold for early-exit models with performance guarantees via risk control. I’m presenting our work at two ICML workshops this week: one focused on probabilistic inference (SPIGM), and one on efficient inference (ES-FOMO). Hoping this will get both communities excited (and beyond) about the usefulness of statistical ML for all kinds of applications!

Jul 8, 2024: Happy to have our paper titled Adaptive Bounding Box Uncertainties via Two-Step Conformal Prediction accepted at this year’s European Conference on Computer Vision (ECCV)! We extended our workshop paper from last year to incorporate classification uncertainty using a two-step conformal approach – looking forward to sharing it with the vision community. And yes, that will be me visiting Milan twice in September.

May 23, 2024: On the side, I’ve been working with a small student team on an idea related to conformal prediction for time series. We’ve fleshed it out into a full paper titled Conformal time series decomposition with component-wise exchangeability, which has been accepted at this year’s COPA symposium, the dedicated venue for conformal prediction research! Looking forward to attending the conference in Milan in September, and hopefully meeting some of the field’s founding figures like Prof. Vladimir Vovk.

Mar 18, 2024: Good times attending the Machine Learning Summer School in Okinawa at OIST, with particularly interesting talks by Shai Ben-David and Pierre Alquier on PAC and PAC-Bayes respectively – the evolution of generalization bounds continues. The OIST campus is breath-taking, and Okinawa seems like a bit of a hidden gem for beach getaways. Continuing the adventure with travels across Japan (Tokyo, Fuji, Kyoto, Osaka) until end of month, including a short visit to the RIKEN AIP in Tokyo!

Jan 29, 2024: Due to Eric’s switch from the University of Amsterdam to Johns Hopkins in Baltimore, I am happy to have Christian Naesseth join as a co-supervisor alongside Eric. Christian has done very interesting work in probabilistic modeling and we share a common interest in statistical ML topics such as conformal prediction and e-values – here’s to a nice collaboration!

Dec 4, 2023: We’ve published a preprint on a multiple testing correction dedicated to conformal prediction applications, titled A powerful rank-based correction to multiple testing under positive dependency.

Nov 20, 2023: I am glad to see my student Derck Prinzhorn receive an Amsterdam AI Thesis award for his bachelor thesis on conformal prediction, and present his work to the local community. He worked diligently at the intersection of multiple complicated topics – well done!

Oct 23, 2023: I will be helping out as a teaching assistant for the courses on Human-in-the-loop ML (MSc) and Introduction to ML (BSc) for the winter teaching block until the end of the year – hoping to excite some new faces to the field!

Oct 7, 2023: Participated in ICCV 2023 in Paris to present our poster on ‘Adaptive Bounding Box Uncertainty via Conformal Prediction’ at the Workshop on Uncertainty Quantification in Computer Vision – thanks to the organizers! It was fun to see the thriving ML community and the various research directions, particularly in the computer vision domain. Notably, there was a lot of work on multi-modality and foundation model finetuning. Some interesting thoughts on the current state of the field were given in the Quo Vadis, Computer Vision workshop by the likes of David Forsyth, Bill Freeman, and Jitendra Malik – is “putting language into vision” the best approach? Also got to catch Yann LeCun in person at the Meta booth!

Sept 1, 2023: I was happy to participate in the ELLIS Doctoral Symposium 2023 in Helsinki, Finland! A lot of fun activities (and sauna galore) and great to see so many fellow ML PhD students enthusiastically present their work. Particularly interesting discussions on calibration for regression and conformal prediction for time series – thanks and maybe see you next year!

Aug 23, 2023: The AMLab yearly retreat was good fun, with karaoke and escape room action as well as plenty of socializing – I’m happy to be working with such a nice crowd!

Aug 11, 2023: The research paper Uncertainty Quantification for Image-based Traffic Prediction across Cities, a resulting work from my master thesis in collaboration with the Mobility Information Engineering Lab at ETH Zurich, is finally publicly available on arXiv!

Aug 9, 2023: Our workshop submission on ‘Adaptive Bounding Box Uncertainty via Conformal Prediction’ got accepted to the Workshop on Uncertainty Quantification in Computer Vision at ICCV 2023 – looking forward to presenting our work in Paris!

Aug 1, 2023: My bachelor student Derck Prinzhorn successfully graduated with a thesis on the topic of Benchmarking conformal prediction methods for time series regression – great job and congratulations!

Apr 4, 2023: Started a bachelor thesis supervision project on conformal prediction for time series, as well as TA’ing the Deep Learning 2 course with a module on uncertainty quantification. I am also supervising a course project on Bayesian subnetwork inference.

Mar 28, 2023: Had an interesting time at the Bosch Center for Artificial Intelligence HQ in Renningen in the context of a lab visit organized by the Delta Lab. It was great to meet my Bosch co-supervisors in person and see how our research can potentially translate to Bosch applications, such as in autonomous driving and robotics.

Feb 16, 2023: Participated in the CWI Machine Learning Theory Bootcamp just next door to the University campus. I particularly enjoyed the talk by Bob Williamson on imprecise probability.

Oct 10, 2022: Joined the Amsterdam Machine Learning Lab (AMLab) as a PhD candidate supervised by Eric Nalisnick!

Research

Max-Rank: Efficient Multiple Testing for Conformal Prediction Alexander Timans, Christoph-Nikolas Straehle, Kaspar Sakmann, Eric Nalisnick Preprint (arXiv), 2024 Links: Paper We suggest max-rank, a multiple testing correction based on rank permutations that is particularly suitable for multiple testing issues arising in conformal prediction.
Fast yet Safe: Early-Exiting with Risk Control Metod Jazbec, Alexander Timans, Tin Hadzi Veljkovic, Kaspar Sakmann, Dan Zhang, Christian A. Naesseth, Eric Nalisnick Neural Information Processing Systems (NeurIPS), 2024 Also in: ICML Workshops on Structured Probabilistic Inference and Generative Modelling & Efficient Systems for Foundation Models * Equal contribution Links: Paper \| Code \| Poster We investigate how to adapt frameworks of risk control to early-exit neural networks, providing a distribution-free, post-hoc solution that tunes the EENN’s exiting mechanism so that exits only occur when the output is guaranteed to satisfy user-specified performance goals.
Adaptive Bounding Box Uncertainties via Two-Step Conformal Prediction Alexander Timans, Christoph-Nikolas Straehle, Kaspar Sakmann, Eric Nalisnick European Conference on Computer Vision (ECCV) Oral, 2024 Also in: ECCV Workshop on Uncertainty Quantification for CV Links: Paper \| Code \| Poster We propose a two-step conformal approach that propagates uncertainty in predicted class labels into the uncertainty intervals of bounding boxes. This broadens the validity of conformal coverage guarantees to include incorrectly classified objects. This work builds on our workshop paper.
Conformal time series decomposition with component-wise exchangeability Derck Prinzhorn, Thijmen Nijdam, Putri van der Linden, Alexander Timans Conformal and Probabilistic Prediction with Applications (PLMR), 2024 Links: Paper \| Code We present a novel use of conformal prediction for time series forecasting that incorporates time series decomposition, allowing us to customize employed methods to account for the different exchangeability regimes underlying each time series component.
Adaptive Bounding Box Uncertainty via Conformal Prediction Alexander Timans, Christoph-Nikolas Straehle, Kaspar Sakmann, Eric Nalisnick ICCV Workshop on Uncertainty Quantification for CV, 2023 Links: Paper We quantify the uncertainty in multi-object 2D bounding box predictions via conformal prediction, producing tight prediction intervals with guaranteed per-class coverage for the bounding box coordinates.
Uncertainty Quantification for Image-based Traffic Prediction across Cities Alexander Timans, Nina Wiedemann, Nishant Kumar, Ye Hong, Martin Raubal Preprint (arXiv), 2023 Links: Paper \| Code We explore different uncertainty quantification methods on a large-scale image-based traffic dataset spanning multiple cities and time periods, originally featured as a NeurIPS 2021 prediction challenge. Meaningful uncertainty relating to traffic dynamics is recovered by a combination method of deep ensembles and patch-based deviations. In a case study, we demonstrate how uncertainty estimates can be employed for unsupervised outlier detection on traffic behaviour.

Other activities

Reviewing: AISTATS 2025, ICLR 2025, NeurIPS 2024, ICCV 2023
Teaching & Supervision: Project AI (MSc/UvA), Human-in-the-loop ML (MSc/UvA), Introduction to ML (BSc/UvA), Derck Prinzhorn (BSc Thesis/UvA, Thesis award), Deep Learning 2 (MSc/UvA), ANOVA (MSc/ETH), Econometrics (BSc/KIT)

Source: adapted from Dharmesh Tailor's fork of Leonid Keselman's fork of John Barron's website.