Research

Preprints

Environment-Robust Representation Learning with Empirical Bayes

We consider multi-environment prediction problems in which environments change the distribution of a latent variable while the mechanisms generating covariates and targets remain stable conditional on that variable. We formulate a Bayesian model, derive a variational objective with an empirical-Bayes prior, and use amortized variational inference to learn representations for prediction in new environments.

Slavutsky, Y., Shen, M., Wu, B., & Blei, DM. (2026). "Environment-Robust Representation Learning with Empirical Bayes." ArXiv Preprint.

Robust Representation Learning through Explicit Environment Modeling

We study multi-environment prediction when the environment may directly affect the target, so invariant-representation assumptions can fail. The paper analyzes representations learned by explicitly modeling environment variation and marginalizing it out, and proposes a method based on generalized random-intercept models.

Slavutsky, Y., & Blei, DM. (2026). "Robust Representation Learning through Explicit Environment Modeling." ArXiv Preprint.

Neural Generalized Mixed-Effects Models

Generalized linear mixed-effects models are widely used for grouped and hierarchical data, but their linear structure can be limiting. We replace the linear predictor with neural networks, yielding neural generalized mixed-effects models, and introduce a differentiable approximation to the marginal likelihood for efficient optimization.

Slavutsky, Y., Salazar, S., & Blei, DM. (2026). "Neural Generalized Mixed-Effects Models." ArXiv Preprint.

Input Adaptive Bayesian Model Averaging

This paper studies prediction with multiple candidate models in heterogeneous settings, where different models may be better suited to different inputs. We propose input-adaptive Bayesian model averaging, which assigns model weights conditional on the input through an adaptive prior and estimates the resulting posterior with amortized variational inference.

Slavutsky, Y., Salazar, S., & Blei, DM. (2025). "Input Adaptive Bayesian Model Averaging." ArXiv Preprint.

Published

Variational Learning of Disentangled Representations

Existing variational methods for learning disentangled representations often suffer from leakage between latent variables. We introduce DISCoVeR, a variational framework with a dual-latent architecture, parallel reconstruction paths, and a max-min objective that separates condition-invariant and condition-specific factors. The method improves disentanglement on synthetic data, natural images, and single-cell RNA-seq data.

Slavutsky Y.*, Beker O.*, Blei, DM., & Dumitrascu, B. (2026). "Variational Learning of Disentangled Representations." International Conference on Machine Learning (ICML).
* Equally contributing authors.

Quantifying Uncertainty in the Presence of Distribution Shifts

We propose a Bayesian framework for uncertainty estimation that explicitly accounts for covariate shifts. The method uses an adaptive prior conditioned on both training and new covariates, increasing uncertainty for inputs far from the training distribution in regions where predictive performance is likely to degrade. The posterior predictive distribution is approximated using amortized variational inference, and synthetic environments are constructed from bootstrap samples of the training data.

Slavutsky, Y., & Blei, DM. (2025). "Quantifying Uncertainty in the Presence of Distribution Shifts." Conference on Neural Information Processing Systems (NeurIPS).

Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations

Class distribution shifts challenge zero-shot classifiers, which rely on representations from training classes but are tested on unseen ones. We propose a model that assumes an unknown class attribute causes the shift and present an algorithm to learn robust representations where ERM would fail.

Slavutsky, Y., & Benjamini, Y. (2024). "Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations." Conference on Neural Information Processing Systems (NeuRIPS).

CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models

Different estimators of joint probabilities of word spans arise by marginalizing the LLM’s scores from various completion orders. If the predictions are reliable, these estimates should be consistent. In this paper, we develop a statistical framework to test this and apply it to evaluate different LLMs. We show that both Masked Language Models and autoregressive models exhibit inconsistent predictions.

Wagner, E., Slavutsky, Y., & Abend, O. (2024). "CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models." The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP).

Yuli Slavutsky