Mark Crowley | New Journal Publication on Causal Representation Learning

We have an exciting new journal publication (Bagi et al., 2026) based on the work of former PhD student Shayan Shirahmadi Gale Bagi, along with Professors Zahra Gharaee and Oliver Schulte. The paper can be found in the latest issue of Nature Machine Learning

Bagi, S. S. G., Gharaee, Z., Schulte, O., & Crowley, M. (2026). Modeling Soft Intervention Effects for Implicit Causal Representation Learning. Machine Learning, 115(199). https://link.springer.com/article/10.1007/s10994-026-07144-5

This paper falls into the general research topic is causality, including modeling, representation, learning about causal relations, which was the core of Shayan’s PhD thesis.

Shayan defended his thesis in 2025 (Bagi, 2025) and now works at Huawei Canada as an AI Engineer. His thesis work was on theoretical and applied methods for improving our ability to learn latent causal relationships from observational data. This included a paper “Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting” at ICML in 2023 (Bagi et al., 2023). In that paper, Shayan proposed a new framework for leveraging causal information to improve robustness of learning in the presence of distribution shift and included. Empirical results on the motion forecasting domain support the theoretical findings.

The current paper expands on some of the final insights in Shayan’s thesis about learning causal representations from data without access to ground-truth causal graphs, which presents a difficult challenge in representation learning.

Implicit approaches—where the model learns causal dependencies without explicitly parameterizing the causal graph—offer a compelling advantage over explicit methods by avoiding optimization difficulties such as local minima associated with adjacency matrix estimation. However, existing implicit methods often assume access to hard interventions, which are rarely feasible in real-world scenarios. In contrast, soft interventions—more prevalent in practice—modify causal mechanisms without severing parental dependencies, introducing subtle, ambiguous effects that confound learning.

To address this, this paper introduces ICRL-SM, a novel method for Implicit Causal Representation Learning from Soft Interventions using a causal Mechanism switch variable. This variable captures unwanted changes induced by soft interventions, enabling the model to focus on the necessary variations that reflect underlying causal structure. Our framework leverages a variational autoencoder trained on paired pre- and post-intervention samples, and is theoretically grounded under a set of assumptions that ensure identifiability of causal representations. Although some assumptions (e.g., Gaussianity of latent variables, diffeomorphic decoders) are strict, we show empirically that our method performs robustly even when these are violated—demonstrating strong results on both synthetic benchmarks and real-world image datasets. These findings highlight the potential of ICRL-SM to bridge the gap between theoretical identifiability and practical applicability, advancing causal representation learning under realistic conditions.

Source Code: The source code for this paper is available at: [https://github.com/sshirahmad/ICRL]

References:

PhD Thesis

Advancing Causal Representation Learning: Enhancing Robustness and Transferability in Real-World Applications

Shayan Shirahmad Gale Bagi.

UWSpace Thesis Repository, University of Waterloo, Feb, 2025.

Abs URL

Conventional supervised learning methods heavily depend on statistical inference, of- ten assuming that data is identically and independently distributed (i.i.d). However, this assumption rarely holds in real-world scenarios, where environments or domains frequently shift, posing significant challenges to model robustness and generalization. Moreover, sta- tistical models are typically treated as black boxes, with their learned representations re- maining opaque and challenging to interpret. My research addresses these issues through a causal learning perspective, aiming to enhance the interpretability and adaptability of machine learning models in dynamic and uncertain environments. I have developed innovative methods for learning causal models that are applicable to a wide range of machine learning tasks, including transfer learning, out-of-distribution gen- eralization, reinforcement learning, and action classification. The first method introduces a generative model tailored to learn causal variables in scenarios where the causal graph is known, such as Human Trajectory Prediction. By incorporating domain knowledge, this approach models the underlying causal mechanisms, leading to improved performance on both synthetic and real-world datasets. The results demonstrate that this generative model outperforms traditional statistical models, particularly in out-of-distribution contexts. The second method targets the more challenging scenario where the causal structure is unknown. I have explored various conditions and assumptions that facilitate the dis- covery of causal relationships without prior knowledge of the causal graph. This method combines advanced techniques in causal inference and machine learning to uncover the underlying causal graph and variables from observed data. Evaluations on both real-world and synthetic datasets show that this method not only surpasses existing approaches in causal representation learning but also brings AI systems closer to practical, real-world applications by enhancing reliability and interpretability. Overall, my research contributes significant advancements to the field of causal learn- ing, providing novel solutions that improve model interpretability and robustness. These methods lay a strong foundation for developing AI systems capable of adapting to diverse and evolving real-world conditions, thereby broadening the scope and impact of machine learning across various domains.

CausalJournal

Modeling Soft Intervention Effects for Implicit Causal Representation Learning

Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

Machine Learning. 115, (199). Springer Nature, Aug, 2026.

Abs arXiv PDF URL

Learning causal representations from data without access to ground-truth causal graphs remains a central challenge in representation learning. Implicit approaches—where the model learns causal dependencies without explicitly parameterizing the causal graph—offer a compelling advantage over explicit methods by avoiding optimization difficulties such as local minima associated with adjacency matrix estimation. However, existing implicit methods often assume access to hard interventions, which are rarely feasible in real-world scenarios. In contrast, soft interventions—more prevalent in practice—modify causal mechanisms without severing parental dependencies, introducing subtle, ambiguous effects that confound learning. To address this, we propose ICRL-SM, a novel method for Implicit Causal Representation Learning from soft interventions using a causal mechanism switch variable. This variable captures unwanted changes induced by soft interventions, enabling the model to focus on the necessary variations that reflect underlying causal structure. Our framework leverages a variational autoencoder trained on paired pre- and post-intervention samples, and is theoretically grounded under a set of assumptions that ensure identifiability of causal representations. Although some assumptions (e.g., Gaussianity of latent variables, diffeomorphic decoders) are strict, we show empirically that our method performs robustly even when these are violated—demonstrating strong results on both synthetic benchmarks and real-world image datasets. These findings highlight the potential of ICRL-SM to bridge the gap between theoretical identifiability and practical applicability, advancing causal representation learning under realistic conditions. The source code for this paper is available at: https://github.com/sshirahmad/ICRL.

GCRL

Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting

Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

In Proceedings of the 40th International Conference on Machine Learning (ICML). PMLR, Honolulu, Hawaii, USA. Jul, 2023.

Abs arXiv PDF Code Poster URL Hypoth

Conventional supervised learning methods typically assume i.i.d samples and are found to be sensitive to out-of-distribution (OOD) data. We propose Generative Causal Representation Learning (GCRL) which leverages causality to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well. First, we propose a novel causal model that explains the generative factors in motion forecasting datasets using features that are common across all environments and with features that are specific to each environment. Selection variables are used to determine which parts of the model can be directly transferred to a new environment without fine-tuning. Second, we propose an end-to-end variational learning paradigm to learn the causal mechanisms that generate observations from features. GCRL is supported by strong theoretical results that imply identifiability of the causal model under certain assumptions. Experimental results on synthetic and real-world motion forecasting datasets show the robustness and effectiveness of our proposed method for knowledge transfer under zero-shot and low-shot settings by substantially outperforming the prior motion forecasting models on out-of-distribution prediction.