Mark Crowley

Research Lab (UWECEML)
Blog: Computationally Thinking
ORC-ID
Google Scholar
Contact
Twitter / Mastodon
BOOK! : "Elem. of Dim. Reduction and Manifold Learning"

My research seeks dependable and transparent ways to augment human decision making in complex domains in the presence of many agents, spatial structure, or uncertainty. My focus is on developing new algorithms within the fields of Reinforcement Learning, Deep Learning and Ensemble Methods, and Manifold Learning (Dimensionality Reduction) as well as other Research Topics. I often work in collaboration with researchers in applied fields such as Computational Sustainability (including Sustainable Forest Management, Autonomous Driving, AI for Science and Medical Imaging. Read more about me in my bio.

Mark Crowley is an Associate Professor in the Department of Electrical and Computer Engineering at the University of Waterloo (also cross-appointed in the Cheriton School of Computer Science). He is a member of the Waterloo Artificial Intelligence Institute (WAII), the Waterloo Institute for Complexity and Innovation (WICI) and is National Secretary of the Canadian Artificial Intelligence Association (CAIAC) which coordinates the yearly organization of the Canadian Conference on AI. He and his students carry out research into single-agent and multi-agent Reinforcement Learning large-scale 2D/3D image-like processing, and manifold learning/dimensionality reduction. Some of this research is motivated by theoretical opportunities, particularly in manifold learning and multi-agent reinforcement learning. But most of the work flows out of challenges raised by real-world domains including forest fire management, the automotive domain, medical imaging, and digital chemistry/material design.

I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future, but I do not take on many incoming students, so potential students should read this note about joining my lab.

Besides my publications, you can find my thoughts on Artificial Intelligence, Machine Learning and how technology and science are advancing on my blog Computationally Thinking (updated intermittently) or through social media as @compthink on Twitter(at least for now) or increasingly, on Mastodon. I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future.

recent news

Feb 21, 2025	PhD Defense of Shayan Shirahmadi Gale Bagi
Dec 20, 2024	MASc Thesis of Oleksandra Nahorna

recent highlighted publications

Dynamic programming with incomplete information to overcomenavigational uncertainty in POMDPs

Chris Beeler, Xinkai Li, Colin Bellinger, Mark Crowley, Maia Fraser, and Isaac Tamblyn.

In Proceedings of the Canadian Conference on Artificial Intelligence. Canadian Artificial Intelligence Association (CAIAC), Guelph, Ontario, Canada. May, 2024.

Abs PDF URL

Using a generalizable novel nautical navigation environment, we show how dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.
ChemGymRL

ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague, Nouha Chatti, Colin Bellinger, Mitchell Shahen, Nicholas Paquin, Mark Baula, Amanuel Dawit, Zihan Yang, Xinkai Li, Mark Crowley, and Isaac Tamblyn.

Digital Discovery. 3, Feb, 2024.

Abs PDF Code

This paper provides a simulated laboratory for making use of reinforcement learning (RL) for material design, synthesis, and discovery. Since RL is fairly data intensive, training agents ‘on-the-fly’ by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, implementing the standard gymnasium API. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.
Causal2

[preprint] Implicit Causal Representation Learning via Switchable Mechanisms

Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

In Arxiv Preprint. 2024.

arXiv PDF
GCRL

Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting

Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

In Proceedings of the 40th International Conference on Machine Learning (ICML). PMLR, Honolulu, Hawaii, USA. Jul, 2023.

Abs arXiv PDF Code Poster URL Hypoth

Conventional supervised learning methods typically assume i.i.d samples and are found to be sensitive to out-of-distribution (OOD) data. We propose Generative Causal Representation Learning (GCRL) which leverages causality to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well. First, we propose a novel causal model that explains the generative factors in motion forecasting datasets using features that are common across all environments and with features that are specific to each environment. Selection variables are used to determine which parts of the model can be directly transferred to a new environment without fine-tuning. Second, we propose an end-to-end variational learning paradigm to learn the causal mechanisms that generate observations from features. GCRL is supported by strong theoretical results that imply identifiability of the causal model under certain assumptions. Experimental results on synthetic and real-world motion forecasting datasets show the robustness and effectiveness of our proposed method for knowledge transfer under zero-shot and low-shot settings by substantially outperforming the prior motion forecasting models on out-of-distribution prediction.
Textbook

Elements of Dimensionality Reduction and Manifold Learning

Benyamin Ghojogh, Mark Crowley, Fakhri Karray, and Ali Ghodsi.

Springer Nature, Feb, 2023.

Abs URL

Dimensionality reduction, also known as manifold learning, is an area of machine learning used for extracting informative features from data, for better representation of data or separation between classes. This book presents a cohesive review of linear and nonlinear dimensionality reduction and manifold learning. Three main aspects of dimensionality reduction are covered – spectral dimensionality reduction, probabilistic dimensionality reduction, and neural network-based dimensionality reduction, which have geometric, probabilistic, and information-theoretic points of view to dimensionality reduction, respectively. This book delves into basic concepts and recent developments in the field of dimensionality reduction and manifold learning, providing the reader with a comprehensive understanding. The necessary background and preliminaries, on linear algebra, optimization, and kernels, are also explained in the book to ensure a comprehensive understanding of the algorithms.