Mark Crowley

Research Lab (UWECEML)
Blog: Computationally Thinking
ORC-ID
Google Scholar
Contact
Twitter / Mastodon
BOOK! : "Elem. of Dim. Reduction and Manifold Learning"

My research seeks dependable and transparent ways to augment human decision making in complex domains in the presence of many agents, spatial structure, or uncertainty. My focus is on developing new algorithms within the fields of Reinforcement Learning, Deep Learning and Ensemble Methods, and Manifold Learning (Dimensionality Reduction) as well as other Research Topics. I often work in collaboration with researchers in applied fields such as Computational Sustainability (including Sustainable Forest Management, Autonomous Driving, AI for Science and Medical Imaging. Read more about me in my bio.

Mark Crowley is an Associate Professor in the Department of Electrical and Computer Engineering at the University of Waterloo (also cross-appointed in the Cheriton School of Computer Science). He is a member of the Waterloo Artificial Intelligence Institute (WAII), the Waterloo Institute for Complexity and Innovation (WICI) and is National Secretary of the Canadian Artificial Intelligence Association (CAIAC) which coordinates the yearly organization of the Canadian Conference on AI. He and his students carry out research into single-agent and multi-agent Reinforcement Learning large-scale 2D/3D image-like processing, and manifold learning/dimensionality reduction. Some of this research is motivated by theoretical opportunities, particularly in manifold learning and multi-agent reinforcement learning. But most of the work flows out of challenges raised by real-world domains including forest fire management, the automotive domain, medical imaging, and digital chemistry/material design.

I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future, but I do not take on many incoming students, so potential students should read this note about joining my lab.

Besides my publications, you can find my thoughts on Artificial Intelligence, Machine Learning and how technology and science are advancing on my blog Computationally Thinking (updated intermittently) or through social media as @compthink on Twitter(at least for now) or increasingly, on Mastodon. I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future.

recent news

May 25, 2023	AAMAS talk, PhD Dissertation Award, ChemGymRL Updates

recent highlighted publications

Machine-learning Assisted Swallowing Assessment: a deep learning-based quality improvement tool to screen for post-stroke dysphagia

Rami Saab, Arjun Balachandar, Hamza Mahdi, Eptehal Nashnoush, Lucas Perri, Ashley Waldron, Alireza Sadeghian, Gordon Rubenfeld, Mark Crowley, Mark I. Boulos, Brian Murray, and Houman Khosravani.

Frontiers in Neuroscience. 17, Nov, 2023.

Abs Code URL

Post-stroke dysphagia is common and associated with significant morbidity and mortality, rendering bedside screening of significant clinical importance. Using voice as a biomarker coupled with deep learning has the potential to improve patient access to screening and mitigate the subjectivity associated with detecting voice change, a component of several validated screening protocols. In this single-center study, we developed a proof-of-concept model for automated dysphagia screening and evaluated the performance of this model on training and testing cohorts. Patients were admitted to a comprehensive stroke center, where primary English speakers could follow commands without significant aphasia and participated on a rolling basis. The primary outcome was classification either as a pass or fail equivalent using a dysphagia screening test as a label. Voice data was recorded from patients who spoke a standardized set of vowels, words, and sentences from the National Institute of Health Stroke Scale. Seventy patients were recruited and 68 were included in the analysis, with 40 in training and 28 in testing cohorts, respectively. Speech from patients was segmented into 1,579 audio clips, from which 6,655 Mel-spectrogram images were computed and used as inputs for deep-learning models (DenseNet and ConvNext, separately and together). Clip-level and participant-level swallowing status predictions were obtained through a voting method. Model performance on the individual clip-level demonstrated dysphagia screening sensitivity of 71% and specificity of 77% (F1=0.73, AUC=0.80 [95% CI: 0.78-0.82]. On the participant-level the sensitivity and specificity were 89% and 79% respectively (F1=0.81, AUC=0.91 [95% CI: 0.77–1.05]). Our study is the first to demonstrate the feasibility of applying deep learning to classify vocalizations to detect post-stroke dysphagia.
ChemGymRL

ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague, Nouha Chatti, Colin Bellinger, Mitchell Shahen, Nicholas Paquin, Mark Baula, Amanuel Dawit, Zihan Yang, Xinkai Li, Mark Crowley, and Isaac Tamblyn.

In ICML 2023 Synergy of Scientific and Machine Learning Modeling (SynS&ML) Workshop. Jul, 2023.

Abs PDF Code Poster Slides URL

This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents ‘on-the-fly’ by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.
Multi-Advisor-QL

Multi-Agent Advisor Q-Learning

Sriram Ganapathi Subramanian, Kate Larson, Matthew Taylor, and Mark Crowley

In International Joint Conference on Artificial Intelligence (IJCAI) : Journal Track. Macao, China. Aug, 2023.

Abs PDF Code URL Video

In the last decade, there have been significant advances in multi-agent reinforcement learn- ing (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possi- ble. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub- optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experi- ments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.
GCRL

Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting

Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

In International Conference on Machine Learning (ICML). Honolulu, Hawaii, USA. Jul, 2023.

Abs arXiv PDF Code Poster Hypoth

Conventional supervised learning methods typically assume i.i.d samples and are found to be sensitive to out-of-distribution (OOD) data. We propose Generative Causal Representation Learning (GCRL) which leverages causality to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well. First, we propose a novel causal model that explains the generative factors in motion forecasting datasets using features that are common across all environments and with features that are specific to each environment. Selection variables are used to determine which parts of the model can be directly transferred to a new environment without fine-tuning. Second, we propose an end-to-end variational learning paradigm to learn the causal mechanisms that generate observations from features. GCRL is supported by strong theoretical results that imply identifiability of the causal model under certain assumptions. Experimental results on synthetic and real-world motion forecasting datasets show the robustness and effectiveness of our proposed method for knowledge transfer under zero-shot and low-shot settings by substantially outperforming the prior motion forecasting models on out-of-distribution prediction.
IOTSMS

Aggressive Driver Behavior Detection using Parallel Convolutional Neural Networks on Simulated and Real Driving Data

Zehra Camlica, Jim Quesenberry, Daniel Carballo, and Mark Crowley

In 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS). IEEE, Milan, Italy. Nov, 2022.

Abs PDF Slides URL

The novel method proposed in this paper is com- promised of application of two Convolutional Neural Networks (CNN) working in parallel to simultaneously classify driver be- haviors while classifying maneuvers by using time series data. We claim that the Parallel Convolutional Neural Network (PCNN) not only speeds-up training time but also increases performance since having information about the maneuver helps to improve behavior classification performance and vice versa. In this study, both simulation and real-world driving datasets are utilized for driver behavior analysis. As simulation data, mobile phone sensor data are simulated as a time series using a combination of a traffic simulator (SUMO) and a car simulation system (Webots). The same type of data is collected with a specially designed vehicle traveled on a defined route around a predefined region. The collected data are then separately utilized as training and testing data for classification of both maneuvers (e.g turns and lane changes) and driver behaviors (e.g aggressive, non-aggressive) applying a novel method using deep learning on time series data. In addition, other methods which are commonly used for time series analysis, Hidden Markov Models(HMMs) and Recurrent Neural Networks (RNN), are applied to the same datasets to compare with PCNN. According to the results, the CNN classifiers perform efficiently for a single task and PCNN outperforms both single task-CNN and RNN with an average accuracy of 86%.