Mark Crowley

Site being updated, see my Research Lab page for current information.

My research seeks dependable and transparent ways to augment human decision making in complex domains in the presence of spatial structure, large scale streaming data, or uncertainty. My focus is on developing new algorithms within the fields of Reinforcement Learning, Deep Learning and Ensemble Methods, and Manifold Learning (Dimensionality Reduction) as well as other Research Topics.
I often work in collaboration with researchers in applied fields such as Computational Sustainability (including Sustainable Forest Management), Autonomous Driving, AI for Physics (including Combustion Modelling and Digital Chemistry) and Medical Imaging.

Note, that I do not take on many incoming graduate students, so if you are hoping to join my lab as a graduate student for research please read this note.

Besides my publications, you can follow my Computationally Thinking blog or @compthink on twitter for links and thoughts on Artificial Intellgience, Machine Learning and how technology and science are advancing. I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future.

news

Mar 7, 2022	Lab News for March 2022 more: Some great publications accepted in past four months.

selected showcase publications

QQE

Quantile–Quantile Embedding for distribution transformation and manifold embedding with ability to choose the embedding distribution

Ghojogh, Benyamin, Karray, Fakhri, and Crowley, Mark

Machine Learning with Applications (MLWA) 2021

Abs arXiv PDF URL

We propose a new embedding method, named Quantile-Quantile Embedding (QQE), for distribution transformation and manifold embedding with the ability to choose the embedding distribution. QQE, which uses the concept of quantile-quantile plot from visual statistical tests, can transform the distribution of data to any theoretical desired distribution or empirical reference sample. Moreover, QQE gives the user a choice of embedding distribution in embedding the manifold of data into the low dimensional embedding space. It can also be used for modifying the embedding distribution of other dimensionality reduction methods, such as PCA, t-SNE, and deep metric learning, for better representation or visualization of data. We propose QQE in both unsupervised and supervised forms. QQE can also transform a distribution to either an exact reference distribution or its shape. We show that QQE allows for better discrimination of classes in some cases. Our experiments on different synthetic and image datasets show the effectiveness of the proposed embedding method.
Upcoming Book

Elements of Dimensionality Reduction and Manifold Learning

Ghojogh, Benyamin, Crowley, Mark, Karray, Fakhri, and Ghodsi, Ali

2022

Abs URL

Dimensionality reduction, also known as manifold learning, is an area of machine learning used for extracting informative features from data, for better representation of data or separation between classes. This book presents a cohesive review of linear and nonlinear dimensionality reduction and manifold learning. Three main aspects of dimensionality reduction are covered – spectral dimensionality reduction, probabilistic dimensionality reduction, and neural network-based dimensionality reduction, which have geometric, probabilistic, and information-theoretic points of view to dimensionality reduction, respectively. This book delves into basic concepts and recent developments in the field of dimensionality reduction and manifold learning, providing the reader with a comprehensive understanding. The necessary background and preliminaries, on linear algebra, optimization, and kernels, are also explained in the book to ensure a comprehensive understanding of the algorithms. The tools this book introduces are applied to various applications involving feature extraction, image processing, computer vision, and signal processing. This book is applicable to a wide audience who would like to acquire a deep understanding of the various ways to extract, transform, and understand the structure of data. The intended audiences are academics, students, and industry professionals. Academic researchers and students can use this book as a textbook for machine learning and dimensionality reduction. Data scientists, machine learning scientists, computer vision scientists, computer scientists, statisticians, and mathematicians in the fields of applied mathematics, statistical learning, Riemannian manifolds, subspace analysis, linear algebra, and optimization can use this book as a reference book for both technical and applied concepts. Industry professionals, including applied engineers, data engineers, and engineers in various fields of science dealing with machine learning, can use this book as a guidebook for feature extraction from their data as the raw data in industry often requires pre-processing. Data feature extraction is useful in various fields of science including engineering, physics, chemistry, biometrics, biomedical signals and images, etc. This book is structured as a reference textbook, so that it can be used for advanced courses, as an in-depth supplementary resource or for researchers or practitioners who want to learn about dimensionality reduction and manifold learning. The book is grounded in theory, but provides thorough explanations and diverse examples to improve the readers comprehension of the advanced topics. Advanced methods are explained in a step-by-step manner so that readers of all levels can follow the reasoning and come to a deep understanding of the concepts. This book does not assume the reader has an advanced theoretical background in machine learning and provides necessary background, though an undergraduate-level background in linear algebra and calculus is recommended.
Mean Field MARL

Decentralized Mean Field Games

Ganapathi Subramanian, Sriram, Taylor, Mathew, Crowley, Mark, and Poupart, Pascal

In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-2022) 2022

Abs arXiv PDF URL Hypoth

Multiagent reinforcement learning algorithms have not been widely adopted in large scale environments with many agents as they often scale poorly with the number of agents. Using mean field theory to aggregate agents has been proposed as a solution to this problem. However, almost all previous methods in this area make a strong assumption of a centralized system where all the agents in the environment learn the same policy and are effectively indistinguishable from each other. In this paper, we relax this assumption about indistinguishable agents and propose a new mean field system known as Decentralized Mean Field Games, where each agent can be quite different from others. All agents learn independent policies in a decentralized fashion, based on their local observations. We define a theoretical solution concept for this system and provide a fixed point guarantee for a Q-learning based algorithm in this system. A practical consequence of our approach is that we can address a ‘chicken-and-egg’ problem in empirical mean field reinforcement learning algorithms. Further, we provide Q-learning and actor-critic algorithms that use the decentralized mean field learning approach and give stronger performances compared to common baselines in this area. In our setting, agents do not need to be clones of each other and learn in a fully decentralized fashion. Hence, for the first time, we show the application of mean field learning methods in fully competitive environments, large-scale continuous action space environments, and other environments with heterogeneous agents. Importantly, we also apply the mean field method in a ride-sharing problem using a real-world dataset. We propose a decentralized solution to this problem, which is more practical than existing centralized training methods.
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

Frontiers in Artificial Intelligence 2022

Abs URL

Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on seven PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. For the cooperative setting, we show that independent algorithms can perform on par with multi-agent algorithms in fully-observable environments, while adding recurrence improves the learning of independent algorithms in partially-observable environments. In the competitive setting, independent algorithms can perform on par or better than multi-agent algorithms, even in more challenging environments. We also show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies in mixed environments.
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

In NeurIPS 2021 Deep Reinforcement Learning Workshop 2021

Abs arXiv

Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on four PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. We show that in fully-observable environments, independent algorithms can perform on par with multi-agent algorithms in cooperative and competitive settings. For the mixed environments, we show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies. We also show that adding recurrence improves the learning of independent algorithms in cooperative partially observable environments.

Tweets by compthink