Mark Crowley

Site being updated, see my Research Lab page for current information.

My research seeks dependable and transparent ways to augment human decision making in complex domains in the presence of spatial structure, large scale streaming data, or uncertainty. My focus is on developing new algorithms within the fields of Reinforcement Learning, Deep Learning and Ensemble Methods, and Manifold Learning (Dimensionality Reduction) as well as other Research Topics.
I often work in collaboration with researchers in applied fields such as Computational Sustainability (including Sustainable Forest Management), Autonomous Driving, AI for Physics (including Combustion Modelling and Digital Chemistry) and Medical Imaging.

Note, that I do not take on many incoming graduate students, so if you are hoping to join my lab as a graduate student for research please read this note.

Besides my publications, you can follow my Computationally Thinking blog or @compthink on twitter for links and thoughts on Artificial Intellgience, Machine Learning and how technology and science are advancing. I’m always happy to talk to people about my research, or the impact of AI/ML/RL on our world and its role in our society in the future.

news

Nov 2, 2021	Recently Accepted Papers more: Three great papers accepted to AAAI, NeurIPS Workshop and EMBC in past month!
Nov 1, 2021	Patent arose in collaboration with DENSO Inc. on a remote controlled car demo for CES. more: Patent awarded for Multi-level collaborative vehicle control

selected showcase publications

Mean Field MARL

Decentralized Mean Field Games

Ganapathi Subramanian, Sriram, Taylor, Mathew, Crowley, Mark, and Poupart, Pascal

In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-2022) 2022

Abs

Multiagent reinforcement learning algorithms have not been widely adopted in large scale environments with many agents as they often scale poorly with the number of agents. Using mean field theory to aggregate agents has been proposed as a solution to this problem. However, almost all previous methods in this area make a strong assumption of a centralized system where all the agents in the environment learn the same policy and are effectively indistinguishable from each other. In this paper, we relax this assumption about indistinguishable agents and propose a new mean field system known as Decentralized Mean Field Games, where each agent can be quite different from others. All agents learn independent policies in a decentralized fashion, based on their local observations. We define a theoretical solution concept for this system and provide a fixed point guarantee for a Q-learning based algorithm in this system. A practical consequence of our approach is that we can address a ‘chicken-and-egg’ problem in empirical mean field reinforcement learning algorithms. Further, we provide Q-learning and actor-critic algorithms that use the decentralized mean field learning approach and give stronger performances compared to common baselines in this area. In our setting, agents do not need to be clones of each other and learn in a fully decentralized fashion. Hence, for the first time, we show the application of mean field learning methods in fully competitive environments, large-scale continuous action space environments, and other environments with heterogeneous agents. Importantly, we also apply the mean field method in a ride-sharing problem using a real-world dataset. We propose a decentralized solution to this problem, which is more practical than existing centralized training methods.
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

Frontiers in Artificial Intelligence (Machine Learning and Artificial Intelligence) 2021
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

In NeurIPS 2021 Deep Reinforcement Learning Workshop 2021

Abs arXiv

Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on four PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. We show that in fully-observable environments, independent algorithms can perform on par with multi-agent algorithms in cooperative and competitive settings. For the mixed environments, we show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies. We also show that adding recurrence improves the learning of independent algorithms in cooperative partially observable environments.
Multi-Advisor-QL

Multi-Agent Advisor Q-Learning

Ganapathi Subramanian, Sriram, Larson, Kate, Taylor, Mathew, and Crowley, Mark

Journal of Artificial Intelligence Research (JAIR) 2022

URL
Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling

Poorheravi, Parisa, Ghojogh, Benyamin, Gaudet, Vincent, Karray, Fakhri, and Crowley, Mark

Journal of Computational Vision and Imaging Systems 2021

Abs PDF URL

Metric learning is a technique in manifold learning to find a projection subspace for increasing and decreasing the inter- and intra-class variances, respectively. Some metric learning methods are based on triplet learning with anchor-positive-negative triplets. Large margin metric learning for nearest neighbor classification is one of the fundamental methods to do this. Recently, Siamese networks have been introduced with the triplet loss. Many triplet mining methods have been developed for Siamese nets; however, these techniques have not been applied on the triplets of large margin metric learning. In this work, inspired by the mining methods for Siamese nets, we propose several triplet mining techniques for large margin metric learning. Moreover, a hierarchical approach is proposed, for acceleration and scalability of optimization, where triplets are selected by stratified sampling in hierarchical hyper-spheres. We analyze the proposed methods on three publicly available datasets.

Tweets by compthink