Members of the UWECEML lab have had a great couple months, with some nice a publications and lots of career news.
A New Approach to Scaling Decision Making to Many Agents
Ganapathi Subramanian, S., Larson, K., Taylor, M., & Crowley, M. (2022). Multi-Agent Advisor Q-Learning. Journal of Artificial Intelligence Research (JAIR), 73, 1–74. https://doi.org/https://doi.org/10.1613/jair.1.13445
In situations where the number of agents is very large, we need to make some assumptions about structure to make RL feasible. Mean Field Theory is one such approach where each agent considers the impact to its interactions via another “cloud” agent which is actually an aggregation of all other agents, via a mean field calculation. This work was a core topic in the final stages of my PhD student Sriram Ganapathi Subramanian into Multi-Agent Reinforcement Learning (MARL).
In this paper, we build on a previous work in the lab to take advantage of the fact that many real-world environments already deploy sub-optimal or heuristic approaches for generating policies and show how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings.
Empirical Study of Reinforcement Learning Algorithms in Multi-Agent Settings
Lee, K. M., Ganapathi Subramanian, S., & Crowley, M. (2022). Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments. Frontiers in Artificial Intelligence, 27. https://doi.org/10.3389/frai.2022.805823
This Frontiers AI Paper was led by fourth-year undergraduate student Ken Ming Lee with lots of guidance and help from my PhD student Sriram Ganapathi Subramanian (Lee et al., 2022). It’s a great empirical comparison of a number of single-agent and multi-agent RL algorithms on a standard set of MARL problems. Often single-agent algorithms are quickly hacked in order to to decision making on multi-agent domains and seem to work fairly well. Our motivating question was how often is this true and what kinds of problems require approaches that consider more dedicated multi-agent interaction.
This work expands on a previous study (Lee et al., 2021) from a NeurIPS Workshop in 2021.
Lab Member News
On June 27, 2022, long-time lab member Sriram Ganapathi Subramanian successfully defended his PhD thesis: “Multi-Agent Reinforcement Learning in Large Complex Environments”
Subramanian, S. G. (2022). Multi-Agent Reinforcement Learning in Large Complex Environments [PhD thesis, UWSpace]. https://doi.org/http://hdl.handle.net/10012/18442
Sriram started a postdoc position at the Vector Institute in September 2022 working with Prof. Sheila Mcllraith at UofT and Prof. Pascal Poupart at UWaterloo.
Former PhD student Benyamin Ghojogh has taken up a new position at Research Associate at KisoJi Biotechnology working on exciting research on
We are working on developing AI tools for paratope-epitope interaction prediction, such as Machine Learning methods to describe VHH Antibodies for semantic clustering, selection, and matching. We aim to expedite the drug discovery process through the in-silico methodologies.
Can’t wait to hear more Benyamin!
Promotion and Tenure
As of July 1, 2022, Dr. Mark Crowley is now a tenured, Associate Professor in the ECE department! Phew.
- MARLEmpircalInvestigation of Independent Reinforcement Learning Algorithms in Multi-Agent EnvironmentsFrontiers in Artificial Intelligence. sep, 2022.
- MARLEmpircalInvestigation of Independent Reinforcement Learning Algorithms in Multi-Agent EnvironmentsIn NeurIPS 2021 Deep Reinforcement Learning Workshop dec, 2021.