Optimization and Decision Making

I am interested in how make decision in complex domains especially when there are vary large state and action spaces. A wide range of optimization algorithms have been developed that are each appropriate in different types of problems. My policy gradient optimization algorithm for spatiotemporal domains, Equilibrium Policy Gradients utilizes a parametrized policy defined as the equilibrium distribution of a Markov chain built from local causal policies. I have used this algorithm to deal with the complexity caused by spatial structure common in computational sustainability domains. I also work on MDP planning algorithms with provable confidence intervals upon completion and other Reinforcement Learning approaches.

Graphical Models

Some of my work looks at modelling, inference and learning of interpretable probabilistic graphical models. I am particularly interested in cases where the underlying data is relational, such as social networks, or there is a need for both causal and correlational structure, cyclic structure in other words, within the same model. This arises naturally when trying to represent spatial policies using graphical models but cyclic structure can arise in many relational domains and existing fully directed or fully undirected models have trouble dealing with it.

Tree Learning

Other topics I am working on in Machine Learning include streaming decision tree learning for large datasets and scale invariant anomaly detection.

Computational Sustainability

The field of Computational Sustainability is at the intersection between computational sciences (such as artificial intelligence,  computational modelling, optimization and planning research) with applied research in environmental/ecological domains (such as land use management, invasive species spread, sustainable ecology management, smart grids and species tracking).

Some Past Papers

Here are some of the problems I am looking at and related papers:

• How can we compactly represent expressive and interpretable policies for acting in large spatial domains with correlated actions?
  •  Cyclic Causal Models with Discrete Variables : Markov Chain Equilibrium Semantics and Sample Ordering
• How can we effectively and efficiently find optimal (or approximately optimal) policies for large spatiotemporal problems?
  • Optimization Algorithms
  • Using Equilibrium Policy Gradients for Spatiotemporal Planning in Forest Ecosystem Management
  • Policy Gradient Planning for Environmental Decision Making with Existing Simulators
• If our optimization approach is approximate, how do we bound the error so that we know something about how close we are to optimal?
  • PAC Optimal Planning for Invasive Species Management : Improved Exploration for Reinforcement Learning from Simulator-Defined MDPs
• How can we use probabilistic models for important problems without creating biases?
  • Adding Local Constraints to Bayesian Networks
  • Cyclic Causal Models with Discrete Variables : Markov Chain Equilibrium Semantics and Sample Ordering