Convergent reinforcement learning
When do value-based methods with function approximation converge as representations or data distributions change?
Research
Reinforcement learning across timescales and levels of abstraction.
My work studies convergence, hierarchy, temporal abstraction, planning, and multi-agent coordination—from mathematical analysis to the evaluation of deployed agentic systems.
Hierarchical reinforcement learning
How should agents learn, reuse, and terminate temporally extended skills across long-horizon tasks?
Adaptive planning & computation
When should an agent plan, act, reuse a plan, or allocate more reasoning to a decision?
Reliable agentic systems
How can trace-based evaluations detect state, coordination, and execution failures in conversational and commerce agents?
Connect
Academic profiles
Research community
Advisors & coauthors
People behind the work
Publications
Complete list
12 works · 2020—2026
01
02
User-State Verification in Conversational Commerce: Detecting Journey Hallucinations via Trace Invariants
34th ACM Conference on User Modeling, Adaptation and Personalization (UMAP)
A lightweight verification framework that reconstructs transactional user state from execution logs and checks agent claims against deterministic invariants.
03
Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning via Predictive Observation Imputation
Artificial Intelligence; extended abstract at AAMAS 2024 and abstract reprint at AAAI 2026
Multi-agent policies use predictive observation imputation to adapt to arbitrary communication levels at execution time.
04
05
06
Regularization and Two Time Scales for Convergence of Reinforcement Learning
Applied Mathematics & Optimization
Convergence results for reinforcement learning through regularization and two-time-scale stochastic approximation.
07
Reinforcement Learning in Convergently Non-Stationary Environments: Feudal Hierarchies and Learned Representations
Artificial Intelligence
A principled treatment of reinforcement learning when environment dynamics converge over time, using hierarchical structure and learned representations.
08
09
10
11
12
A New Convergent Variant of Q-Learning with Linear Function Approximation
Advances in Neural Information Processing Systems (NeurIPS)
Research materials
Talks, posters & research notes
Browser-friendly PDFs
Publication materials
Multi-Bellman Operator for Convergence of Q-Learning
Conference posterPoster overview of the multi-Bellman operator and the convergence of Q-learning with linear function approximation.
Open PosterAutoclave Loading Using Deep Reinforcement Learning
EPIA 2025 presentationConference slides on deep reinforcement learning for two-dimensional autoclave loading.
Open SlidesTheoretical Remarks on Feudal Hierarchies and Reinforcement Learning
ECAI 2023 posterPoster accompanying the ECAI 2023 outstanding-paper presentation.
Open PosterTheoretical Remarks on Feudal Hierarchies and Reinforcement Learning
ECAI 2023 presentationPresentation slides for the formal analysis of learning in feudal reinforcement-learning hierarchies.
Open SlidesFeudal Hierarchies and Reinforcement Learning
RLDM 2022 posterThe distinct poster presented at the 2022 Multidisciplinary Conference on Reinforcement Learning and Decision Making.
Open PosterLectures & seminars
Hierarchical Reinforcement Learning
Invited lecture · KTH Royal Institute of TechnologyFull progressive-build lecture deck on abstraction, feudal hierarchies, options, task discovery, and eigenoptions. The lecture video remains the primary record.
Open SlidesReinforcement Learning with Many Discrete Actions
GAIPS Reinforcement Learning Reading GroupReading-group lecture on reinforcement learning methods for large discrete action spaces.
GAIPS RL Reading Group
Open Slides The Problem of Reinforcement Learning with Many Discrete Actions
Accompanying survey · GAIPS Reinforcement Learning Reading GroupA short survey of the challenges, existing approaches, and research directions for reinforcement learning with large discrete action spaces.
GAIPS RL Reading Group
Open Document Metrics on Markov Decision Processes
GAIPS Reading Group · undatedA research lecture on metrics for finite MDPs, approximate MDP homomorphisms, and policy transfer.
GAIPS Reading Group
Open Slides Research milestones
Cover Time of Options in One-Dimensional Environments
Joint research note / work in progressWork in progress with Marlos C. Machado on structured exploration and the cover time of learned options in a one-dimensional environment.
Amii · University of Alberta research visit
Open Research note Convergence and Performance Results for Reinforcement Learning with Two Time Scales
PhD thesis proposalThe thesis proposal defended at Instituto Superior Técnico on 11 September 2023.
Instituto Superior Técnico
Open Document PhD Thesis Proposal Defence
PhD milestone · Instituto Superior TécnicoProgressive-build slides used for the thesis proposal defence.
Instituto Superior Técnico
Open Slides Appearances
Talks & presentations archive
11 records
presentation
Benchmarking Models for Conversational E-Commerce
GenAIECommerce at RecSys 2026 · MinneapolisWorkshop presentation of a reproducible evaluation framework for shopping agents.
presentation
Multi-Bellman Operator
ELLIS Doctoral Symposium 2026 · LisbonPoster presentation at the ELLIS Doctoral Symposium on Trustworthy AI.
presentation
User-State Verification in Conversational Commerce
UMAP 2026 · GothenburgPresentation on detecting journey hallucinations through trace invariants.
presentation
Multi-Bellman Operator
ICLR 2026 · Rio de JaneiroJournal-to-conference poster presentation of the TMLR paper.
presentation
Autoclave Loading Using Deep Reinforcement Learning
EPIA 2025 · FaroPresentation on deep reinforcement learning for two-dimensional autoclave loading.
talk
Hierarchical Reinforcement Learning
KTH Royal Institute of TechnologyInvited lecture for the Reinforcement Learning course; video is the primary record and the full progressive-build deck is also available.
presentation
Theoretical Remarks on Feudal Hierarchies and Reinforcement Learning
ECAI 2023 · KrakówOutstanding-paper presentation and poster at the 26th European Conference on Artificial Intelligence.
milestone
PhD Thesis Proposal Defence
Instituto Superior Técnico · LisbonDefence of the thesis proposal on convergence and performance results for reinforcement learning with two time scales.
presentation
Feudal Hierarchies and Reinforcement Learning
RLDM 2022 · ProvidencePoster presentation at the Multidisciplinary Conference on Reinforcement Learning and Decision Making.
talk
Reinforcement Learning with Many Discrete Actions
GAIPS Reinforcement Learning Reading GroupReading-group lecture on methods for large discrete action spaces.
talk
Metrics on Markov Decision Processes
GAIPS Reading GroupResearch lecture on MDP metrics, approximate homomorphisms, and policy transfer.