Research

An Informed Actor-Critic Method for Economic Dynamics. (With Bo Li and Serguei Maliar)
Working paper, June 2026 draft.
Abstract: This paper applies deep reinforcement learning to high-dimensional optimal control in dynamic economic models. We introduce an informed actor-critic framework that embeds structural equations and constraints directly into the policy optimization loop. Unlike standard model-free reinforcement learning algorithms that treat the economy as a black box, the iAC method leverages analytical economic transitions. Applications to benchmark consumption-savings, Krusell-Smith, and overlapping-generations economies demonstrate that iAC eliminates unnecessary exploration noise and provides a precise, scalable, and sample-efficient framework for complex macroeconomic control problems.
Keywords: Reinforcement Learning; Dynamic Programming; Deep Learning; Heterogeneous Agents; Overlapping Generations; Computational Macroeconomics
[Working Paper]

Solving Discrete-Continuous Dynamic Choice Models: A Soft-to-Hard AI Solver with an Application to Sovereign Default. (With Bo Li and Serguei Maliar)
Working paper, June 2026 draft.
Abstract: This paper introduces a soft-to-hard (StH) deep reinforcement learning framework to solve discrete-continuous dynamic choice models. To overcome vanishing gradients caused by non-differentiable kinks, we augment an actor-critic architecture with distinct training and evaluation modes. During training, hard discrete switches are replaced with soft choice probabilities, allowing gradients to pass through value branches and policy networks. The smoothing is temporary: the temperature parameter is annealed toward zero, and evaluation returns to the original hard-switch environment. Applications to one- and multi-period sovereign default problems show that the StH solver delivers accurate, scalable solutions without relying on permanent taste shocks or random maturity assumptions.
Keywords: Sovereign Default; Long-Term Debt; Reinforcement Learning; Actor-Critic; Computational Economics
[Working Paper]

AI Taxing AI: A Multi-Agent Reinforcement Learning Approach to Optimal Taxation (With Bo Li)
Abstract: Artificial intelligence (AI) is improving productivity, yet its economy-wide diffusion may reshape the labor market and the distribution of income. To address these issues, we develop a Multi-Agent Reinforcement Learning (MARL) model featuring endogenous skill accumulation. We find that while AI boosts aggregate output via rapid process of skill sharing, it displaces middle-skilled humans and increases inequality. However, optimal taxation can redistribute the technological surplus to achieve Pareto improvements. To find the optimal tax schedule, we employ Dual-Loop Multi-Agent Reinforcement Learning to account for both government's fiscal policy and agent behaviors. Furthermore, we demonstrate that a ``Digital Commons'' regime---combining universal access with real-time knowledge sharing---maximizes social welfare by mitigating the trade-off between efficiency and equity.
Artificial Intelligence; Optimal Taxation ; Multi-Agent Reinforcement Learning; Inequality; Knowledge Diffusion
[Link to Paper/Draft]

A Quantitative Assessment of Pension Reform in China: Contribution, Benefit and Delayed Retirement. (With Bo Li and Yi Lu)
Working paper, December 2025 draft.
Abstract: How does delayed retirement policy affect agent behavior and the aggregate economy in China? To evaluate this policy, we develop a heterogeneous OLG model with endogenous retirement choice and the contribution/benefit rules of the social security system in China. We find that an increase in the lower contribution bound significantly boosts consumption and welfare, while simultaneously improving pension finances and income equity. Furthermore, introducing a delayed retirement policy yields additional gains, as it increases welfare and reduces the pension deficit. However, the policy's effects are heterogeneous across different groups, and life-cycle contribution rules are important for policy design.
Keywords: Social Security; Delayed Retirement; Heterogeneous Agents
[Working Paper]

Heterogeneity Models Introduce Time-Varying Risk Aversion Affected by Economic Shocks. (With Bo Li)
Abstract: We introduce a time-varying risk aversion to the Aiyagari model and link the time-varying risk aversion to heterogeneous shocks. We also introduce different types of individuals who change their risk aversion differently by shocks. The new model enriches the variety of individual heterogeneity without changing the random shocks and fits better Gini coefficients for wealth. We find that the new model obtains the above effect through two mechanisms: more parameter dimensions and influence on saving behavior.
Keywords: Heterogeneous Agents; Risk Aversion; Economic Shocks; Wealth Inequality
[Link to Paper/Draft]

Forecasting Crude Oil Spot Prices Using a Transformer-BiLSTM Architecture with NRBO-Based Hyperparameter Optimization. (With Wenhui Huang et al.)
R&R at North American Journal of Economics and Finance.
Abstract: A future vision of global crude oil markets is crucial for market participants and policymakers. However, making an accurate prediction of crude oil prices is always challenging, due to the difficulty in extracting both global and local information from historical price data and optimizing hyperparameter settings of the forecasting framework. This research proposes a Transformer-BiLSTM-NRBO forecasting model architecture. Experiments conducted on daily Brent and WTI crude oil spot prices from 2013 to 2024 illustrate the robust forecasting capability of the proposed method.
Keywords: Crude Oil Prices; Forecasting; Transformer; BiLSTM; Hyperparameter Optimization

Asset Pricing in China's Stock Market. (With Shuaiyu Jiang)
Finance & Economic Vision, 2022.
Abstract: This paper uses traditional machine learning methods and deep neural networks based on both firm-specific characteristics and macroeconomic variables to price China's A-share stock market. We give the stochastic discount factor a flexible form and compare different models' performances. Since the Chinese government adopts various policies to maintain financial stability, we borrow the idea from generative adversarial networks to find the true SDF by selecting moment conditions that minimize return volatility.
Keywords: Asset Pricing; Machine Learning; Deep Learning; Stochastic Discount Factor; China Stock Market
[Link to Paper]

Learning to Expect: Conditional Expectation Functions for Neural Economic Solvers.
Abstract: Neural and model-free methods provide flexible solution frameworks for dynamic economic models, but their Bellman targets still rely on conditional expectations of continuation values, commonly approximated by one-shot Monte Carlo draws that are then discarded. This project develops W-exp, a learned conditional expectation module that converts simulated shock information into a reusable expectation function over post-decision states and improves the expectation-construction step shared by many neural and simulator-based solvers.
Keywords: Conditional Expectations; Neural Solvers; Reinforcement Learning; Dynamic Economic Models

Reduced-State Deep Learning for Heterogeneous Agent Models.
Abstract: This project develops a reduced-state neural solution method for heterogeneous agent models with aggregate uncertainty. Instead of feeding the full cross-sectional distribution into policy and value networks, we approximate the aggregate state using a low-dimensional vector of moments, such as mean capital and its dispersion, that is updated via simulation. The approach bridges the gap between belief-based approximations and full-distribution neural methods while maintaining computational tractability for high-dimensional HAM environments.
Keywords: Heterogeneous Agent Models; Deep Learning; Aggregate Uncertainty; Reduced-State Approximation