Economic modeling meets deep reinforcement learning

Abstract: Solving large-scale dynamic models hinges on methods capable of optimizing functions efficiently. Whereas classic dynamic programming methods are known to converge to optimal solutions, they generally are computationally too expensive to solve large-scale economic models. While reinforcement learning (RL) usually approximates the value function not as precise, it applies to much more complicated problems. Combined with function approximations like deep neural networks, RL has recently been successfully applied to robotic control, board games and economic models. However, while popular deep RL algorithms have been tested extensively on arcade games, it is an open question how they perform in terms of solving dynamic systems of equations with stochastic dynamics. This paper reviews several widely used deep RL algorithms and applies them to various economic models. Their performance is rigorously tested in different settings to find strengths and weaknesses regarding accuracy, robustness, computational resources, and ease of use.

A working paper is to be published very soon.

In case of questions, feedback or collaboration ideas, you can reach me via email: simon.haastert@wiwi.uni-muenster.de