Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics
In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that, relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.
Replication materials may be found at https://github.com/tphelanECON/EslamiPhelan_MCA
Working Papers of the Federal Reserve Bank of Cleveland are preliminary materials circulated to stimulate discussion and critical comment on research in progress. They may not have been subject to the formal editorial review accorded official Federal Reserve Bank of Cleveland publications. The views expressed in this paper are those of the authors and do not represent the views of the Federal Reserve Bank of Cleveland or the Federal Reserve System.
Suggested Citation
Phelan, Tom, and Keyvan Eslami. 2022. “Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics.” Federal Reserve Bank of Cleveland, Working Paper No. 21-04R. https://doi.org/10.26509/frbc-wp-202104r
- Share