抽象动态规划(第2版）

"本书以动态规划为基础，运用抽象映射的单调性和压缩映射理论研究近似动态规划或动态规划的若干典型问题，主要特点是：不涉及所讨论问题的随机特性，也不涉及特殊类型的动态规划问题的某些有趣特征。本书中展示的理论方法位居随机运筹学和随机很优控制领域的学科前沿，其严谨的分析方法和处理技巧具有重要的理论价值，在数学与人工智能科学的交叉研究领域具有广阔的应用前景。 "

本书利用不动点理论问题的紧密关联性，阐述以总成本为目标的序贯决策问题的核心理论和算法的近期新研究进展，重点讨论以动态规划为基础的抽象映射，并定义了相关的数学特征。本书聚焦于抽象映射的两个基本性质——单调性和（加权超范数）压缩性。事实上，动态规划理论分析和算法的本质主要取决于这两种属性是否存在，而问题的其余结构在很大程度上是无关紧要的。除了极特别的情况，本书将始终假设单调性成立，围绕抽象映射的压缩性特点研究了四类主要模型：压缩模型、半压缩模型、非压缩模型、受限策略和Borel空间模型。本书以动态规划为基础，运用抽象映射的单调性和压缩映射理论研究近似动态规划或动态规划的若干典型问题，主要特点是：不涉及所讨论问题的随机特性，也不涉及特殊类型的动态规划问题的某些有趣特征。本书中展示的理论方法位居随机运筹学和随机很优控制领域的学科前沿，其严谨的分析方法和处理技巧具有重要的理论价值，在数学与人工智能科学的交叉研究领域具有广阔的应用前景。

德梅萃 P.博塞克斯（Dimitri P. Bertseka）,美国MIT终身教授，美国国家工程院院士，清华大学复杂与网络化系统研究中心客座教授。电气工程与计算机科学领域国际知名作者，著有《非线性规划》《网络优化》《凸优化》等十几本畅销教材和专著。

1 Introduction
1.1 Structure of Dynamic Programming Problems
1.2 Abstract Dynamic Programming Models
1.2.1 Problem Formulation
1.2.2 Monotonicity and Contraction Properties
1.2.3 Some Examples
1.2.4 Approximation Models-Projected and Aggregation Bellman Equations
1.2.5 Multistep Models-Temporal Difference and Proximal Algorithms
1.3 Organization of the Book
1.4 Notes, Sources, and Exercises
2 Contractive Models
2.1 Bellman's Equation and Optimality Conditions
2.2 Limited Lookahead Policies
2.3 Value Iteration
2.4 Policy Iteration
2.4.1 Approximate Policy Iteration
2.4.2 Approximate Policy Iteration Where Policies Converge
2.5 Optimistic Policy Iteration and λ-Policy Iteration
2.5.1 Convergence of Optimistic Policy Iteration
2.5.2 Approximate Optimistic Policy Iteration
2.5.3 Randomized Optimistic Policy Iteration
2.6 Asynchronous Algorithms
2.6.1 Asynchronous Value Iteration
2.6.2 Asynchronous Policy Iteration
2.6.3 Optimistic Asynchronous Policy Iteration with a Uniform Fixed Point
2.7 Notes, Sources, and Exercises
3 Semicontractive Models
3.1 Pathologies of Noncontractive DP Models
3.1.1 Deterministic Shortest Path Problems
3.1.2 Stochastic Shortest Path Problems
3.1.3 The Blackmailer's Dilemma
3.1.4 Linear-Quadratic Problems
3.1.5 An Intuitive View of Semicontractive Analysis
3.2 Semicontractive Models and Regular Policies
3.2.1 S-Regular Policies
3.2.2 Restricted Optimization over S-Regular Policies
3.2.3 Policy Iteration Analysis of Bellman's Equation
3.2.4 Optimistic Policy Iteration and λ-Policy Iteration
3.2.5 A Mathematical Programming Approach
3.3 Irregular Policies/Infinite Cost Case
3.4 Irregular Policies/Finite Cost Case-A Perturbation Approach
3.5 Applications in Shortest Path and Other Contexts
3.5.1 Stochastic Shortest Path Problems
3.5.2 Affine Monotonic Problems
3.5.3 Robust Shortest Path Planning
3.5.4 Linear-Quadratic Optimal Control
3.5.5 Continuous-State Deterministic Optimal Control
3.6 Algorithms
3.6.1 Asynchronous Value Iteration
3.6.2 Asynchronous Policy Iteration
3.7 Notes, Sources, and Exercises
4 Noncontractive Models
4.1 Noncontractive Models-Problem Formulation
4.2 Finite Horizon Problems
4.3 Infinite Horizon Problems
4.3.1 Fixed Point Properties and Optimality Conditions
4.3.2 Value Iteration
4.3.3 Exact and Optimistic Policy Iteration-λ-Policy Iteration
4.4 Regularity and Nonstationary Policies
4.4.1 Regularity and Monotone Increasing Models
4.4.2 Nonnegative Cost Stochastic Optimal Control
4.4.3 Discounted Stochastic Optimal Control
4.4.4 Convergent Models
4.5 Stable Policies for Deterministic Optimal Control
4.5.1 Forcing Functions and p-Stable Policies
4.5.2 Restricted Optimization over Stable Policies
4.5.3 Policy Iteration Methods
4.6 Infinite-Spaces Stochastic Shortest Path Problems
4.6.1 The Multiplicity of Solutions of Bellman's Equation
4.6.2 The Case of Bounded Cost per Stage
4.7 Notes, Sources, and Exercises
Appendix A: Notation and Mathematical Conventions
A.1 Set Notation and Conventions
A.2 Functions
Appendix B: Contraction Mappings
B.1 Contraction Mapping Fixed Point Theorems
B.2 Weighted Sup-Norm Contractions
References
Index