Chapters 9-10: Prediction and Control using Approximation

Introduction

• Explanation: What is the on-Policy State Distribution
• Example: Prediction and Control with linear approximation for a simple grid world (no walls). Note: This contains the construction of state-action features.

Examples Where Simple Linear Features Fail

• Linear approximation for the 4x3 Grid World.
• Linear approximation for the L-Maze.

Advanced Feature Construction

• Control with linear approximation and Fourier basis features (4x3 Gridworld).
• Control with linear approximation and Fourier basis features (L-Maze).

Exercise

• Value Function Approximation for the Lunar Lander Problem using Tile Coding

License

© 2026 Michael Hahsler. All code and documents in this repository are provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.