Chapters 9-10: Prediction and Control using Approximation

Introduction

• Example: Prediction and Control with linear approximation for a simple grid world (no walls).
• Explanation: On-Policy State Distribution

Examples Where Simple Linear Features Fail

• Example: Linear approximation for the 4x3 Grid World.
• Example: Linear approximation for the L-Maze.

Advanced Feature Construction

• Example: Control with linear approximation and Fourier basis features (4x3 Gridworld).
• Example: Control with linear approximation and Fourier basis features (L-Maze).

License

© 2026 Michael Hahsler. All code and documents in this repository are provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.