DE

Modul

Markov Decision Processes [M-MATH-102907]

Credits
5
Recurrence
Unregelmäßig
Duration
1 Semester
Language
Level
4
Version
1

Responsible

Organisation

  • KIT-Fakultät für Mathematik

Part of

Bricks

Identifier Name LP
T-MATH-105921 Markov Decision Processes 5

Competence Certificate

The module will be completed by an oral exam (about 20 min).

Competence Goal

At the end of the course, students

  • can name the mathematical foundations of Markov Decision Processes and apply solution algorithm,
  • can formulate stochastic, dynamic optimization problems as Markov Decision Processes,
  • are able to work in a self-organized and reflective manner.

Prerequisites

none

Content

  • MDPs with finite time horizon
    - Bellman equation
    - Problems with structure
    - Applications

  • MDPs with infinite time horizon
    - contracting MDPs
    - positive MDPs
    - Howards policy improvement
    - Solution by linear programs

  • Stopping problems
    - finite and infinite time horizon
    - One-step-look-ahead rule

Recommendation

The course 'Probability theory' is strongly recommended and 'Markov chains' is recommended.

Workload

Total workload: 150 hours

Attendance: 60 hours

  • lectures, problem classes, and examination 

Self-studies: 90 hours

  • follow-up and deepening of the course content,
  • work on problem sheets,
  • literature study and internet research relating to the course content,
  • preparation for the module examination