Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Markov decision process
open-in-new
<p>Markov decision process (MDP), also called a <a href="/facts/Stochastic_dynamic_programming/0UhPYmgI">stochastic dynamic program</a> or stochastic control problem, is a model for <a href="/facts/Sequential_decision_making/48oS2neR">sequential decision making</a> when <a href="/facts/Outcome_(probability)/1hdJcoQR">outcomes</a> are uncertain. </p><p>Originating from <a href="/facts/Operations_research/H5yJkV5n">operations research</a> in the 1950s, MDPs have since gained recognition in a variety of fields, including <a href="/facts/Ecology/YE8upuJU">ecology</a>, <a href="/facts/Economics/HYctA2Dn">economics</a>, <a href="/facts/Health_care/LPS1WKbI">healthcare</a>, <a href="/facts/Telecommunications/vxoj1cxR">telecommunications</a> and <a href="/facts/Reinforcement_learning/NrgPPS0Q">reinforcement learning</a>. Reinforcement learning utilizes the MDP framework to model the interaction between a learning agent and its environment. In this framework, the interaction is characterized by states, actions, and rewards. The MDP framework is designed to provide a simplified representation of key elements of <a href="/facts/Artificial_intelligence/lJGauQwX">artificial intelligence</a> challenges. These elements encompass the understanding of <a href="/facts/Causality/Swh496X7">cause and effect</a>, the management of uncertainty and nondeterminism, and the pursuit of explicit goals. </p><p>The name comes from its connection to <a href="/facts/Markov_chain/5oP5hntk">Markov chains</a>, a concept developed by the Russian mathematician <a href="/facts/Andrey_Markov/F409PDqD">Andrey Markov</a>. The "Markov" in "Markov decision process" refers to the underlying structure of <a href="/facts/Transition_system/AABVmNzn">state transitions</a> that still follow the <a href="/facts/Markov_property/g1Fs9GFS">Markov property</a>. The process is called a "decision process" because it involves making decisions that influence these state transitions, extending the concept of a Markov chain into the realm of decision-making under uncertainty. </p>