Directed information

Directed information is an <a href="/facts/Information_theory/qNLH3U5P">information theory</a> measure that quantifies the <a href="/facts/Information/3LdrzXtJ">information</a> flow from the random string 
 
 
 
 
 X
 
 n
 
 
 =
 (
 
 X
 
 1
 
 
 ,
 
 X
 
 2
 
 
 ,
 …
 ,
 
 X
 
 n
 
 
 )
 
 
 {\displaystyle X^{n}=(X_{1},X_{2},\dots ,X_{n})}
 
 to the random string 
 
 
 
 
 Y
 
 n
 
 
 =
 (
 
 Y
 
 1
 
 
 ,
 
 Y
 
 2
 
 
 ,
 …
 ,
 
 Y
 
 n
 
 
 )
 
 
 {\displaystyle Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})}
 
. The term directed information was coined by <a href="/facts/James_Massey/qyfqKnQH">James Massey</a> and is defined as

I
        (
        
          X
          
            n
          
        
        →
        
          Y
          
            n
          
        
        )
        ≜
        
          ∑
          
            i
            =
            1
          
          
            n
          
        
        I
        (
        
          X
          
            i
          
        
        ;
        
          Y
          
            i
          
        
        
          |
        
        
          Y
          
            i
            −
            1
          
        
        )
      
    
    {\displaystyle I(X^{n}\to Y^{n})\triangleq \sum _{i=1}^{n}I(X^{i};Y_{i}|Y^{i-1})}

where 
 
 
 
 I
 (
 
 X
 
 i
 
 
 ;
 
 Y
 
 i
 
 
 
 |
 
 
 Y
 
 i
 −
 1
 
 
 )
 
 
 {\displaystyle I(X^{i};Y_{i}|Y^{i-1})}
 
 is the <a href="/facts/Conditional_mutual_information/MRsfVyhC">conditional mutual information</a> 
 
 
 
 I
 (
 
 X
 
 1
 
 
 ,
 
 X
 
 2
 
 
 ,
 .
 .
 .
 ,
 
 X
 
 i
 
 
 ;
 
 Y
 
 i
 
 
 
 |
 
 
 Y
 
 1
 
 
 ,
 
 Y
 
 2
 
 
 ,
 .
 .
 .
 ,
 
 Y
 
 i
 −
 1
 
 
 )
 
 
 {\displaystyle I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})}
 
. 
Directed information has applications to problems where <a href="/facts/Causality/Swh496X7">causality</a> plays an important role such as the <a href="/facts/Channel_capacity/55wJYsS6">capacity of channels</a> with feedback, capacity of discrete <a href="/facts/Memoryless/8EgzaWtS">memoryless</a> networks, capacity of networks with in-block memory, <a href="/facts/Gambling/Hgw4a9DH">gambling</a> with causal side information, <a href="/facts/Data_compression/gMFuY4FT">compression</a> with causal side information, <a href="/facts/Real-time_control/p9HxgL4L">real-time control</a> communication settings, and statistical physics.

Directed information open-in-new

Directed information