Basu's theorem

<h2 id="statement">Statement</h2>
Let 
 
 
 
 (
 
 P
 
 θ
 
 
 ;
 θ
 ∈
 Θ
 )
 
 
 {\displaystyle (P_{\theta };\theta \in \Theta )}
 
 be a family of distributions on a <a href="/facts/Measurable_space/AJ98lhT4">measurable space</a> 
 
 
 
 (
 X
 ,
 
 
 A
 
 
 )
 
 
 {\displaystyle (X,{\mathcal {A}})}
 
 and a <a href="/facts/Statistic/av21WaMK">statistic</a> 
 
 
 
 T
 
 
 {\displaystyle T}
 
 maps from 
 
 
 
 (
 X
 ,
 
 
 A
 
 
 )
 
 
 {\displaystyle (X,{\mathcal {A}})}
 
 to some measurable space 
 
 
 
 (
 Y
 ,
 
 
 B
 
 
 )
 
 
 {\displaystyle (Y,{\mathcal {B}})}
 
. If 
 
 
 
 T
 
 
 {\displaystyle T}
 
 is a boundedly complete sufficient statistic for 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
, and 
 
 
 
 A
 
 
 {\displaystyle A}
 
 is ancillary to 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
, then conditional on 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
, 
 
 
 
 T
 
 
 {\displaystyle T}
 
 is independent of 
 
 
 
 A
 
 
 {\displaystyle A}
 
. That is, 
 
 
 
 T
 ⊥
 
 
 
 ⊥
 A
 
 |
 
 θ
 
 
 {\displaystyle T\perp \!\!\!\perp A|\theta }
 
.

<h3>Proof</h3>
Let 
 
 
 
 
 P
 
 θ
 
 
 T
 
 
 
 
 {\displaystyle P_{\theta }^{T}}
 
 and 
 
 
 
 
 P
 
 θ
 
 
 A
 
 
 
 
 {\displaystyle P_{\theta }^{A}}
 
 be the <a href="/facts/Marginal_distribution/U9XBWAd1">marginal distributions</a> of 
 
 
 
 T
 
 
 {\displaystyle T}
 
 and 
 
 
 
 A
 
 
 {\displaystyle A}
 
 respectively.
Denote by 
 
 
 
 
 A
 
 −
 1
 
 
 (
 B
 )
 
 
 {\displaystyle A^{-1}(B)}
 
 the <a href="/facts/Preimage/KANefMAl">preimage</a> of a set 
 
 
 
 B
 
 
 {\displaystyle B}
 
 under the map 
 
 
 
 A
 
 
 {\displaystyle A}
 
. For any measurable set 
 
 
 
 B
 ∈
 
 
 B
 
 
 
 
 {\displaystyle B\in {\mathcal {B}}}
 
 we have

P
          
            θ
          
          
            A
          
        
        (
        B
        )
        =
        
          P
          
            θ
          
        
        (
        
          A
          
            −
            1
          
        
        (
        B
        )
        )
        =
        
          ∫
          
            Y
          
        
        
          P
          
            θ
          
        
        (
        
          A
          
            −
            1
          
        
        (
        B
        )
        ∣
        T
        =
        t
        )
         
        
          P
          
            θ
          
          
            T
          
        
        (
        d
        t
        )
        .
      
    
    {\displaystyle P_{\theta }^{A}(B)=P_{\theta }(A^{-1}(B))=\int _{Y}P_{\theta }(A^{-1}(B)\mid T=t)\ P_{\theta }^{T}(dt).}

The distribution 
 
 
 
 
 P
 
 θ
 
 
 A
 
 
 
 
 {\displaystyle P_{\theta }^{A}}
 
 does not depend on 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
 because 
 
 
 
 A
 
 
 {\displaystyle A}
 
 is ancillary. Likewise, 
 
 
 
 
 P
 
 θ
 
 
 (
 ⋅
 ∣
 T
 =
 t
 )
 
 
 {\displaystyle P_{\theta }(\cdot \mid T=t)}
 
 does not depend on 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
 because 
 
 
 
 T
 
 
 {\displaystyle T}
 
 is sufficient. Therefore

∫
          
            Y
          
        
        
          
            [
          
        
        P
        (
        
          A
          
            −
            1
          
        
        (
        B
        )
        ∣
        T
        =
        t
        )
        −
        
          P
          
            A
          
        
        (
        B
        )
        
          
            ]
          
        
         
        
          P
          
            θ
          
          
            T
          
        
        (
        d
        t
        )
        =
        0.
      
    
    {\displaystyle \int _{Y}{\big [}P(A^{-1}(B)\mid T=t)-P^{A}(B){\big ]}\ P_{\theta }^{T}(dt)=0.}

Note the integrand (the function inside the integral) is a function of 
 
 
 
 t
 
 
 {\displaystyle t}
 
 and not 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
. Therefore, since 
 
 
 
 T
 
 
 {\displaystyle T}
 
 is boundedly complete the function

g
        (
        t
        )
        =
        P
        (
        
          A
          
            −
            1
          
        
        (
        B
        )
        ∣
        T
        =
        t
        )
        −
        
          P
          
            A
          
        
        (
        B
        )
      
    
    {\displaystyle g(t)=P(A^{-1}(B)\mid T=t)-P^{A}(B)}

is zero for 
 
 
 
 
 P
 
 θ
 
 
 T
 
 
 
 
 {\displaystyle P_{\theta }^{T}}
 
 almost all values of 
 
 
 
 t
 
 
 {\displaystyle t}
 
 and thus

P
        (
        
          A
          
            −
            1
          
        
        (
        B
        )
        ∣
        T
        =
        t
        )
        =
        
          P
          
            A
          
        
        (
        B
        )
      
    
    {\displaystyle P(A^{-1}(B)\mid T=t)=P^{A}(B)}

for almost all 
 
 
 
 t
 
 
 {\displaystyle t}
 
. Therefore, 
 
 
 
 A
 
 
 {\displaystyle A}
 
 is independent of 
 
 
 
 T
 
 
 {\displaystyle T}
 
.

<h2 id="example">Example</h2>
<h3>Independence of sample mean and sample variance of a normal distribution</h3>
Let X1, X2, ..., Xn be <a href="/facts/Independent_and_identically-distributed_random_variables/othIRaWt">independent, identically distributed</a> <a href="/facts/Normal_distribution/UapjjPyQ">normal</a> <a href="/facts/Random_variable/TwTBXnLT">random variables</a> with <a href="/facts/Mean/swcEd4Pg">mean</a> μ and <a href="/facts/Variance/ULBJKXD1">variance</a> σ2.
Then with respect to the parameter μ, one can show that

μ
              ^
            
          
        
        =
        
          
            
              ∑
              
                X
                
                  i
                
              
            
            n
          
        
        ,
      
    
    {\displaystyle {\widehat {\mu }}={\frac {\sum X_{i}}{n}},}

the sample mean, is a complete and sufficient statistic – it is all the information one can derive to estimate μ, and no more – and

σ
                ^
              
            
          
          
            2
          
        
        =
        
          
            
              ∑
              
                
                  (
                  
                    
                      X
                      
                        i
                      
                    
                    −
                    
                      
                        
                          X
                          ¯
                        
                      
                    
                  
                  )
                
                
                  2
                
              
            
            
              n
              −
              1
            
          
        
        ,
      
    
    {\displaystyle {\widehat {\sigma }}^{2}={\frac {\sum \left(X_{i}-{\bar {X}}\right)^{2}}{n-1}},}

the sample variance, is an ancillary statistic – its distribution does not depend on μ.
Therefore, from Basu's theorem it follows that these statistics are independent conditional on 
 
 
 
 μ
 
 
 {\displaystyle \mu }
 
, conditional on 
 
 
 
 
 σ
 
 2
 
 
 
 
 {\displaystyle \sigma ^{2}}
 
.
This independence result can also be proven by <a href="/facts/Cochran%27s_theorem/maUMfYzP">Cochran's theorem</a>.
Further, this property (that the sample mean and sample variance of the normal distribution are independent) <a href="/facts/Characterization_(mathematics)/mmT1knFI">characterizes</a> the normal distribution – no other distribution has this property.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a>

<h2 id="notes">Notes</h2>

<ul><li><a href="/facts/Debabrata_Basu/Agx5Sk1V">Basu, D.</a> (1955). "On Statistics Independent of a Complete Sufficient Statistic". <a href="/facts/Sankhya_(journal)/8lxTxfts">Sankhyā</a>. 15 (4): 377–380. <a href="/facts/JSTOR_(identifier)/YTeVmaJ7">JSTOR</a> <a href="https://www.jstor.org/stable/25048259">25048259</a>. <a href="/facts/MR_(identifier)/uP137L11">MR</a> <a href="https://mathscinet.ams.org/mathscinet-getitem?mr=0074745">0074745</a>. <a href="/facts/Zbl_(identifier)/P6rFxKKx">Zbl</a> <a href="https://zbmath.org/?format=complete&q=an:0068.13401">0068.13401</a>.</li>
<li>Mukhopadhyay, Nitis (2000). Probability and Statistical Inference. Statistics: A Series of Textbooks and Monographs. 162. Florida: CRC Press USA. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-8247-0379-0.</li>
<li>Boos, Dennis D.; Oliver, Jacqueline M. Hughes (Aug 1998). <a href="http://www.lib.ncsu.edu/resolver/1840.4/229">"Applications of Basu's Theorem"</a>. <a href="/facts/The_American_Statistician/XJPACVAB">The American Statistician</a>. 52 (3): 218–221. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.2307%2F2685927">10.2307/2685927</a>. <a href="/facts/JSTOR_(identifier)/YTeVmaJ7">JSTOR</a> <a href="https://www.jstor.org/stable/2685927">2685927</a>. <a href="/facts/MR_(identifier)/uP137L11">MR</a> <a href="https://mathscinet.ams.org/mathscinet-getitem?mr=1650407">1650407</a>.</li>
<li><a href="/facts/Malay_Ghosh/jICjWnyn">Ghosh, Malay</a> (October 2002). "Basu's Theorem with Applications: A Personalistic Review". Sankhyā: The Indian Journal of Statistics, Series A. 64 (3): 509–531. <a href="/facts/JSTOR_(identifier)/YTeVmaJ7">JSTOR</a> <a href="https://www.jstor.org/stable/25051412">25051412</a>. <a href="/facts/MR_(identifier)/uP137L11">MR</a> <a href="https://mathscinet.ams.org/mathscinet-getitem?mr=1985397">1985397</a>.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Basu (1955) <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Ghosh, Malay; Mukhopadhyay, Nitis; Sen, Pranab Kumar (2011), Sequential Estimation, Wiley Series in Probability and Statistics, vol. 904, John Wiley & Sons, p. 80, ISBN 9781118165911, The following theorem, due to Basu ... helps us in proving independence between certain types of statistics, without actually deriving the joint and marginal distributions of the statistics involved. This is a very powerful tool and it is often used ... <a href="9781118165911" target="_blank">9781118165911</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Geary, R.C. (1936). "The Distribution of "Student's" Ratio for Non-Normal Samples". Supplement to the Journal of the Royal Statistical Society. 3 (2): 178–184. doi:10.2307/2983669. JFM 63.1090.03. JSTOR 2983669. <a href="/wiki/Roy_C._Geary" target="_blank">/wiki/Roy_C._Geary</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
</ol>

Basu's theorem open-in-new

Basu's theorem