If the uncoded BPSK system in AWGN environment has a bit error rate (BER) of 10−2 at the SNR level 4 dB, and the corresponding coded (e.g., BCH) system has the same BER at an SNR of 2.5 dB, then we say the coding gain = 4 dB − 2.5 dB = 1.5 dB, due to the code used (in this case BCH).
In the power-limited regime (where the nominal spectral efficiency ρ ≤ 2 {\displaystyle \rho \leq 2} [b/2D or b/s/Hz], i.e. the domain of binary signaling), the effective coding gain γ e f f ( A ) {\displaystyle \gamma _{\mathrm {eff} }(A)} of a signal set A {\displaystyle A} at a given target error probability per bit P b ( E ) {\displaystyle P_{b}(E)} is defined as the difference in dB between the E b / N 0 {\displaystyle E_{b}/N_{0}} required to achieve the target P b ( E ) {\displaystyle P_{b}(E)} with A {\displaystyle A} and the E b / N 0 {\displaystyle E_{b}/N_{0}} required to achieve the target P b ( E ) {\displaystyle P_{b}(E)} with 2-PAM or (2×2)-QAM (i.e. no coding). The nominal coding gain γ c ( A ) {\displaystyle \gamma _{c}(A)} is defined as
This definition is normalized so that γ c ( A ) = 1 {\displaystyle \gamma _{c}(A)=1} for 2-PAM or (2×2)-QAM. If the average number of nearest neighbors per transmitted bit K b ( A ) {\displaystyle K_{b}(A)} is equal to one, the effective coding gain γ e f f ( A ) {\displaystyle \gamma _{\mathrm {eff} }(A)} is approximately equal to the nominal coding gain γ c ( A ) {\displaystyle \gamma _{c}(A)} . However, if K b ( A ) > 1 {\displaystyle K_{b}(A)>1} , the effective coding gain γ e f f ( A ) {\displaystyle \gamma _{\mathrm {eff} }(A)} is less than the nominal coding gain γ c ( A ) {\displaystyle \gamma _{c}(A)} by an amount which depends on the steepness of the P b ( E ) {\displaystyle P_{b}(E)} vs. E b / N 0 {\displaystyle E_{b}/N_{0}} curve at the target P b ( E ) {\displaystyle P_{b}(E)} . This curve can be plotted using the union bound estimate (UBE)
where Q is the Gaussian probability-of-error function.
For the special case of a binary linear block code C {\displaystyle C} with parameters ( n , k , d ) {\displaystyle (n,k,d)} , the nominal spectral efficiency is ρ = 2 k / n {\displaystyle \rho =2k/n} and the nominal coding gain is kd/n.
The table below lists the nominal spectral efficiency, nominal coding gain and effective coding gain at P b ( E ) ≈ 10 − 5 {\displaystyle P_{b}(E)\approx 10^{-5}} for Reed–Muller codes of length n ≤ 64 {\displaystyle n\leq 64} :
In the bandwidth-limited regime ( ρ > 2 b / 2 D {\displaystyle \rho >2~b/2D} , i.e. the domain of non-binary signaling), the effective coding gain γ e f f ( A ) {\displaystyle \gamma _{\mathrm {eff} }(A)} of a signal set A {\displaystyle A} at a given target error rate P s ( E ) {\displaystyle P_{s}(E)} is defined as the difference in dB between the S N R n o r m {\displaystyle SNR_{\mathrm {norm} }} required to achieve the target P s ( E ) {\displaystyle P_{s}(E)} with A {\displaystyle A} and the S N R n o r m {\displaystyle SNR_{\mathrm {norm} }} required to achieve the target P s ( E ) {\displaystyle P_{s}(E)} with M-PAM or (M×M)-QAM (i.e. no coding). The nominal coding gain γ c ( A ) {\displaystyle \gamma _{c}(A)} is defined as
This definition is normalized so that γ c ( A ) = 1 {\displaystyle \gamma _{c}(A)=1} for M-PAM or (M×M)-QAM. The UBE becomes
where K s ( A ) {\displaystyle K_{s}(A)} is the average number of nearest neighbors per two dimensions.
MIT OpenCourseWare, 6.451 Principles of Digital Communication II, Lecture Notes sections 5.3, 5.5, 6.3, 6.4