Generate and return the KullbackLeibler Divergence (KLD) \(D_{KL}(P \parallel Q)\) of a given univariate Normal distribution \(Q\) from a reference Normal distribution \(P\).
The KullbackLeibler Divergence, also known as the relative entropy, of a univariate Normal distribution \(Q\) from a reference univariate Normal distribution \(P\) is defined as,
\begin{equation}
\large
D_{KL}(P \parallel Q) =
\frac{(\mu_P  \mu_Q)^2}{2\sigma_Q^2} +
\frac{1}{2}\bigg( \frac{\sigma_P}{\sigma_Q}  \ln\big(\frac{\sigma_P}{\sigma_Q}\big)  1 \bigg) ~,
\end{equation}
where \(\mu\) and \(\sigma\) represent the respective standard deviations of the Normal distributions.
 Parameters

[in]  meanDiffSq  : The input scalar or array of the same shape as other arraylike arguments of the same type kind as the output kld , representing the differencesquared of the mean parameters of the two Normal distributions ( \((P, Q)\)).
(optional, default = 0. , that is, the two distributions have the same mean.) 
[in]  varP  : The input scalar or array of the same shape as other arraylike arguments, of the same type and kind as the output kld representing the variance of the reference Normal distribution \(P\).
(optional, default = 1. , it must be present if and only if varQ is also present.) 
[in]  varQ  : The input scalar or array of the same shape as other arraylike arguments, of the same type and kind as meanDiffSq representing the variance of the target Normal distribution \(Q\).
(optional, default = 1. , it must be present if and only if varP is also present.) 
 Returns
kld
: The input scalar or array of the same shape as input arraylike arguments, of,

type
real
of kind any supported by the processor (e.g., RK, RK32, RK64, or RK128),
representing the KullbackLeibler divergence of the target distribution with respect to the reference.
Possible calling interfaces ⛓
Generate and return the KullbackLeibler Divergence (KLD) of a given univariate Normal distribution ...
This module contains classes and procedures for computing various statistical quantities related to t...
 Warning
 The conditions
0 <= meanDiffSq
must hold for the corresponding input arguments.
The conditions 0 < varP
must hold for the corresponding input arguments.
The conditions 0 < varQ
must hold for the corresponding input arguments.
These conditions are verified only if the library is built with the preprocessor macro CHECK_ENABLED=1
.

The
pure
procedure(s) documented herein become impure
when the ParaMonte library is compiled with preprocessor macro CHECK_ENABLED=1
.
By default, these procedures are pure
in release
build and impure
in debug
and testing
builds.
Example usage ⛓
10 type(display_type) :: disp
14 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
15 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
16 call disp%show(–
"! Compute the KullbackLeibler Divergence (KLD) distance the Normal distribution Q from P.")
17 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
18 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
22 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKS  1.5_RKS)**2) ! assuming standard deviations of 1.")
24 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKD  1.5_RKD)**2)")
26 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKH  1.5_RKH)**2)")
31 call disp%show(
"getNormKLD(varP = 2._RKS , varQ = 3._RKS) ! assuming similar means.")
33 call disp%show(
"getNormKLD(varP = 2._RKD , varQ = 3._RKD)")
35 call disp%show(
"getNormKLD(varP = 2._RKH, varQ = 3._RKH)")
40 call disp%show(
"getNormKLD(varP = 3._RKS , varQ = 2._RKS ) ! assuming similar means.")
42 call disp%show(
"getNormKLD(varP = 3._RKD , varQ = 2._RKD )")
44 call disp%show(
"getNormKLD(varP = 3._RKH, varQ = 2._RKH)")
49 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKS , varP = 3._RKS , varQ = 2._RKS ! assuming similar means.")
51 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKD , varP = 3._RKD , varQ = 2._RKD )")
53 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKH, varP = 3._RKH, varQ = 2._RKH)")
This is a generic method of the derived type display_type with pass attribute.
This is a generic method of the derived type display_type with pass attribute.
This module contains classes and procedures for input/output (IO) or generic display operations on st...
type(display_type) disp
This is a scalar module variable an object of type display_type for general display.
This module defines the relevant Fortran kind typeparameters frequently used in the ParaMonte librar...
integer, parameter IK
The default integer kind in the ParaMonte library: int32 in Fortran, c_int32_t in CFortran Interoper...
integer, parameter RKD
The double precision real kind in Fortran mode. On most platforms, this is an 64bit real kind.
integer, parameter SK
The default character kind in the ParaMonte library: kind("a") in Fortran, c_char in CFortran Intero...
integer, parameter RKH
The scalar integer constant of intrinsic default kind, representing the highestprecision real kind t...
integer, parameter RKS
The singleprecision real kind in Fortran mode. On most platforms, this is an 32bit real kind.
Generate and return an object of type display_type.
Example Unix compile command via Intel ifort
compiler ⛓
3ifort fpp standardsemantics O3 Wl,rpath,../../../lib I../../../inc main.F90 ../../../lib/libparamonte* o main.exe
Example Windows Batch compile command via Intel ifort
compiler ⛓
2set PATH=..\..\..\lib;%PATH%
3ifort /fpp /standardsemantics /O3 /I:..\..\..\include main.F90 ..\..\..\lib\libparamonte*.lib /exe:main.exe
Example Unix / MinGW compile command via GNU gfortran
compiler ⛓
3gfortran cpp ffreelinelengthnone O3 Wl,rpath,../../../lib I../../../inc main.F90 ../../../lib/libparamonte* o main.exe
Example output ⛓
14+0.500000000000000000000000000000000000
20+0.36065887387415563E1
22+0.360658873874155243223398910655078833E1
28+0.47267445945917808E1
30+0.472674459459178090109934422678254821E1
33getNormKLD(meanDiffSq
= 0.2_RKS , varP
= 3._RKS , varQ
= 2._RKS
35getNormKLD(meanDiffSq
= 0.2_RKD , varP
= 3._RKD , varQ
= 2._RKD )
36+0.97267445945917810E1
37getNormKLD(meanDiffSq
= 0.2_RKH, varP
= 3._RKH, varQ
= 2._RKH)
38+0.972674459459178090109934422678254845E1
 Test:
 test_pm_distNorm
 Todo:
 High Priority: The KLD implementation should be implemented with
log1mexp
for better accuracy.
Final Remarks ⛓
If you believe this algorithm or its documentation can be improved, we appreciate your contribution and help to edit this page's documentation and source file on GitHub.
For details on the naming abbreviations, see this page.
For details on the naming conventions, see this page.
This software is distributed under the MIT license with additional terms outlined below.

If you use any parts or concepts from this library to any extent, please acknowledge the usage by citing the relevant publications of the ParaMonte library.

If you regenerate any parts/ideas from this library in a programming environment other than those currently supported by this ParaMonte library (i.e., other than C, C++, Fortran, MATLAB, Python, R), please also ask the end users to cite this original ParaMonte library.
This software is available to the public under a highly permissive license.
Help us justify its continued development and maintenance by acknowledging its benefit to society, distributing it, and contributing to it.
 Copyright
 Computational Data Science Lab
 Author:
 Amir Shahmoradi, Oct 16, 2009, 11:14 AM, Michigan
Definition at line 3942 of file pm_distNorm.F90.