Density transformation and parameter estimation from back propagation algorithm

We look at the neural network as a non-linear probability density function (pdf) transformer by stochastic learning cumulative (SLC) technique. We formulate a potential function that drives a neural network to non-linearly transform the input pdf to the desired pdf. We show the working of the algorithm using synthetic data drawn from three different pdfs and estimate the parameters of the distributions. The estimated parameter values match with the true values and the maximum likelihood estimates. We also derive bounds on the number of hidden neurons needed for parametric estimation in terms of the input data statistics.