Optimum pulse amplitude modulation--II: Inclusion of timing jitter

A method is presented for specifying transmitter waveforms and receiver filters for pulse amplitude modulation (PAM) systems which are subject to distortion from additive noise, intersymbol interference, and timing jitter. A criterion of minimum mean-squared error is used. For the case of band-limited transmission, specific procedures are developed for computing optimum transmitters, optimum receivers, and jointly optimum transmitters and receivers. For two special jitter distributions the joint optimization problem is solved without a bandwidth limitation, and the resulting systems are found to be band-limited. In cases of localized timing jitter and high signal-to-noise ratio, we are led to system pulse responses of the form[\sin x/x]^{2}. A nonparametric method of reducing the effects of timing jitter is briefly discussed. In this approach the noise interference is minimized subject to the constraints that the intersymbol interference be zero and that the system pulse response have zero derivative at each nominal sampling point.