A NEW METHOD FOR COMPARING EXPERIMENTS AND MEASURING INFORMATION

A statistical experiment can consist of taking a sample. Often, statistics are formed that are based on the sample data. Two different experiments can yield two different statistics. Under certain conditions, we can compare these statistics by comparing the experiments. A statistic that summarizes an entire data set without losing any information about the family of distributions or the model is a sufficient statistic. Sometimes it may be desirable to use a statistic that, though not sufficient, does summarize the data set somewhat. How much information will we lose? How can we compare two statistics that are not sufficient in terms of the amount of information they provide? A new method for comparing experiments and measuring information is introduced. The new method is used to evaluate the expected efficiency of a statistic in discriminating between any two values of the parameter as compared to a sufficient statistic. This new method can be self-calibrated to give this expected efficiency a meaningful scale. This new method is applied to Casino Blackjack. Several card-counting statistics are compared by the amount of information each provides in discriminating between different deck compositions as compared to a sufficient statistic. This new method provides new insight about information in card-counting statistics by putting this information on a meaningful scale.