A MONTE CARLO SIMULATION OF THE IMPACT OF SAMPLE SIZE AND PERCENTILE METHOD IMPLEMENTATION ON IMAGERY GEOLOCATION ACCURACY ASSESSMENTS

Many methods exist to estimate image absolute geolocation accuracy statistics from check-point analyses. Among them is the Percentile Method (PM), which orders the differences between measurements and truth, and estimates the geolocation accuracy statistic at a specific percentile, such as the 90th percentile used for the Circular Error 90% (CE90) statistic. The Percentile Method is elegant because it uses the measured values directly and does not make any assumptions about the population distribution. However, several implementations exist to estimate the value at the desired percentile among the ordered data points. In addition, the number of images available for evaluations is limited, and the precision of a parameter estimate from any method decreases with smaller sample sizes. A Monte Carlo simulation was run using all sample sizes between ten and thirty, inclusive, and using eleven Percentile Method implementations to estimate accuracy values for every decile from 10% to 90%. Sampling distributions were formed for each case using many trials. The difference between the mean of each sampling distribution and the known population mean represents the bias for each estimation method and sample size. The standard deviation of each sampling distribution provides a measure of the variability of the estimation method for each sample size. The results quantify the improvement in precision as the sample size increases. The results also show that some Percentile Method implementations can be heavily biased and that these biases vary with sample size and percentile value.