Role of Nonperiodic Orbits in the Semiclassical Quantization of the Truncated Hyperbola Billiard

Based on an accurate computation of the first 1851 quantal energy levels of the truncated hyperbola billiard, we have found an anomalous long-range modulation in the integrated level density. It is shown that the observed anomaly can be explained by an additional term in Gutzwiller's trace formula. This term is given as a sum over families of closed, nonperiodic orbits which are reflected in a point of the billiard boundary where the boundary is continuously differentiable, but its curvature radius changes discontinuously.