Metadata Performance Optimization in Distributed File System

Nowadays distributed file systems have to manage billions of small files for applications, and face with more metadata operations than data. So how to provide high metadata performance with such tremendous number of files and such large scale directories is big challenge for distributed file system. We observe that metadata lookup operations dominate metadata workload and incur poor metadata performance. In this paper, we present and implement an aggressive and delegable metadata cache to accelerate metadata access while guarantee consistency for distributed file system. Through this optimization, the file system can gain as much as a 50% improvement in creation rates and 40 times improvement in stat rates.