A Reliable DHT-Based Metadata Server Cluster

Abstract

This paper proposes a Distributed Hash Table-Based Metadata Server Cluster (DHT-MDSC), which provides an efficient routing strategy and an automatic reconfiguration protocol to eliminate the bottleneck of a centralized hash table. The design is based on a new concept merging p2p system with conventional metadata file system into a scalable and high performance distributed metadata server. A novel caching mechanism (LC-RIC) is implemented to improve the performance and scalability of the system. In addition, we have also implemented a reliable DHT-MDSC (RDHT-MDSC) which can tolerate the failure of multiple metadata servers. The system can reconstruct itself upon detecting any fault under normal operation. The time it takes to reconstruct the system is also short in our design.