Home > Published Issues > 2016 > Volume 11, No. 1, January 2016 >

Distributed Metadata Search for the Cloud

Yang Yu, Yongqing Zhu, and Juniarto Samsudin
Data Storage Institute, A*STAR, Singapore 138632

Abstract—The ever increasing amounts of digital data being stored in public and private clouds are challenging users to access and manage the data. With the corresponding storage system reaches Petabyte-scale, or even Exabyte-scale, metadata access will become a severe performance bottleneck. Hence, this paper proposes an efficient multi-dimensional metadata index and search solution for cloud data. By proposing several new mechanisms for K-D-B tree based index/search, including two-level space borrowing and signature files for point pages, and implementing index partitioning technique, our system can achieve optimized performance in terms of memory utilization and search speed. Experiments show that our system performs much better as compared with other state-of-art solutions. In addition, our system can safely scale out in a distributed manner with guaranteed performance.

Index Terms—Metadata, index, search, multi-dimension, cloud

Cite: Yang Yu, Yongqing Zhu, and Juniarto Samsudin, “Distributed Metadata Search for the Cloud," Journal of Communications, vol. 11, no. 1, pp.100-107, 2016. Doi: 10.12720/jcm.11.1.100-107