PROJECT TITLE :

Application-Aware Big Data Deduplication in Cloud Environment - 2017

ABSTRACT:

Deduplication has become a widely deployed technology in cloud information centers to enhance IT resources potency. But, traditional techniques face a nice challenge in massive knowledge deduplication to strike a smart tradeoff between the conflicting goals of scalable deduplication throughput and high duplicate elimination ratio. We tend to propose AppDedupe, an application-aware scalable inline distributed deduplication framework in cloud atmosphere, to satisfy this challenge by exploiting application awareness, knowledge similarity and locality to optimize distributed deduplication with inter-node two-tiered knowledge routing and intra-node application-aware deduplication. It initial dispenses application knowledge at file level with an application-aware routing to keep application locality, then assigns similar application data to the identical storage node at the super-chunk granularity using a handprinting-primarily based stateful knowledge routing theme to maintain high global deduplication efficiency, meanwhile balances the workload across nodes. AppDedupe builds application-aware similarity indices with super-chunk handprints to speedup the intra-node deduplication process with high efficiency. Our experimental evaluation of AppDedupe against state-of-the-art, driven by real-world datasets, demonstrates that AppDedupe achieves the highest global deduplication efficiency with a higher international deduplication effectiveness than the high-overhead and poorly scalable ancient scheme, however at an overhead only slightly higher than that of the scalable but low duplicate-elimination-ratio approaches.