ZFS gets inline dedupe
Found on The Register on Sunday, 01 November 2009
Sun's Zettabyte File System (ZFS) now has built-in deduplication, making it probably the most space-efficient file system there is.
The deduplication is done inline, with ZFS assuming it's running with a multi-threaded operating system and on a server with lots of processing power. A multi-core server, in other words.
The beauty of ZFS dedupe is that you don't need special storage arrays to deduplicate data. Ordinary arrays are quite acceptable, and its applicability at a data-set level means that you need only to deduplicate the datasets with redundant data and not the others.
ZFS is probably the best filesystem out there currently; even though there are of course some feature differences with others. Putting that aside, thanks to incompatible licenses (CDDL vs GPL), there won't be a native support in Linux anytime soon. Sometimes Open Source just blocks itself.