Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have read a lot of information about planning RAM requirements forZFS deduplication. I've just upgraded my file server's RAM to support some very limited dedupe on ZFS zvols which I cannot use snapshots and clones on (as they're zvols formatted as a different filesystem) yet will contain much duplicated data.

I want to make sure that the new RAM I added will support the limited deduplication I intend to be doing. In planning, my numbers look good but I want to be sure.

How can I tell the current size of the ZFS dedupe tables (DDTs) on my live system? I read this mailing list thread but I'm unclear on how they're getting to those numbers. (I can post the output of zdb tank if necessary but I'm looking for a generic answer which can help others)

share|improve this question
add comment

1 Answer 1

up vote 9 down vote accepted

You can use the zpool status -D poolname command.

The output would look similar to:

root@san1:/volumes# zpool status -D vol1
  pool: vol1
 state: ONLINE
 scan: scrub repaired 0 in 4h38m with 0 errors on Sun Mar 24 13:16:12 2013

DDT entries 2459286, size 481 on disk, 392 in core

bucket              allocated                       referenced          
______   ______________________________   ______________________________
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
------   ------   -----   -----   -----   ------   -----   -----   -----
     1    2.23M   35.6G   19.0G   19.0G    2.23M   35.6G   19.0G   19.0G
     2     112K   1.75G   1005M   1005M     240K   3.75G   2.09G   2.09G
     4    8.03K    129M   73.8M   73.8M    35.4K    566M    324M    324M
     8      434   6.78M   3.16M   3.16M    4.61K   73.8M   35.4M   35.4M
    16      119   1.86M    811K    811K    2.33K   37.3M   15.3M   15.3M
    32       24    384K   34.5K   34.5K    1.13K   18.1M   1.51M   1.51M
    64       19    304K     19K     19K    1.63K   26.1M   1.63M   1.63M
   128        7    112K      7K      7K    1.26K   20.1M   1.26M   1.26M
   256        3     48K      3K      3K     1012   15.8M   1012K   1012K
   512        3     48K      3K      3K    2.01K   32.1M   2.01M   2.01M
    1K        2     32K      2K      2K    2.61K   41.7M   2.61M   2.61M
    2K        1     16K      1K      1K    2.31K   36.9M   2.31M   2.31M
 Total    2.35M   37.5G   20.1G   20.1G    2.51M   40.2G   21.5G   21.5G

The important fields are the Total allocated blocks and the Total referenced blocks. In the example above, I have a low deduplication ratio. 40.2G is stored on disk in 37.5G of space. Or 2.51 million blocks in 2.35 million block's worth of space.

To get the actual size of the table, see:

DDT entries 2459286, size 481 on disk, 392 in core

2459286*392=964040112 bytes Divide by 1024 and 1024 to get: 919.3MB in RAM.

share|improve this answer
1  
my apologies for being dense... in the example you provided, how can I determine how large the dedpue table is, specifically, how much RAM / L2ARC it consumes? –  Josh Aug 26 '13 at 16:42
1  
See updated post. –  ewwhite Aug 26 '13 at 16:52
1  
I especially like @ewwhite's example DDT, because it also makes for a stellar example of a ratio that precludes using dedupe at all. I'd zfs send/recv the datasets on this pool, from deduped datasets to non-deduped datasets, and count myself lucky they were still small enough to make that manageable. :) Be careful assuming your zvols will dedupe. As a block-level dedupe, a single offset difference could skew the whole thing. If I have any advice, it is move mountains to test the production dataset in a TEST lab /before/ putting ZFS dedupe into any production environment. –  Nex7 Sep 12 '13 at 19:03
    
constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe has some good information on calculating your expected wins from dedup and your expected costs. –  jlp Sep 15 '13 at 20:51
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.