The deduplication tag has no wiki summary.
1
vote
1answer
78 views
Backing up multiple git repos of the same project (deduplication)
I use a central git server (a gitlab instance). Developers clone projects to a samba share on another server. At the moment, I make backups of the gitlab server and all of the developers directorys on ...
2
votes
2answers
153 views
Deduplication of tar archives
Now I'm trying to configure ZFS with deduplication on my linux backup server and have a question.
For example I have 1k files in backup1.tar and 1k same files and 1 new in backup2.tar. This files ...
2
votes
1answer
106 views
Accessing windows server deduplicated files from other operating systems
If you enable data deduplication on the Windows Server 2012, it will change certain files (according to the policy) and introduce reparse points.
If the drive containing deduplicated volumes is ...
2
votes
1answer
187 views
Mixing Volume Shadow Copy and Data Deduplication in Windows Server
I'm using Windows Server 2012 and I'm creating a scheduled backup for multiple Hyper-V guests (all of them are Windows Server 2012) to a dynamic VHDX on a removable USB drive (so that I can alternate ...
1
vote
2answers
389 views
Server 2012 Data Deduplication Skipping Replica VHDs
I'm currently trying to use data deduplication on two seperate Windows Server 2012 Datacenter edition hyper-v hosts. On one, I am trying to dedupe replicas that are still being resync'd every 5 ...
12
votes
1answer
286 views
Technical details for Server 2012 de-duplication feature
Now that Windows Server 2012 comes with de-duplication features for NTFS volumes I am having a hard time finding technical details about it. I can deduce from the TechNet documentation that the ...
0
votes
2answers
262 views
De-duplicating backup tool on a block basis? [closed]
I am looking for an (ideally free as in speech or beer) backup tool for Unix-like OS which can store deduplicated backups, i.e. only nonredundant content takes up additional space.
I already looked ...
1
vote
1answer
77 views
ZFS dedup block usage?
How can I list all the ZFS dedup blocks and the files that are associated with each one?
e.g.:
BLOCK FILES
e6b678a /pool/a/file1
/pool/a/file2
If that's not possible, what about ...
1
vote
0answers
103 views
HP D2D 4312 Bacula configuration
I have configured 5 libraries on the HP D2D system
Discovery on the Bacula server shows only the last library and not all libraries. Why?
[root@server bacula]# iscsiadm --mode discovery --type ...
1
vote
1answer
129 views
ZFS for offline backups
I'm thinking of using ZFS to backups files to single drives that are going to be kept mostly offline. Does anyone know if the FreeBSD ZFS implementation still uses memory if a dedupped pool is ...
1
vote
1answer
633 views
How to activate deduplication in Windows Server 2012?
I just installed Windows Server 2012, and while I see and read about a new deduplication feature, the Server Manager keeps reporting empty fields in the columns for deduplication rate and savings. How ...
1
vote
0answers
726 views
Looking for a deduplication program/script [closed]
I have two systems which hold backups and I would like to de-duplicate.
Both are existing systems so new filesystems such as zfs or lessfs aren't really an option.
There seems to be a HUGE list of ...
8
votes
3answers
933 views
Backup storage server with ZFS
I am IT everything man at a small company. I want to design a new infrastructure including a new server and a separate backup server with company wide backup policy.
The most important thing in the ...
2
votes
2answers
279 views
File System with Real-Time Data Deduplication
Is there a file system that stores files under a hash so there are no duplicates? It can be under any operating system. I know Git does that, but I'm looking for something that can run in real-time.
-1
votes
1answer
1k views
Free SAN Storage Software for Linux [closed]
I'm looking for a free solution to build virtual disk targets accessible over SAN (iSCSI). Primary requirement for me is data deduplication support. After much search on the internet i came across ...
1
vote
1answer
85 views
ZFS: can per-user encrypted filesystems be deduplicated on block-level?
Let's assume I build a storage system where each user gets his/her own encrypted filesystem on top of a ZFS pool. Would deduplication on pool-level still work when each fs is encrypted?
What ...
2
votes
3answers
990 views
Fastest way to copy large file directory over slow internet
I need to copy a directory from one of our servers several hundred miles away. Both sites where I am and where the files are have business cable. The cable is 5 Mbps(Down) / 768 Kbps(Up). This poses a ...
1
vote
2answers
1k views
How does ZFS Block Level Deduplication fit with Variable Block Size?
According to The First Google Result for "ZFS Deduplication"
...
What to dedup: Files, blocks, or bytes?
...
Block-level dedup has somewhat higher overhead than file-level dedup when whole ...
1
vote
2answers
548 views
Should I use USB drive for Solaris 11 Express
Debating on if I should use a 16gb USB drive for my Solaris 11 Express SAN server. I intend to use Deduplication which I believe may be IO intensive for the DB. Though I can't be sure.
The SAN ...
5
votes
3answers
1k views
Proper use of disk to disk to tape backup using de-duplication and LTO5
I currently have ~12TB of data for a full disk to tape (LTO3) backup. Needless to say, it's now requiring over 16 tapes so I'm looking at other solutions. Here is what I've come up with. I'd like ...
1
vote
1answer
442 views
Extracting and saving attachments from maildir, deduplicated
I would like to know if there exists a solution for archiving email, stripping out the attachments and saving them separately, deduplicated?
The ideal would be a maildir-setup, with a script running ...
0
votes
2answers
441 views
Best filesystem for VM storage? [closed]
What is the best file system for storing VM images?
I am looking for features that can save some disk space like compression, deduplication, etc. ZFS is known...but any other alternatives in Linux ...
5
votes
5answers
318 views
Full-featured online backup providers for medium-sized enterprise? [closed]
I've been banging my head against the wall trying to find an online backup service that supports all of the following enterprisey features:
Full-system backup for Linux and Windows 2003/2008 ...
0
votes
2answers
183 views
Differential backup of dedup volume
We are currently doing a complete remote backup of our VM's once a week. This takes the whole weekend, and I'd like to speed up the process some if possible.
We are backing up the VM's using Dedup, ...
5
votes
2answers
1k views
ZFS - destroying deduplicated zvol or data set stalls the server. How to recover?
I'm using Nexentastor on a secondary storage server running on an HP ProLiant DL180 G6 with 12 Midline (7200 RPM) SAS drives. The system has an E5620 CPU and 8GB RAM. There is no ZIL or L2ARC device.
...
1
vote
4answers
323 views
IMAP server that saves space on identical attachments
Looking for IMAP server that can detect duplicate attachments across mailboxes and would not waste space on them.
So, if 4 local mailboxes receive the same attachment I would like it to be stored ...
1
vote
2answers
4k views
Deduplication option in Backup Exec 2010 - does it work like this?
We have a 6TB backup-to-disk system running with Backup Exec 2010. Each week, a full backup is carried out with differentials on the other days. We manage to get about four weeks of backups in there.
...
1
vote
1answer
640 views
VMware ESXi 4 On-Disk Data Deduplication - possible and supported?
Environment:
We are running multiple web, database, and application servers which usually share a pretty common installation (gentoo linux) and similar configuration in VMware ESXi 4. The differences ...
7
votes
2answers
299 views
What is “deduplication”?
I mean, I can look up the dictionary definition, but why is everyone suddenly talking about it in reference to virtual tape libraries? What's "new" here so that it's so much in the news lately?
0
votes
5answers
835 views
Does (when?) OpenSolaris support ZFS deduplication and l2arc
Deduplication and L2ARC in ZFS would be nice to have. Does OpenSolaris support them? I can't quite figure out which version of Solaris they are in, and how that maps to OpenSolaris. Are they there? ...
4
votes
18answers
4k views
How to Eliminate Tape Backup and Off-site Storage Service?
PLEASE READ UPDATE AT THE BOTTOM. THANKS! ;)
Environment Info (all Windows):
2 sites
30 servers site #1 (3TB of backup data)
5 servers site #2 (1TB of backup data)
MPLS backbone tunnel connecting ...
9
votes
9answers
9k views
Block-level deduplication on Linux
NetApp provides block-level deduplication (ASIS). Do you know any filesystem (even FUSE-based) on Linux (or OpenSolaris, *BSD) that provides the same functionnality ?
(I'm not interested in false ...
4
votes
6answers
529 views
Capacity Optimization / Deduplication Options for Primary Storage
I'm exploring options for making more efficient use of our primary storage.
Our current NAS is an HP ProLiant DL380 G5 with an HP Storageworks MSA20, and one other disk shelf which I'm not sure what ...
