Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [deduplication]

98 questions
1
vote
1 answer

zfs zpool dedup stat seems very wrong - how to interpret these values?

I run a fileserver receiving backups of user containers. Two containers are poorly run docker systems with hundreds of near-identical directories not using overlayfs nor zfs clones. (I cannot touch the user's containers to remedy their use nor seem…
math
  • 230
  • 2
  • 9
1
vote
1 answer

rsync Delete only Matching files

How can I use rsync (or another program) to delete identical files between directories? To be identical, location and metadata should match (permissions, ownership, timestamp) For example, I backup dir X to Y. After time, there are files added and…
user206746
  • 146
  • 1
  • 6
1
vote
1 answer

Dedup.sys 0x7E after inplace upgrade Server 2016 => 19

I've recently performed an in-place upgrade from Server 2016 => 2019 (non-domain). 2 of the drives connected has been configured with data deduplication (mostly VHDs for Hyper-V). Ever since the upgrade, attempting to access, reverse dedupe…
Callumwill1991
  • 13
  • 2
1
vote
0 answers

btrfs compress existing data with snapshots

I'm runninga a btrfs raid1 on 5 8 TB disks, where primarily urbackup stores backups to. Urbackup uses snapshots extensively. compsize produces the following output: Processed 106161732 files, 57335496 regular extents (190449694 refs), 63766108…
Christian
  • 111
  • 2
0
votes
0 answers

postfix dovecot avoid duplicate sendings from two different IPs but same computer, same sender/recipients, same subject & body

Is there a way to avoid duplicate sendings from two different IPs but same computer, same sender, same recipients, same subject & body ? I'm using a postfix/dovecot server to which one of the clients connects using two concurrent internet…
Fabien Auréjac
  • 11
  • 3
0
votes
5 answers

Does (when?) OpenSolaris support ZFS deduplication and l2arc

Deduplication and L2ARC in ZFS would be nice to have. Does OpenSolaris support them? I can't quite figure out which version of Solaris they are in, and how that maps to OpenSolaris. Are they there? If not, do you know when they are scheduled.
Brian Carlton
  • 161
  • 2
  • 12
0
votes
1 answer

Size on disk smaller than capacity but no free space

We have a Windows 2012 Veeam Server with a volume that we use to store our backups on. The volume has a capacity of 14.9 TB and has data deduplication enabled. On the volume there are folders for the different backup jobs. These folders have a…
Skanderbeg
  • 15
  • 4
0
votes
0 answers

recurring, queryable, cached checksumming of all files in a filesystem

I'm looking for a way to efficiently manage and leverage file-level checksums for all files in a filesystem over time. Goals: Configurable, fast refresh - only re-checksumming large files when other criteria indicate a likely change (file size,…
Royce Williams
  • 1,362
  • 8
  • 16
0
votes
0 answers

Is there a way to measure or change how fast a Windows Server 2012 rehydrates data that it previously deduplicated?

Since the deduplication option was enabled on the Windows Server (currently at about 40 percent deduplication rate overall), the speed of the Backup Exec 16 job became much slower - less than 30 percent of the speed observed previously. Apparently,…
Kit
  • 101
  • 1
0
votes
1 answer

Does rsync -H convert file to hard link on destination after deduplication on source?

I've got two servers, the second one being synced via rsync every now and then. I've got some (few thousands ?) of big duplicate files on which I would like to use freedup to spare some space. If I launch freedup on source, would an rsync -H replace…
vincent.m
  • 85
  • 1
  • 1
  • 6
0
votes
2 answers

With backups, is deduplication a function of the server independent of the backup software?

In order to save space with backups, I wanted to know if I can use any deduplication or if I have use a special implementation. In other words, say I use backup software x. If I ship those backups off to a server that has deduplication, like…
johnny
  • 2,328
  • 9
  • 37
  • 57
0
votes
1 answer

HUGE ammount of duplicate files in this directory

I maintain our companies SBS2011 server, and I have duplicate file reports emailed to me on a monthly basis. I've noticed an increasing build up of duplicated files in the following directories: NT…
user3407675
  • 41
  • 1
  • 6
0
votes
0 answers

Data deduplication works vice versa

I have enabled data deduplication on one of my SSD's volumes and then run Start-DedupJob cmdlet with optimizastion parameter on this volume (Windows Server 2012). After dedup completed i check Get-Dedupstatus and get this output: FreeSpace …
ko4evneg
  • 234
  • 2
  • 10
0
votes
1 answer

Free SAN Storage Software for Linux

I'm looking for a free solution to build virtual disk targets accessible over SAN (iSCSI). Primary requirement for me is data deduplication support. After much search on the internet i came across quadstor (quadstor.com) and nexentastor…
joel billy
  • 11
  • 1
  • 1
0
votes
2 answers

Best filesystem for VM storage?

What is the best file system for storing VM images? I am looking for features that can save some disk space like compression, deduplication, etc. ZFS is known...but any other alternatives in Linux world? How good is the new EXT4 filesystem for…
JMS77
  • 1,275
  • 5
  • 27
  • 45
1 2 3 4 5
6
7