Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [deduplication]

98 questions
2
votes
1 answer

ZFS interpret output of zdb -S tank

I wanted to know if it would pay it out for me to activate the zfs deduplication so I ran the command zdb -S tank but know I need some help to interpret the output. Simulated DDT histogram: bucket allocated …
White
  • 25
  • 3
2
votes
1 answer

ZFS deduplication

This is my first post on this website so please be kind with me :) On Centos 7, using ZFS deduplication in the past on a storage pool, I moved all file-systems that dedup was on to another storage pool. But, doing a "zpool status -D " I can see that…
Nicolas-V13
  • 21
  • 2
1
vote
0 answers

Is Path-based Unoptimization Possible?

I have a deduplicated volume in Windows Server 2016 that some software is having problems dealing with. Specifically, Matlab 2015 and older couldn't read deduplicated files, and while updating to a current version allowed for the files to be read,…
Louis Waweru
  • 755
  • 1
  • 9
  • 29
1
vote
0 answers

Deduplication Savings differs a lot between same two volumes

I run this robocopy command to copy all data from the existing volume to my new volume which will replace the old one in the near future: robocopy I:\ E:\ /E /COPYALL /PURGE /MIR /B /R:1 /W:1 /XD "Recycler" "Recycled" "$Recycle.bin" "System Volume…
donduc_jr
  • 73
  • 5
1
vote
1 answer

VHDX drive space being consumed

I have a host running Server 2012 R2, and a file server VM also running 2012 R2. The VM has 3 drives, OS.vhdx, file-1.vhdx and file2.vhdx. Dedupe is running inside the VM on file-1.vhdx and file2.vhdx. I have checked via file shares and no-one is…
morleyc
  • 1,150
  • 13
  • 47
  • 89
1
vote
0 answers

Deduplication assistance

We have consolidated a couple of drives and NAS systems of a former colleague before the disks failed (some actually showed signs of degradation). Now, we know that the colleague has done "RAID by hand" aka "copying stuff between the disks"... and…
Marco Schuster
  • 53
  • 7
1
vote
0 answers

How to calculate dedupe ratio per folder

Is there a way to determine the zfs dedup ratio on a per folder basis? I realize the potential difficulties in computing this as every block would need to be analyzed, but can such a method be scripted?
Dan Buhler
  • 486
  • 5
  • 9
1
vote
1 answer

Windows deduplication doesn't yield usable space

Dedup size picture Hi all, When I setup Windows depuplication, I get great savings. See the attached picture. 1.28 TB deduped down to 23 GB. However, the space savings are not real. Lets say that volume is 2TB. According to the screenshot, there…
GeoffM
  • 75
  • 8
1
vote
1 answer

Unable to mount borg backup repository residing on a ZFS snapshot

I've posted the question and my own solution to this problem, to make it available to others. This relates to borg backup and was posted to this borg github issue: Allow check on read-only filesystems Trying to access a remote borg repository for…
zoot
  • 257
  • 1
  • 3
  • 13
1
vote
0 answers

Clarifying sizing of dedup table on ZFS when only some datasets are deduped

I am trying to be clear about how zfs dedup handles the case where some (but not all) datasets in a pool are deduped, from a dedup table/RAM impact perspective. I found this quote from the FreeBSD mailing list in 2012: "Note that only file systems…
Stilez
  • 714
  • 7
  • 15
1
vote
0 answers

Better to deduplicate inside VHDX on client or outside VHDX on host?

I have a 16TB array that I'm hosting in a virtual environment. It contains video and photos from our marketing department. The data resides on the Hyper-V host itself as a VHDX file on a 8-drive SATA array. Our file server is virtualized and running…
Mike Capps
  • 41
  • 2
1
vote
1 answer

18.1 TB Drive Shows .75 TB Free but only 4.66 TB Deduped on Windows Server 2012 R2

I have a Dell PE R520 with 18.1 TB drive (built on RAID 6, disk internal to server). I setup Microsoft's Deduplication functionality on this drive and began moving large amounts of material onto it. Now it shows that it has .75 TB free out of the…
David Mackey
  • 697
  • 2
  • 15
  • 30
1
vote
0 answers

Error 0x80070519 reading file: revision level is unknown

I have mounted a dedup volume in Windows Server 2012 R2. When I try to copy a file from the volume to another I get the error 0x80070519 revision level is unknown The volume seems to be healthy otherwise: PS C:\Users\Administrator>…
Ronnie Overby
  • 681
  • 2
  • 12
  • 24
1
vote
0 answers

Stop Windows Backup from making Deduplication Exclusion for backup drive

With Windows Server Backup after every backup, the server puts an exclusion in the Deduplication list for the backup folder. Is there a way to stop this behavior? If not, is there a way to get Powershell to remove the exclusion? Edit: The Powershell…
Kayot
  • 261
  • 1
  • 2
  • 7
1
vote
2 answers

Is there a way to get the cmdlet for data deduplication when running an Unoptimization job?

I deactivated data deduplication on one of my drives because it doesn't work well with the backup system. I then started an unoptimization job in Administrative PowerShell. After that I realized I didn't put the priority up high so the job runs…
Gopoi
  • 547
  • 5
  • 21
1 2 3
4
5 6 7