Powershell, Windows knowhow

Windows: Using deduplication on a volume

Leave a comment

Hi,

since Server 2012R2 Windows has the ability to enable deduplication on volumes. This means Windows can find identically blocks on a volume and create references to them. So the block is just stored one time.

To use deduplication the Windows feature Data Deduplication (FS-Data-Deduplication) has to be enabled.

1
2
PS d:\> Import-Module ServerManager
PS d:\> Add-WindowsFeature -name FS-Data-Deduplication

The deduplication ratio can evaluated without enabling. Use the ddpeval command.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
C:\>ddpeval e:\FolderToDeDup
Data Deduplication Savings Evaluation Tool
Copyright (c) 2013 Microsoft Corporation.  All Rights Reserved.
 
Evaluated folder: e:\FolderToDeDup
Evaluated folder size: 1,59 TB
Files in evaluated folder: 1041406
 
Processed files: 409789
Processed files size: 1,59 TB
Optimized files size: 610,76 GB
Space savings: 1012,87 GB
Space savings percent: 62
 
Optimized files size (no compression): 634,29 GB
Space savings (no compression): 989,35 GB
Space savings percent (no compression): 60
 
Files excluded by policy: 631617
     Small files (<32KB): 631617
Files excluded by error: 0

The expected saving for this volume is 62%. It makes sense to active deduplication.

Load the necessary powershell modules for managing deduplication.

1
PS d:\> Import-Module Deduplication

Enable deduplication on Volume E:

1
PS d:\> Enable-DedupVolume -Volume E: -UsageType Default

To List all volumes where dedup is enabled use Get-DeDupVolume

1
PS d:\> Get-DedupVolume

Deduplication does not work on the fly during files are stored on the volume. Means the deduplication process on a volume must triggered periodically.
Start-DedupJob starts a deduplication one time.

1
PS d:\> Start-DedupJob –Volume E: –Type Optimization -InputOutputThrottleLevel Medium -Memory 70 -Priority Normal

To Schedule a deduplication use New-DedupSchedule

1
PS d:\> New-DedupSchedule -Days Saturday -DurationHours 0 -InputOutputThrottleLevel Medium -Memory 70 -Name "Dedup Scheduled optimization" -Start "18:00" -StopWhenSystemBusy -Type Optimization

To disable dedup on a volume. Remove the schedule and disable it. Ensure that the volume is large enough to hold the whole data :-).

1
PS d:\> if((Get-Volume E).Size -le (Get-DedupStatus -Volume E:).UnoptimizedSize){write-warning "Not enough Space for disable deduplication"}

Disable deduplication

1
2
3
PS d:\>
PS d:\> Remove-DedupSchedule -Name "Dedup Scheduled optimization"
PS d:\> Disable-DedupVolume -Volume E:

Michael

Leave a Reply