Notes From The Field Using ODX With HP 3PAR Storage Arrays

Several months ago I had a very short encounter with a Dell Compellent storage array which had just been installed for a customer in their New Jersey office. In a short blog I showed that creation of a 250GB VHDX could be done in just a few seconds. This was the result of Compellent’s support for Microsoft Offloaded Data Transfer (ODX).

In the current project I have access to an HP 3PAR P10000 V400 with HP 3PAR OS v3.1.2 and just wanted to get some idea of 3PAR’s ODX capability in a customer environment with Windows Server 2012 servers. HP 3PAR StoreServ storage is ODX compatible beginning with the HP 3PAR OS v3.1.2 (December 2012). The servers we are testing with are HP ProLiant DL380 Gen8 (32 logical processors, 256GB RAM) and are configured as Windows Server 2012 Hyper-V servers and some as SMB3 Scale-Out File Servers directly connected with 3PAR for the backend storage. The servers I use for testing are using a 10Gb Ethernet ISCSI connection using redundant Juniper EX3300 switches. The SMB3 File Servers and Hyper-V Fabric Management cluster nodes are connected to redundant 10Gb Ethernet Arista switches, just like the Internet facing Hyper-V tenant cluster nodes. The tests are done on servers with direct 10Gb Ethernet ISCSI connections to the 3PAR storage array.

image


SOME ODX RELATED CONSTRAINTS

To use ODX with 3PAR, your environment must support the following:

  • The computer initiating the data transfer must be running Windows 8 or Windows Server 2012.
  • File system filter drivers such as antivirus and encryption programs need to opt-in to ODX. ODX is not supported by the following file system filter drivers:
    o Data Deduplication
    o BitLocker Drive Encryption
  • Files must be on an unencrypted basic partition.
  • Storage Spaces and dynamic volumes are not supported.
  • Files must be on a volume formatted using NTFS. ReFS and FAT are not supported.
  • Files can be directly transferred to or from this volume, or from one of the following containers:
    o A Virtual Hard Disk (VHD) that uses the VHD or VHDX formats
    o A file share that uses the SMB protocol
  • The files must be 256 KB or larger – smaller files are transferred using a traditional (non-ODX) file transfer.
  • The application that performs the data transfer must be written to support ODX.
    The following currently support ODX:
    o Hyper-V management operations that transfer large amounts of data at a time, such as creating a fixed size virtual hard disk (VHD), merging snapshot or converting virtual hard disks.
    o File Explorer
    o Copy commands in Windows PowerShell
    o Copy commands in Windows command prompt (including Robocopy)
  • Files should not be highly fragmented. Transfers of highly fragmented files will have reduced performance.

For more details about HP 3PAR and ODX performance see:
http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA4-5439ENW.pdf.

According to this technical whitepaper at least the following Windows Server 2012 features benefit from ODX:

  1. Virtual Hard Disk Format (VHD and VHDX)
  2. Virtual Machine Live Storage Migration feature
  3. Virtual Machine Export feature
TEST RESULTS

I simply tested the creation of three VHDX sizes: 50GB, 100GB and 1000GB both without ODX and with ODX. These tests were run 3 times without any load on the server tested which is of course only relevant for the tests with ODX disabled. Your mileage may vary based on the number of disks, RAID type and connectivity between the server and the storage array.

Basically what you see when you have ODX enabled is a flat CPU and a flat network while creating/copying hundreds of Gigabytes.

image

Creation of 50GB fixed sized VHDX file on CSV with ODX disabled

image

While creating a fixed sized 50GB file, a running VM on the same server using the same CSV became very unresponsive.

Creation of 50GB fixed sized VHDX file on CSV with ODX enabled

image

The VM on the same server was responsive and seemed not impacted by the VHDX creation process offloaded to the 3PAR storage array.

Creation of 100GB fixed sized VHDX file on CSV with ODX disabled

image

Creation of 100GB fixed sized VHDX file on CSV with ODX enabled

image

Creation of 500GB fixed sized VHDX file on CSV with ODX disabled

image

Creation of 500GB fixed sized VHDX file on CSV with ODX enabled

image

Creation of 1TB fixed sized VHDX file on CSV with ODX disabled

image

Creation of 1tb fixed sized VHDX file on CSV with ODX enabled

image

TEST SUMMARY

Summarizing all tests here are the results for all eight tests:

GB

ODX disabled

Seconds

ODX enabled

Seconds

Improvement

50

6 minutes
37 seconds

397

2 minutes
36 seconds

156

254%

100

14 minutes
4 seconds

845

3 minutes
6 seconds

186

454%

500

1 hour
19 minutes
20 seconds

4760

15 minutes
56 seconds

957

504%

1000

2 hours
33 minutes
33 seconds

9513

33 minutes
18 seconds

1998

476%

image

As you can see there is a significant difference between creating fixed sized VHDX files by using ODX or not, but compared to what I see with Dell Compellent. 3PAR still seems to be configured to zero out the created blocks. We have to check back to see if the configuration can be changed to make this any faster.

My fellow Hyper-V MVP Didier van Hoye aka @WorkingHardInIT has recently run a number of tests on a Dell Compellent with an ODX capable firmware:
http://workinghardinit.wordpress.com/2013/07/24/some-odx-fun-with-windows-server-2012-r2-and-a-dell-compellent-san/

He ran this script to create ten 50GB and ten 475GB fixed sized VHDX files which took 42 seconds:

$Folder = "C:ClusterStorageVolume1"
$TotalTime = 0

for ($DiskNumber = 1; $DiskNumber -lt 11; $DiskNumber++ )
{
$TimeToCreateDisk = measure-command { New-VHD -Path "$FolderOSDisk$DiskNumber.vhdx" 50GB -Fixed }
$TotalTime = $TotalTime + $TimeToCreateDisk.TotalSeconds
$TimeToCreateDisk = measure-command { New-VHD -Path "$FolderOSDisk$DiskNumber.vhdx" 475GB -Fixed }
$TotalTime = $TotalTime + $TimeToCreateDisk.TotalSeconds
}
Write-Host $TotalTime

image

In a live Skype session Didier showed me how he created 5 x 1TB fixed sized VHDX files files on a Dell Compellent

image

The next test on the Compellent creates a 100GB VHDX file WITHOUT using ODX which took only 138 seconds.

clip_image002

To make it even worse he also showed how fast he could create 100 x 100GB fixed sized VHDX files WITH using ODX: 107 seconds.

image

I don’t even want to try this test on the 3PAR now ….

image19

4 Comments

  1. July 26, 2013    

    Hi Hans,

    SCVMM R2 is supposed to be able to deploy VM using ODX did you had a chance to test that?

    • adminHans's Gravatar adminHans
      July 26, 2013    

      Hi Emmanuel,
      No I did not have a chance to test this with R2, but you are correct that VMM 2012 R2 supports ODX for the VMM Library.
      -Hans

  2. July 29, 2013    

    Hi Hans,

    We are building a MS cloud solution on BL460C G8 with HP 3PAR 7400 and have sadly run into many issues.

    One of the issues was ODX wouldn’t work at all. We found the root cause to be a time difference on the 3par nodes. When time was synced correctly ODX started to work.

    Maybe you can add this to your list of prereqs :)

    Another thing you could add is that ODX is not available to a VM when the VHD(X) file of a VM is attached using IDE (for 2012 this means all OS disks). This will be resolved by the Generation 2 VMs in 2012 R2 which support OS disks as SCSI disks.

    Regards, Ben

  3. Jerry Huizinga's Gravatar Jerry Huizinga
    March 16, 2014    

    Hi Hans,

    We tested ODX with Windows 2012 R2 server connected to a Compellent system with FW 6.3.1 and indeed the performance creating fixed disks is excellent.
    Although regular coping or moving data via the explorer between CSV volumes have ODX a negative impact. Copy and move operations is 20-50 % slower with ODX enabled.

    Do you have any idea what’s the cause ?

    Thanks in advance,

    Jerry.

  1. Week of July 26: Start testing Windows Server 2012 R2 with the Windows Server MVPs - Server and Cloud Partner and Customer Solutions Team Blog - Site Home - TechNet Blogs on August 1, 2013 at 20:43

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>