Powered by System Center
Windows 8 Storage & Hyper-V part 4 – Offload Data Transfer (ODX)
This is another installment in my series on Windows 8 Storage & Hyper-V. Previous blogs in the series can be found here:
Another promising new storage functionality that can be found in Windows Server 8 is the new transparent fast copy feature called Offload Data Transfer or ODX. If you know VMware’s vStorage API for Array Integration (VAAI), you probably know where to place ODX because it is more or less in the same league.
What’s the Challenge?
If you have a large Hyper-V guest with multi-Terabyte VHDX files, it depends on the amount of memory, the activity of the VM and the available bandwidth how long it takes to Live Migrate that VM to another node in your Hyper-V cluster. However, it is an entirely different story if you also need to move these very large VHDX files from one disk to another, from one array to another, from one cluster to another or even from one cloud to another. It would take ages doing this the classic way. Every read and every write including its confirmation would have to go through the sending server and the receiving server. Even if there would only be one Hyper-V server involved (copying between two CSV’s on the same server) this is highly inefficient. After all the VHD(X) is already on the storage array. Why let the data travel all the way from CSV1 through server A to server B and then back to CSV2 again? Why would the data have to leave the storage array at all?
With ODX we can avoid taking the long route and let the storage array controller(s) do the hard work. All you need is status information (how much has been copied; copy has been completed).
Even when you are moving a Windows 8 Hyper-V VM between hosts in different locations with a multiple arrays (replicating synchronously or asynchronously), there is no need to push the bulk of the data through the Hyper-V servers and the network.
ODX takes advantage of the more advanced features of modern storage arrays to speed up the movement of data. Instead of passing the data around, it passes around a token which contains a point in time view of the data. The good thing is that it supports fast copy of data within a machine or between machines on the same location or on multiple locations. It is not constrained by any protocol or transport type. Even better, any application can use this efficient way of data transfer.
ODX instructs the storage array to generate and return a token which shows the state of the transfer. The operation behaves like a non-cached read and is totally transparent to the operating system. In other words the storage array is in full control. The storage array determines if the write can take place, performs the data movement and confirms its success or failure. This operation is functionally identical to a normal non-cached write operation. The only requirement is that the disk space must be available and pre-allocated.
The reason why ODX is so transparent is because the offload transfer technique is integrated into the Win23 CopyFile API. This means that any copy, xcopy, robocopy or even drag & drop will benefit from ODX if the underlying storage array supports this functionality.
Are there any limitations?
ODX works fine on any NTFS partitioned disk. However files cannot be compressed or encrypted. Neither are spare files or BitLocker protected volumes supported.
ODX and file system filters
ODX was written with other system filters in mind and tries to avoid compatibility issues with other system filters that are not aware of ODX. When an ODX operation is issued, the IO Manager checks all of the filters on the file system stack to determine if the feature is supported or not and takes appropriate action. You can run fltmc instances at a command prompt to see all the filters for a given volume.
What does it require on the array side?
Microsoft is currently working with several array vendors to get ODX supported. Hopefully a firmware update of the array controllers will be enough to get things started.
If any vendors would like to comment, please do!
Source: Windows 8 File System Performance and Reliability Enhancements, SDC11 presentation, SNIA
|Print article||This entry was posted by Hans Vredevoort on April 11, 2012 at 12:00, and is filed under Hans Vredevoort, Hyper-v. Follow any responses to this post through RSS 2.0. You can leave a response or trackback from your own site.|
No comments yet.
about 4 weeks ago - 3 comments
A hotfix has been released today for Windows Server 2012 Hyper-V servers which are unable to access LUNs over a Synthetic Fibre Channel after a VM is live migrated to another host in the cluster. This problem can occur if the following conditions are met: You have two Windows Server 2012-based computers that have the…
about 1 month ago - 3 comments
If you are a regular reader of this blog you have noticed that a lot of content that is written is related to Windows Azure Pack (WAP for short), formerly known as Windows Azure Services for Windows Server. There are also numerous good other blogs as well as videos about WAP written by fellow MVPs…
about 4 months ago - 4 comments
Several months ago I had a very short encounter with a Dell Compellent storage array which had just been installed for a customer in their New Jersey office. In a short blog I showed that creation of a 250GB VHDX could be done in just a few seconds. This was the result of Compellent’s support…
about 4 months ago - 4 comments
Not very often do I remember a Windows Update KB article off the top of my head, but this time I have talked and written about KB2855336 so often that it was probably written into my short term read-cache.. If you have been careful and missed the first version of this update because you rather…
about 4 months ago - 11 comments
In my last blog I sent out a red alert on a killer Windows Update that had not been sufficiently tested. The net result was a full crash of a two-node System Center fabric management cluster. The fabric was still in the making and backups were only provisionally taken in the form of Virtual Machine…
about 5 months ago - 2 comments
[Update July 13, 2013 - I was able to deploy the newly issued KB2855336 to all of my physical and guest cluster nodes. There have been no issues so far. The same KB will also show up in most of your VMs as it is a collection of 21 updates touching all kinds of bugs…
about 5 months ago - 3 comments
Today I received word from Microsoft that I am re-awarded as a Microsoft Most Valuable Professional for Virtual Machine. When I received my first award for Cluster back in 2009, I could not begin to understand its consequences and how much such an award would mean in terms of recognition, access to knowledge, direct contact…
about 5 months ago - 7 comments
Update June 26, 2013: Cristian Edwards notified me that he has updated the script to now support using a cluster name. That will save you some typing if you cluster counts 64 nodes. See end of blog Update July 1, 2013: Great to see that Niklas Akerlund and Trond Hindenes made great extensions on the…
about 6 months ago - No comments
We already know that we wouldn’t have to wait four years to get significant new features in Windows Server & Hyper-V. Looking at the list of builds since the first version of Hyper-V, we can observe there were considerable intervals between the releases of Windows Server 2008 (R2) and Windows Server 2012. We’ve seen three…
about 6 months ago - 2 comments
Some time ago I blogged about a great TechNet Wiki listing available hotfixes and updates for Windows Server 2012 Hyper-V and Failover Clustering: As many of you know, keeping your Windows Server 2012 Hyper-V clusters up-to-date has become a whole lot easier with Cluster Aware Updating, which assists not only in a managed installation of…