Powered by System Center
While cleaning up my mailbox, I found a note to myself dating June 2011 about a few feature requests for the next version of Hyper-V and VMM. While talking to customers throughout the year, I often hear about and collect suggestions and product improvements which I can then pass on to the Hyper-V and System Center Product team. In retrospect most items on my list have become a reality in Windows Server 2012 and much, much more!
Online VHD growth and shrink has not yet been realized (although a 64TB dynamic disk supporting unmap pretty much covers all you can desire).
Perhaps the last item on the list is only partly covered by Hyper-V Replica but needs some additional automation or services to make this a complete Recovery solution that could make it even better than RSM. As we know Microsoft is on the right path and a flood of companies are getting really interested in Windows Server 2012 Hyper-V and System Center 2012 SP1.
Thanks Hyper-V and System Center product teams for listening.
During MVP Summit we spent several great days with Hyper-V Product Manager Ben Armstrong aka @VirtualPCGuy. German fellow Virtual Machine MVP Carsten Rachfahl took the opportunity to interview his ‘rockstar’ at CenturyLink Stadium, home of the SeaHawks football team. Very soon you will notice Ben comes from the other side of this planet, at least when you look at it from our angle: ‘Down Under’.
Ben has delivered many excellent presentations on the Hyper-V hypervisor and related subjects at events like TechEd North America, Europe, Australia and New Zealand:
He has been working on virtualization software for over 10 years now. The last 9 years have been spent working on virtualization as part of the Windows Core Virtualization team. Ben has been maintaining a blog for the last 8 years as Virtual PC Guy. Chances are that if you have a question about virtualization, Ben will know the answer.
You can find the German Hyper-V blog authored by Carsten Rachfahl here:
[UPDATE 1/15/2013]: See problems reported after installing this hotfix at end of this forum post. I have opened a line with the product team to find out what is going on
[UPDATE 2/15/2013]: We are now exactly 1 month after I warned you to be very careful with installing http://support.microsoft.com/kb/2799728/en-us as it caused a severe memory leak. If you have followed the forum post mentioned above, you will have gathered that Microsoft has worked feverishly to get on top of the problem. Mike Jacquet who has been very communicative about this issue, has today confirmed that a fix is now code complete, has been fully tested and is only waiting for the KB article to be written.
The memory leak was caused by a fault found in the CSV filter driver (CSVFLT.sys). When the fix arrives (any time now), you can simple install it whether you have applied the hotfix mentioned in this blog or not. The original kb article will be superseded by this one.
[UPDATE 2/17/2013]: The hotfix is available from http://support.microsoft.com/kb/2813630/en-us
[UPDATE 2/18/2013]: If you are still seeing a problem, take a look at this (which is an excerpt from the forum post mentioned above:
“Cluster Shared Volume ‘Volume2′ (‘ClusterStorage Volume 2′) is no longer available on this node because of ‘STATUS_CLUSTER_CSV_AUTO_PAUSE_ERROR(c0130021)’. All I/O will temporarily be queued until a path to the volume is reestablished.
STATUS_CLUSTER_CSV_AUTO_PAUSE_ERROR is generated when csvfs filter attempts to retrieve the Copy On Write bitmap for a snapshot volume that has been cleaned up. This error is most likely occurring on large scale hyper-v deployments and is one of the issues we discover after fixing other scale out problems addressed in the V2 fix. Due to ongoing long haul testing required to be done, we did not want to hold up V2 of the fix that we just released, so the Windows group will release a more compressive V3 patch a little later to address that and other issues found during large scale testing.
For any customers still experiencing the same symptoms as outlined in KB2813630 after installing the fix, please check binary versions on all nodes.
File name File version File size Date
====== ========= ====== ====
Csvflt.sys 6.2.9200.20626 205,824 06-Feb-2013
Clussvc.exe 6.2.9200.20623 7,217,152 07-Feb-2013
Ntfs.sys 6.2.9200.20623 1,933,544 07-Feb-2013
If Binaries are correct on all nodes, please open a support case so we can investigate the issue further.”
I admit, you don’t have to remove a non-existent VMM Library Server everyday, but today happened to be such a day. Let me explain what happened. In addition to a SQL Server 2012 AllwaysOn cluster for the VMM database and a VMM 2012 SP1 Failover Cluster, I wanted to also make the VMM Library Server highly available because the management foundation for a production Windows Server 2012 Hyper-V hosted cloud would soon become too big too fail.
Let me concentrate on the VMM Library cluster. I created an ordinary guest based 2-node Windows Server 2012 cluster using the new synthetic virtual Fibre Channel adapters, connected to an EMC VNX5300 storage system. After documenting the WWN’s and WWPNs for both adapters and requesting the SAN admin to create a few disks and to correctly zone the FC devices, creating the failover cluster was really a very quick deal. For some reason after installing EMC PowerPath 5.5 SP1, only one node of the cluster detected the correct Multi-Path Disk Device while the other thought it was dealing with a VRAID SCSI Disk Device.
Because uninstalling the devices or removing and reinstalling EMC PowerPath didn’t do the trick, I just went on to create a standard clustered file server role and add a share to be used for the VMM Library.
Today we see the birth of the official build of Windows Azure Services on Windows Server. It will allow you to use Windows Server 2012, System Center 2012 SP1 and Antares (Websites service) in your own environment, just like you might already have experienced with Windows Azure.
As we speak I am installing Windows Azure Services in a new Windows Server 2012 Hyper-V environment with System Center 2012 at one of the large hosters in the Netherlands offering IaaS services based on Hyper-V.
My fellow blogger Marc van Eijk already wrote a fantastic blog on his experiences while the product was still called KATAL, the beta name for Windows Azure Services.
Please take a look at his blog which was published:
The first two services that are offered in this release are high scale, multi-tenant web site hosting and high density virtual machine provisioning and management. These services are lit up in the Service Management Portal and API. We will continue to bring more Windows Azure services on-premise in subsequent releases. You can find more information and download the bits at http://www.microsoft.com/hosting/en/us/services.aspx.
I ran into a problem with a misbehaving Failover Cluster Manager (FCM) on a Windows Server 2012 cluster after allowing Cluster Aware Updating (CAU) to update the cluster. The following updates were gracefully installed, automatically placing the nodes in maintenance mode, installing the updates and moving on to the next node:
After the cluster had been updated it was no longer possible to view the roles (Hyper-V VM’s in this case) on the cluster. Instead the following screen popped up:
Because another unpatched cluster did not have this symptom, I decided to uninstall all updates that had been installed by CAU and after a reboot all is well.
After I reported this problem, I got a quick note from the Microsoft cluster team that KB2750149 was the cause of this problem and that the current advice is to uninstall this update until further notice.
It is important to note that this is only a problem of the Failover Cluster Manager (CluAdmin.msc) snap-in. The cluster and all its roles continue to run fine, which can be verified by opening a PowerShell screen on one of the affected cluster nodes and run Get-ClusterGroup. I was also able to run FCM successfully from another cluster or a management station with the Failover Clustering management tools.
In the quiet days between Christmas and New Year, I had some time to research how DPM2012 SP1 performed with protecting guests on a Windows Server 2012 Hyper-V cluster using CSV v2.0.
According to the SP1 release notes we can expect improved backup performance of Windows Server 2012 Hyper-V over CSV deployments with the following benefits:
900% improvement in Express Full backups
No performance difference between backups from CSV owners and non-owners
Let me first point out that my setup is based on the following configuration:
HP ProLiant BL460c Gen8
Windows Server 2012 Datacenter with iSCSI Target Server Role enabled
Dual-port 10Gb HP FlexFabric 554FLB (Emulex) Adapter
Converged Fabric network configuration
Hyper-V Cluster Nodes
HP ProLiant BL460c Gen8
Windows Server 2012 Datacenter with Hyper-V role and Failover Cluster feature enabled
10Gb HP FlexFabric 554FLB (Emulex) Adapter
Converged Fabric network configuration
DPM 2012 SP1 Server
Windows Server 2012 Hyper-V Virtual Machine
Windows Server 2012 Datacenter
1 x 10Gb virtual network adapter (synthetic)
Both on the iSCSI Target Server and Hyper-V cluster nodes, the two 10Gb network adapters have been teamed using a switch independent teaming mode with Hyper-V Port as the load balancing algorithm. A Hyper-V Extensible Switch is connected to the NIC Team and several virtual networks have been configured using the Converged Fabric method of Windows Server 2012. Each network has a minimum bandwidth Quality of Service configured on the virtual switch level. On the backend, the servers use a Virtual Connect Flex-10 interconnect.
It is only several days since Microsoft made the final bits of System Center 2012 SP1 available, although for a relatively limited audience. If you have a TechNet or MSDN subscription, you have access to SP1 now. All others have to wait till general availability in the course of January 2013.
One of my fellow MVP’s in the Cloud and Datacenter Management department, Graham Davies, raised my interest by claiming there was something in VMM 2012 SP1 that had gone unnoticed: a brand-new SMI-S storage provider for the onboard iSCSI Target Server role in Windows Server 2012! People using the iSCSI Target Server for both demo/test/development and production will get quite excited if they discover how well this software storage solution is integrated, not only in Windows but now also in System Center 2012 SP1.
Back in March 2011, I wrote several blogs about the new Fabric concept in Virtual Machine Manager 2012 which was still in its early beta. One blog focused on deep storage integration using a SNIA standard called Storage Management Initiative Specification (SMI-S). When I was contributing my Fabric chapters for the Microsoft Private Cloud Computing book, there was only a limited number of usually very expensive storage arrays to really dig into this subject. I had access to some HP and NetApp storage to test SMI-S integration which was still very limited at the time. When we saw how the iSCSI Target Server, which was previously a separate install on Windows Server 2008 R2, developed and became included as a role in Windows Server 2012, we begged the product managers responsible for storage in System Center 2012 to also provide SMI-S support. Well guys … here it is!
Fast forwarding about two years, a lot has happened in the storage space and most storage vendors have introduced SMI-S support for their most important storage products. I now consider storage that does not support SMI-S (and some of the other cool technology such as ODX and Unmap/Trim), as a sign that these products are on that vendor’s death list and will soon be obsolete.
Especially for those who want to manage their Windows Server 2012 servers and clusters, an important moment has come. System Center 2012 SP1 is not only RTM but also ready for download on Technet. Several of our projects were delayed because we needed the SP1 version of Virtual Machine Manager 2012, Data Protection Manager 2012 and several of the other components to manage Windows Server 2012 Hyper-V and Hyper-V Server 2012 cluster. The long wait is over!
Because the new Windows operating system introduced such a shipload of new features, you can imagine what this meant for developing SP1.
It is good to know that some of the ISOs provided contain the full product including SP1 which makes it easier if you want to do a completely fresh install.
Release notes and documentation SCVMM 2012 SP1: http://technet.microsoft.com/library/gg702209.aspx
Installation Guide: http://technet.microsoft.com/library/gg610669.aspx
Release notes and documentation SCOM 2012 SP1: http://technet.microsoft.com/library/hh561709.aspx
Installation Guide: http://technet.microsoft.com/library/hh298609.aspx
[Updated: see ]
HP bought 3PAR several years ago and as I expected the recently announced HP 3PAR StoreServ 7000 series are intended to replace HP EVA. There is even a data migration solution to make the migration as easy as possible. Of course I looked at what 3PAR has to offer in terms of features in Windows Server 2012 and integration with Hyper-V and System Center Virtual Machine Manager 2012.
The 3PAR StoreServ 7000 datasheet tells us this:
In Windows® Server 2012 environments, built-in, fine-grained virtualization, system-wide striping, and support for multi-tenancy give you the ability to consolidate mixed workloads onto a single HP 3PAR StoreServ 7000 system. With Windows Server 2012 Offload Data Transfer (ODX) and the HP 3PAR StoreServ 7000, you can migrate large files—such as databases or video files—up to seven times faster with near-zero network impact due to zerodetection capability integrated into the HP 3PAR ASIC. HP 3PAR Thin Technologies and Windows Server 2012 automatic reclamation of storage automate storage growth and shrinkage, while HP 3PAR Adaptive Optimization Software delivers the right QoS to the right data at the right time in Windows Server 2012 environments.
Not only does the new 3PAR StoreServ 7000 support Offloaded Data Transfer (ODX) but also Storage Management Initiative Specification (SMI-S) and a new Web Services API. SMI-S is what Microsoft has promoted vigorously and nobody seemed to believe in. At the end of the day most storage vendors finally comply with this SNIA standard. In our Microsoft Private Cloud Computing book, a good part of the storage fabric chapter is devoted to SMI-S integration in System Center Virtual Machine Manager 2012.
Lower end and non-comparable storage systems such as Dell’s EqualLogic only promise 2x improvement on file copy with ODX. The 7x promise with 3PAR’s ODX capability sounds good but of course we have to verify this in practice. At TechEd we were shown video’s comparing a 10GB regular copy (in about 3 minutes at 80-90MB/sec) versus a 10 second copy with ODX. I’m not sure if Microsoft used NetApp or EMC for the demo.
HP’s P4500 requires SANiQ v10.0 which is finally available and at a minimum will support Windows Server 2012. Many customers who invested in this platform and plan to upgrade to Windows Server 2012 Hyper-V will no doubt be very disappointed to learn that there is no ODX, no SMI-S, no UNMAP, no Dedupe. At least I cannot find any references to these important integrations for Windows Server 2012, Hyper-V and System Center. A 60-day trial for HP StoreVirtual VSA and documentation can be obtained here:
@WorkingHardInIT pointed out that the Dell EqualLogic number for ODX might even be 5x the speed of a regular non-offloaded copy. I based my number on a demo I had seen. In a Dell Techcenter blog on Windows Server 2012 and EqualLogic an increase in copy time of 5x is mentioned. As I stated for the 3PAR number, it is always best to verify these numbers in your own practice.