Powered by System Center
Posts tagged iSCSI
It is only several days since Microsoft made the final bits of System Center 2012 SP1 available, although for a relatively limited audience. If you have a TechNet or MSDN subscription, you have access to SP1 now. All others have to wait till general availability in the course of January 2013.
One of my fellow MVP’s in the Cloud and Datacenter Management department, Graham Davies, raised my interest by claiming there was something in VMM 2012 SP1 that had gone unnoticed: a brand-new SMI-S storage provider for the onboard iSCSI Target Server role in Windows Server 2012! People using the iSCSI Target Server for both demo/test/development and production will get quite excited if they discover how well this software storage solution is integrated, not only in Windows but now also in System Center 2012 SP1.
Back in March 2011, I wrote several blogs about the new Fabric concept in Virtual Machine Manager 2012 which was still in its early beta. One blog focused on deep storage integration using a SNIA standard called Storage Management Initiative Specification (SMI-S). When I was contributing my Fabric chapters for the Microsoft Private Cloud Computing book, there was only a limited number of usually very expensive storage arrays to really dig into this subject. I had access to some HP and NetApp storage to test SMI-S integration which was still very limited at the time. When we saw how the iSCSI Target Server, which was previously a separate install on Windows Server 2008 R2, developed and became included as a role in Windows Server 2012, we begged the product managers responsible for storage in System Center 2012 to also provide SMI-S support. Well guys … here it is!
Fast forwarding about two years, a lot has happened in the storage space and most storage vendors have introduced SMI-S support for their most important storage products. I now consider storage that does not support SMI-S (and some of the other cool technology such as ODX and Unmap/Trim), as a sign that these products are on that vendor’s death list and will soon be obsolete.
With a new operating system around the corner there is an awful lot to learn. The release candidate of Windows Server 2012 that is expected in the first week of June contains a wealth of new features you have most likely already heard about. I have been able to test a large number of very rich features including Hyper-V, File & Storage and Networking. It is very hard to touch everything in a short period.
Therefore I strongly advise you to register for this Windows Server 2012 Jumpstart which is a two day live virtual class presented by Rick Claus and Corey Hynes.
Morning | Beyond Virtualization
• Game changers in the next release of the Hyper-V role on Windows Server 2012
• Massive scale increases, networking improvements, replication and disaster recovery is all in the box
Afternoon | Manageability
• Learn how you can manage a few systems up to a hundred – all from one console
• Server Core installs scaring you off? Learn about all your installation and management options
• Windows PowerShell automation and management at scale – all with built in tools
• Clustering—Cluster-aware updating
• Networking, Network Teaming, network configuration, SMB MultiChannel and RDMA
Morning | Storage
• Learn how Continuous Availability of File Services improves workload reliability and performance
• Storage groups, disk provisioning, iSCSI and SAN integration
Afternoon | Remote Users
• Remote connectivity options for your workforce (DA)
• VDI and Remote Desktop Services deployment and changes
You can find more details here:
At one of our customers site we’ve deployed a Hyper-V cluster (HP Proliant BL460c G7 servers) with two HP (LeftHand) P4000 SAN systems (spread over two locations). We’re using iSCSI for SAN connections. Network adapters are configured through HP VirtualConnect. Each server has 8 NICs (4 from LOM1 and 4 from LOM2):
- 2 adapters in a team for the parent partition
- 2 adapters in a team for the CSV nework
- 2 adapters in a team for the virtual machine network
- 2 adapters for iSCSI connections (teamed through MPIO)
Last week @hypervserver (Carsten Rachfahl) visited me for an interview about topics around Microsoft Virtualization, the Hyper-V community, Hyper-V & System Center (Virtual Machine Manager 2012), Microsoft Software iSCSI target and the choice between Fibre Channel and iSCSI.
Carsten regularly interviews people in the IT industry, but this time it was his first time in English. Although a bit uneasy at first, he did a wonderful job and I never misunderstood his questions.
Before we realized it we had talked for 40 minutes without a break. Although I didn’t know what he was going to ask, they were all well within my field of competence.
Normally I always find it difficult to hear your own voice and see yourself on video. But when I finally convinced my self I had to watch it at least once, I must say I am quite pleased with the result. So here you have a chance of getting to know me a little better and hear about the things I am working on.
I’d like to thank Carsten for a very pleasant and positive experience!
This post is updated after co-blogger Hans Vredevoort contacted some of the storage folks in Microsoft to discuss the MPIO/cluster member initiators issue. It turns out that order list number 3 is not correct.
As a follow-up after releasing the Microsoft iSCSI Software Target 3.3 Microsoft has released an support article specifing the limits.
This topic provides the supported and tested Microsoft iSCSI Software Target 3.3
limits. The following tables display the tested limits and the enforced limits
where applicable.In addition, the following limits apply:
- You should not use network adapter teaming with Microsoft iSCSI Software
Target 3.3 for iSCSI communication.
- If you plan to use multiple network adapters for iSCSI communication, you
should separate them into their own subnets, set up virtual IP addresses, and
then implement MPIO.
- MPIO is
notsupported for iSCSI initiators when used in conjunction with
Microsoft iSCSI Software Target if the initiators are part of a failover
A couple of years ago Microsoft acquired a company called “String Bean Software” which was IMHO a simple but great iSCSI Target solution, and it it was incorporated into the StorageServer product.
Today Microsoft launched the iSCSI Software target 3.3 for public download. This is absolutely great, now we have the possibility to create (Hyper-V) clusters on shared storage!
More info on the Jose Barrato’s Blog and it can be downloaded from the Microsoft download site: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=45105d7f-8c6c-4666-a305-c8189062a0d0
It is not uncommon that hardware vendors have to fix their array controller firmware. In this case it is HP’s P2000 G3 firmware for the FC/iSCSI combined models. Again it is a fix for problems as a result of customers using bigger clusters. Here we have a problem with Windows Server 2008 R2 clusters which surfaces with bigger clusters, long IQN names and a maximum of the space available for SCSI-3 reservations. Haven’t we seen this kind of trouble before. I can assure you that admins of Hyper-V R2 clusters surpassing these limits weren’t all that happy.
Using a long IQN name for iSCSI initiators may lead to issues while using SCSI-3 reservations and Persistent Group Reservations, particularly when trying to create large clusters with multiple paths to each LUN. A predefined area is used to store SCSI-3 reservation keys for each initiator path to each LUN and, when a large number of IQN entries are attempted to be stored, the array can run out of space in this area. This issue is dependent on the number of nodes and the number of paths per node accessing individual LUNs. This issue has been seen during the cluster validation process on Windows 2008 R2 clusters with 16 nodes, where each node had 4 paths to each LUN of the array.
In October 2010 HP published a customer advisory, warning for cluster resource failures in large Microsoft Windows 2008 and R2 clusters using multiple host NICs with HP P4000 SAN and its Device Specific Module (DSM) for MPIO.
If any combination of cluster nodes, MPIO NIC ports and storage nodes resulted in more than 31 iSCSI sessions per volume, these issues would surface. Cluster Validation tests would in fact fail in these configurations. Adding a cluster node or storage node without validation would fail or only partly work.
HP published a firmware update with patch 10085-00 increasing the number of iSCSI sessions from 31 to 64. HP promised to solve this problem in its next major release of P4000 SAN/iQ software.
The formula for calculating the number of iSCSI sessions is:
# of Microsoft cluster nodes *
( # of initiator NICs per cluster node * # of storage nodes)
Now that SAN/iQ 9.0 has been released we can see that HP has followed up on this issue:
A HP Support document released in December 2010 states that with the new release, it has solved problems with SCSI Persistent Group Reservation (PGR) by increasing the limit to 256 iSCSI sessions per volume. This number is high enough to cope with 16 cluster nodes and 8 storage nodes with two iSCSI network adapters. This adds up to 16 x 2 x 8 = 256: so still be careful with bigger configurations.
HP has published a technical whitepaper focusing on the performance characterization of the disk sub-system for HP StorageWorks P4500 21.6TB SAS Multi-site SAN Solution (HP P4500 SAN), addressing questions a customer may have about deploying Microsoft’s Hyper-V R2 virtual machines (VMs) on HP ProLiant BL490c G6 Virtualization Blades (ProLiant BL490c G6) with HP P4500 iSCSI SAN storage device for backend storage.
Target audience: The intended audience includes, but is not limited to, individuals or companies who are interested in the use of Hyper-V R2 virtualization technology for consolidation and migration of servers to ProLiant BL490c G6 servers with HP P4500 SAN storage solutions.
This white paper describes testing performed in April 2010:
Several issues around VSS based protection & recovery are being addressed with this Hyper-V update rollup package. The package can be downloaded from:
The last three issues might sound familiar if you already perform host level protection of virtual machines on Cluster Shared Volumes (CSV) with Microsoft System Center Data Protection Manager 2010.
Issues that are fixed in this update rollup package
Consider the following scenario:
- Some Internet SCSI (iSCSI) connections are created in a virtual machine that is running Windows Server 2003.
- You back up this virtual machine on the virtual machine host server.
In this scenario, the error code 0x800423f4 occurs when you back up the virtual machine. Additionally, the following event is logged into the Hyper-V Virtual Machine Management Service event log:
The number of reverted volumes does not match the number of volumes in the snapshot set for virtual machine "’virtual machine name’ (Virtual machine ID <GUID>)".
Cause of Issue 1
When a virtual machine is being backed up, the VSS writer of the server that is running Hyper-V makes a call to the guest virtual machine to check whether any iSCSI connections exists. This call has a default time-out of 60 seconds. If this call does not return within the time limitation, the VSS writer of the server that is running Hyper-V incorrectly assumes that there is no iSCSI connection. Therefore, the backup operation fails.
Consider the following scenario:
- Cluster shared volumes are enabled on a failover cluster for Hyper-V.
- Some virtual machines are saved on the same volume. But they are running on different nodes.
- These virtual machines are backed up in parallel.
In this scenario, the virtual machine backup operation fails.
Cause of Issue 2
When the virtual machines on different nodes are backed up in parallel, every node waits to become the cluster shared volume owner to create the snapshots. However, the Cluster service moves the volume owner from one node to another node immediately after a snapshot is created without waiting for post-snapshot tasks to be completed. If another node requests the same shared volume for a backup operation before the post-snapshot tasks are completed, the Cluster service changes the volume to another node. Therefore, the VSS writer that is in the previous node cannot find the cluster shared volume locally when it performs post-snapshot tasks. This behavior causes the virtual machine backup operation to fail.
Consider the following scenario:
- A virtual machine is being backed up on a server that is running Hyper-V.
- At the same time, an application backup operation is being performed in the same virtual machine.
In this scenario, some data is truncated from the application backup in the virtual machine. Therefore, this behavior causes data loss.
Cause of Issue 3
The application backup operation in the virtual machine is incorrectly affected by the virtual machine backup operation on the server that is running Hyper-V.
Consider the following scenario:
- A virtual machine that has some snapshots is backed up on a server that is running Hyper-V.
- Then, this virtual machine is restored to another location.
In this scenario, the restore operation fails and the virtual machine may be corrupted.
Cause of Issue 4
The snapshot files are not restored successfully when you restore the virtual machine.
Full details: http://support.microsoft.com/kb/975354